How cheap is cheap?
Your basic kegging supplies will run you approx $150 for a keg, small tank, regulator, hoses connections etc. Also, I've found that it's cheaper overall to get a large tank vs a small tank. For example, if a 20 oz paintball tank runs you 20 bucks, fine, but you probably don't want to get a lot of them, plus you need an adapter for $15. It's cheap.. however if you get a 5 pound reconditioned tank, you'll pay $55 which is under what you'd pay for 4 20 oz tanks ($80+15). A 10 pound reconditioned tank will only cost $75 though, vs the $110 for 2 5 pounders, or the $160+15 for 8@20oz tanks and so on. Plus, a reconditioned 20 pound tank may go for $100, so just compare that to 16-20oz tanks at $335.
If you get a paintball tank free from a friend or relative who doesn't play paintball any more, or you play paintball then that'd be different.
You would want to keep your keg in the fridge to be the least expensive, or even in a cellar to be even less expensive, though not as cold. If you want to use a chest freezer, you *have* to get a temperature control device to "turn it up above 32" since freezers generally don't have that option. $60-$85 from dial to digital.
Not so bad if you already have a chest freezer to use though, otherwise add that into the cost.
Definately go with picnic taps those really have no price comparision and you can even use just one for a while and then move up to a full set of full taps one by one. (Example, I had planned on my freezer w/temp control, then eventually adding a collar but still using one single picnic tap and switch it between kegs whenever I needed a different beer. Then I planned on getting one faucet at a time, so I'd drill one hole, use the faucet, and later on get a second faucet and so on. That kinda went out the window when I got 4 of them on sale.... but it helps not to have all the cost at once.)