I do see bottling as a good way to at least start off, if you want to keep the costs lower. Most of the costs for kegging includes the keezer (and it's components) as well as the CO2 tank(s), corny kegs (they're starting to get more expensive) and all the fittings to go along with them. That can end up as a large chunk of change to shell out... Granted, once you have most of that, the cost to maintain is low. But, the same can be said for bottling.
I use Grolsch and Belgian style bottles. Which means that my cost to bottle each batch, once I've purchased the bottles, is rather low. You can typically go 5-6 fillings before you replace the gaskets on the Grolsch bottles (less than 10 cents each if you buy them in 100 packs, so cheaper than caps)... Belgian bottle corks are also pretty cheap (not as cheap as the gaskets though)... I use Belgians for my bigger brews, with Grolsch for pretty much everything else. I was destined to get a floor corker at some point (I have mead in process that will get bottled eventually) so I did that when I got my first round of Belgian bottles. For a full 5 gallon batch, I use less than 36 Grolsch bottles, and under 24 Belgian bottles.
Sure, it takes time to bottle, but having a bottle to pop open is part of the enjoyment of home brewing.
Eventually I do see myself kegging part of a batch. I plan to get 3 gallon corny kegs, and bottle the balance of most batches. I'm sure I'll eventually get some 5 gallon corny kegs to fill, if I'm NOT planning on sharing a brew with anyone else, or take it with me to anyplace. Not to mention, 3 gallon corny kegs are pretty damned portable.
I know of one person that has a backpack setup with a CO2 cylinder in it, as well as a 3 gallon corny, and he dispenses from a picnic tap... The tap is the only thing visible of the entire assembly. Very cool...
So, basically, there are plenty of choices for keeping costs down. It's just a matter of what you're comfortable with... Brew partial mashes and your cost per batch will be lower than extract. All grain will be the cheapest of all there. For dispensing the brew, that's something you'll need to figure out on a more personal level. If you NEED to keep it as cheap as possible, then bottling will probably be your better choice. Unless you happen upon some of the kegging parts for really cheap money (or free)...