I am just starting to do kegs and so far it seems very easy. There is a little price at first but not that bad if you look around. I found 1 keg at the dump and got a 5lb CO2 can from a friend. I did look around at prices since eventually I will have to give the CO2 can back. At my local HBS there is a 15lb CO2 can with gauges and hoses for $110 and corny kegs used they sell for $25.
As far as space I don't see a fridge taking up more than bottles. I think the fridge is actually a space saver in the end. I bought a little fridge that will hold 2 kegs with the CO2 bottle about 3' high. I just look at the space all the bottles would take up between what was full of beer, what was empty and storing them. Now with the fridge, the kegs are inside and the next batch is on top fermenting.
My first batch to be kegged will be done tonight, I am excited.