From what I've been seeing, there's nothing wrong with starting small. Even 1 gallon extract batches are pretty useful. Anything you buy for a small batch will certainly be just as useful when you're making 5 gallon batches and all grain batches.
I just moved up from 1 gallon batches and the only losses I can think of would be the plastic water bottle jugs I used for a few ciders and one batch of beer. (And that's cause I just didn't feel like keeping 10 of them around.) Even if you get a 1 gallon growler to ferment in, nothing wrong with using that to transport beer from a keg later on. (My 1 gallon pot was a plain cooking stockpot to begin with.)
More than likely, you'll be doing 5 gallon batches, so whatever pot/keg you use will be as useful to you for extract, as it is for doing all grain. Same thing with a burner if you do gas. (You'll probably use that burner for frying things and even cooking too.) You can convert those kegs, but you can also consider using them as fermenters I think.
If you find some sort of great deal on the whole 3 tier system, kegs, 20 pound tanks and so on, sure (you know, if someone's giving it to you for free), but if they're full price, there's really not that much of a need to hurry and get those. You're not going to waste anything by starting with extract, plus you'll get to practice, not worry about mashing, protein rests and so on.
I can't think of one thing you wouldn't use from a kit that you wouldn't still use. With maybe the exception of the fermenting bucket *if* you're using a keg to ferment somehow. And maybe a racking cane/siphon *if* you're pushing the beer from keg to keg with co2. And that'd be pretty minor in cash anyways.
I had to buy everything separate because I *can't* use the kit (or I'd have to toss or give away the malt extract and special grains if I did, cause I'm allergic) and it kind of annoyed me because I had to buy everything that was in a kit. Even as I mention this, I think it could have been cheaper had I done that instead of buying everything separate.
I'm touching on partial mashes or at least using specialty grains now, and I'm using the 1 gallon pot for that. And on doing a batch recently, I used too much water since I didn't consider the water from the partial mash/sparge and used my 1 gallon growler for the extra fermenter along with the 5 gallon keg. I still expect to use my 1 gallon jugs for smaller test batches too. Eventually I'd like to go with more grain, and eventually I may get a MLT, but everything else is still going to be used.