Some of this depends on where you live and what access you might have to sources of ingredients.
I'm about 3 hours from RiteBrew. They have excellent prices on bulk ingredients; 25 or 50 pounds of 2-row is 76 cents per pound; Maris Otter, my fave malt, for 99 cents a pound for a 55-pound sack. Hops by the pound, yeast prices that , at least in dry, are very good.
They have a pickup option that saves a lot of shipping money; only problem is they're not on the way to anywhere I tend to go, and I'm not driving 6 hours to save on shipping. But, I have a buddy who travels up that way from time to time; he picks up stuff for me. I reciprocate with a six pack of something nice, and we're all good.
The key is buying in bulk, as others have noted. You need the containers and space to be able to store grain bought in 50- or 55-pound sacks. I haven't bought hops by the pound, but i'm close to doing that. Just need a way to flush opened packages of O2 before storing in the freezer.
I don't tend to use a lot of hops; 4 ounces is about the most I use. But even then, Ritebrew is stunningly good. My buddy is heading up there next week and I'm putting in an order for pickup. I see they have Northern Brewer hops with high Alpha Acid content, something I want. I'll get some of those, a sack of 2-row, a sack of Maris Otter, probably some other hops and maybe a few dry yeast packages.
***************
If you buy in small lots, everything costs more. I just brewed a California Common, 11# of 2-row, 3 ounces Northern Brewer hops, a Smack Pack of Wyeast 2112. Since I'm getting 2-row for 76 cents/pound, an 11-pound grain bill costs....$8.36. Hops bought at Ritebrew are $1.39/ounce, or $4.17. The yeast is $6. Grand total is $18.53. Add in $1 for PBW for cleanup, maybe 25 cents worth of Star-San....I'm just under $20 not including cost of propane (couple bucks, more or less).
I buy in 50-pound sacks. The trick is to find someplace you or a buddy can get to in order to save shipping costs. Maybe there's nothing nearby, but ask your LHBS if you can buy in bulk. If not, look for sales when you can stock up.
Even Morebeer is funny that way. You can't buy sacks of grain with their free shipping option, but you can buy smaller amounts and get that free shipping. Maybe that would help you out a bit.
***************
I use 5-gallon buckets for grain storage, and gamma-seal lids on them to maintain an airtight seal. I even made a rolling set of shelves to accommodate them: