Like a lot of things, buy in bulk, save the most.
I use hop pellets, I store them in my freezer. Same with yeast in packets. My grain (I do all-grain) I store either in tupperware-type containers if I have a pound or two, ice-cream buckets for up to 5-pounds, and food-safe 5-gallon buckets for sacks of grain (you can get about 25 pounds of barley malt in a 5-gallon bucket). I crush them in my own mill.
I use Gamma Lids to seal the 5-gallon buckets. They're the spin-on lids w/ gaskets to make them airtight. (I lucked into a deal on them a few years back for $2.95 each--bought a dozen. Normally they're like $8).
As noted above, shipping costs matter. Morebeer has free shipping for $59 or more orders, exception sacks of grain. Sometimes places like Northern Brewer have percent-off sales; you may be able to offset the cost of shipping with that.
And depending on where you live and if a supplier does local pickup, you may be able to eliminate shipping entirely. I buy a lot from RiteBrew; they are maybe 3 hours from me, and when I'm going past them I can pick up an order.