I've only bought base malts in bulk through home brew club group buys, but every time I go to the LHBS I pick up 5lbs of some uncrushed specialty/base malt beyond the ones I have in stock. I hit up the local dollar store for 1 gallon airtight containers which I label.
If there is something I need to order from an online retailer, I'll buy up at least a couple of every variety of dry yeast that I have a use for. I don't have anything against liquid, of course, except that the viability isn't long enough for my liking.
My goal is to have enough variety of grains and yeast on hand to make whatever I feel like should the mood strike since the LHBS keeps daytime hours and is at least an hour out of my work day. Since I've done this I've lowed my average (5g) batch cost to about $10 compared to 20 - 30 when I was buying a batch at a time online or at the LHBS.