Well I took the plunge and ordered some stuff from brewbyyou.com. Their prices were fantastic, almost too good to be true, but I figured I'd give them a shot. Overall I'm pleased.
I placed my order on a friday, and it arrived the next thursday. That's pretty quick for a small web shop. In contrast I placed an order from listermann's at the same time and it didn't arrive until a week from the following monday (10 days) and they're only an hour away from me.
My brewbyyou order was for 12 lbs of light extract, 3 lbs of wheat extract, 4 oz cascade hops, 1 oz of 4 other hop varieties, 5 lbs of crystal 60 malt, 1 vial of american ale yeast and an auto siphon. Total on this order came to $50.
The down side? Well, everything came the following thursday, except the hops. No hops were anywhere in the box... I gave them a call and they said they remembered it being packaged up, but if it didn't make it in the box, they'd resend them. This was on friday, and the hops arrived the following tuesday, which is pretty darn quick.
I'll consider the hops an honest mistake, since they quickly sent them out and were very nice about it, that's all I can ask for. Prices for bulk ingredients are really good so overall I'm pleased. Ingredient quality was good for everything I ordered. The only questionable thing was the yeast, which was EasYeast brand, but it seemed to work fine and from what I've found with some google searching, people who have used it haven't noticed any difference between it and white labs/wyeast. It was also half the price...