I highly recommend reading a lot of info before starting. It really helps, I promise. I spent about a month reading and reading before buying anything. Personally I found it fun to learn "all" the terms and how brewing works. It helps for confidence. Last thing you want is to start your kit and then realize you don't have any clue if you should cover your pot, or if you should put your priming sugar in with the boil (don't do either!)
As for places to get your ingredients, here are a couple that I have ordered from.
www.midwestsupplies.com (got my brew kit from there)
www.northernbrewer.com (I buy my ingredients from there)
www.austinhomebrew.com (haven't ever bought anything, but heard it's great)
Have you ever gone down to a local brew store? Usually the folks there are really helpful. And if you happen to make good friends, you might be able to convince them to let you do a brew with their equipment. Buy them some delivery pizza or something just to be nice. I am helping out a new brewer in the next week, using my equipment.