I got the beginner kit from Things Beer through Michigan Brewing Co. along with 1 extract kit and yeast. This kit came with a 6.5 gal plastic primary bucket, a 6.5 gal plastic bottling bucket, a 5 gal glass carboy, caps, plastic funnel, bottle brush, bottle capper, bottle filler, 5ft siphon hose, sticker thermometer. I also went to Wal Mart and got a 5.5 gal pot to brew in.
I would start buying six packs/twelve packs/cases of beer to get 45-50 empty bottles. Trust me, this is more fun the buying empties.
Also, buy some 2 or 3 1 gal bottled water, it's easier than having to boil and sanitize the tap water.