"How to Brew" by John Palmer is available free online. Somebody has probably already mentioned this. It's a great, great source.
My advice is definitely not going to be the cheapest route, but in the long run I think you'll be saving money.
Decent water: if your tap water doesn't make you gag when you drink it, your tap water should be fine. If it does make you gag, then buy some drinking water in gallon jugs.
Start with either a pale ale extract kit or a pale ale extract kit that has specialty grains. Extract kits with specialty grains are not much harder to do than a simple extract kit, and you'll get a better tasting beer from a kit with the specialty grains.
Wyeast yeast smack packs are great. They make life easier, especially for new brewers. You can mess around with dried yeast and yeast starter kits if you like. I wouldn't recommend it.
Northern Brewer is a good place to buy kits and yeast. They have very comprehensive instructions included in their kits.
Probably the most expensive thing you're going to buy starting out of the gate is the brew kettle. Brew kettles can be prohibitively expensive. If I could do it all over again, I would have avoided buying a standard pot. They're usually way too small or way too expensive. The best bang for your buck is to get your hands on an empty keg and have someone cut a big hole in the top of it. Then buy a "kettle conversion kit" from places like Rebel Brewer and get someone to pop a 7/8" hole in the side of it with a step drill bit to install the spigot. Review a few videos on YouTube about "keggle conversion" to get an understanding of what you're shooting for. You'll also need to fashion a dip tube out of copper tubing. Working with copper tubing and solder is not hard, and since the tubing is not going to be under pressure, you are not going to need to make perfect solders.
Invest in a turkey fryer burner and brew outside or on the back deck or in the garage.
I also would have avoided using siphoning to get the wort from the kettle to the fermenter. Just get the kettle conversion kit and be done with it.
Don't screw around with cheap hoses. You'll waste a lot of money and time. Just make a good investment now and get some quality high-temp silicone hoses (also available at Rebel Brewer). You probably won't need more than eight feet of 1/2" ID high-temp silicone tubing ever. It's only about $2.30 per foot.
You're going to need a good dependable thermometer. You can buy dependable digital stick thermometers (CDN DTQ450X ProAccurate Quick-Read Thermometer) from Amazon for about 11 bucks. You'll also need a hydromometer and hydrometer tube to take the readings in. There's a thing called a wine thief that lets you suck up a sample and lets you drop the hydrometer into the wine thief to take a reading. Then you can drain the test sample into a glass to take a taste test. Mmm! Flat beer! But oh so tasty.
Chilling the wort is another big consideration. If I could do it all over again I would have skipped the $100 counterflow chiller that clogs with the slightest amount of hop material and instead would have spent $40 on making a counterflow chiller from a garden hose and some 3/8" copper tubing that is a lot harder to clog. There are many videos on how to make these online as well.
I don't think I'd waste any money on carboys and instead I would have invested my money in corny kegs to use as fermenters AND as kegs for the finished product, once I got into kegging. But for your first few batches you can go really cheap and just get a food-grade bucket for fermenting and a bottling bucket for bottling.
Sanitizers: Star-San is the way to go. Just be sure to use Star-San with distilled water. It will last a lot longer. Don't fear the foam! Read up on a number of threads in here about Star-San.
Let your beer stay in the fermentation chamber for a long time. Your beer can stay in there for a ridiculously long time before you bottle it, and while it's in there, the better tasting and clearer your beer will be. Search these threads for examples of how long people let their beer sit in their fermenters before they bottle or keg. Your patience will be rewarded.
While at the local home brew store, check and see if they have any swing-top bottles. While not a necessity, they are a very nice luxury. Much better than capping. A lot of people worry about never getting the bottles back from friends and family when giving away homebrew beer, but surprisingly, 98% of our bottles come back to us (with requests for more beer, of course).
But most importantly, relax. It's surprisingly hard to really screw up a home brew. And after everything is said and done, YOU HAVE MADE BEER. It probably won't be as good as an $8 craft brew you can buy in a store, but it will beat the pants off of any regular domestic beer you will ever have. Being able to make your own beer is right up there with making a human being in the Importance of Humanity Scale.
And find a friend or two to brew with. Unless you like drinking alone and doing a crapload of dishes alone. I don't.