Start out with a brew kit from any reputable homebrew store. These vary in price from $35 to $150 depending on stuff included. To begin with you need at least a fermenter, 6.5 gallons, a secondary fermenter 5 gallons, bottles, caps, stir spoon, sanitizer, boil pot (Canning kettle of appropriate size will do, stainless steel is the ultimate product. I wouldn't mess with aluminum. It WON'T give you Alzheimer's disease (Thoroughly discredited homebrew myth) but they are a pain in the ass to maintain. Canning kettles are delicate. The enamel quickly chips. They have to be kept intact (You can patch them) Or your beer will develop an iron flavor. Stainless is worth the cost, virually all homebrewers will eventally get one if they stay with it long enough, but a canning kettle will do to begin with if you want to save money. One very important brew itrem is a good text. Homebrewing for Dummies by Marty Nachel is excellent. Dave Miller's Homebrewing guide is good. The new Complete book of Homebrewing by Charlie Papazian is out of date and has MANY inaccuracies. Homebrewing Volume One by Al Korazonas is superb but may be a bit indigestable for a newbie. You will also need a syphon hose setup, access to brewing kits or ingredients, a bottle capper, hydrometer, a thermometer, airlocks for the fermenters, rubber stoppers for the airlocks, and a sampling tube for the hydrometer,. I would say at least one carboy is a good idea. You can ferment in plastic, but glass is better for a number of reasons. A lot of this stuff will be part of the kit, but all of it may not.
A word about bottling.
It's a pain in the ass.
It is probably the one thing that causes more people to give up brewing than any other thing except perhaps infections caused by careless sanitation.
I believe you should move to kegging as soon as possible. THIS is where it starts getting spendy. $150 for the CO2 bottle, Cornie Kegs, hoses, fittings and tapper. Then you find you need a fridge to have cold draft beer on tap... Yada Yada Yada... It never ends. But There is NOTHING like fresh draft beer on tap for sheer panache! For another five bucks, you can make a counter-pressure bottler to produce bottle beer you can actually drink out of the bottle (Filled from the keg, it won't have the sediment you get in bottle conditioned beer.)
Then there is the matter of going whole grain... But that's beyond the scope of your question, so I won't tackle it here. Suffice it to say, All grain gives you TOTAL control over the beer you make, opens the door to experimentation and takes you out of the mercies of someone else's idea of what good beer is. It is also an additonal expense, since you will need a heat source (Kitchen stoves take FOREVER) like a turkey or fish fryer, a 20 lb. LP gas tank, a mash tun... and the list goes on...
That's the beauty of this hobby. I know people who have made nothing but extract-only brews for years and are totally happy with that. Others like me, have become priests in the church of beer, investing hundreds and even thousands of dollars (And Hours) it this avocation. Where you go is up to you. Start small, and grow into the hobby is my recommendation.