Mr. Beer was what I started with. Its good for around $50 dollars, but you'll hardly be exposed to ALL that is home brewing. If home brewing were baseball, Mr. Beer would be a soft bat and nerf ball you give to a 2-year old for Christmas...
For example: The Mr. Beer setup is basically a fermenter, bottles, extract, and directions. The directions mention nothing about certain equipment that makes your beer be all it can be. It doesn't mention wort chilling, aeration, steeping grains, gravity readings, etc. In a nutshell it is basically the lowest denominator of home brewing - thats the con of it... you really can't brew a wide range of nice beers. I started with it and it definitely served as a gateway, but I brewed with it for 2 years thinking I was awesome. Then I was ready to step up to better equipment and felt like I was back at the beginning because I had no idea what certain equipment was. I read books, came to this forum, and basically started all over again trying to gain knowledge so I could start with 5 gallon batches. Don't start with Mr. Beer unless you live in a TINY TINY apartment and don't have a goal of sharing with your friends. You'll drink it up way to fast.
Get a 5 gallon starter kit like
hypergolic recommends. You'll get a fermenting bucket and/or plastic carboy, extract, bottles, capper, directions, etc. Starting this way will enable you to start brewing a wider range of beers, and even making clones of your favorite commercial beers, while adding a good twist of your own to them. Also,
get this book, and learn the ways and terminology of everything. I really wish I new of this book when I first started out.
So anyway, definitely start out with a good 5 gallon kit, tons of threads here recommend which company or website to go through, and you may even get lucky and find a coupon code from a vendor here. With your first batch, start simple and easy with a nice session/summer beer. An American Pale or simple wheat beer may get your gears turning, then move up from there. Or you can even start out with a recipe based on your favorite commercial beers. Tons of sites out there are dedicated to recipes, and you can buy the ingredients online or at your nearest homebrew store if you have one.
Expect the addiction to kick in at the first chilled taste of your first brew. At this point, get a second or third fermenter, and start brewing beers at the same time... I promise you'll thank me!