I've heard that
Roman's Pub is amazing, although I've never been there.
LOVE that bar. There is another bar close by to Roman's, I think it's called Palm Tavern.
If I'm downtown and don't want to drive, I go to Milwaukee Ale House. The beer is pretty darn good and the food is really good. It's walking distance downtown. Water St. Brewery used to be pretty good, but lately the beer just isn't the same. I'd give it a "fair" rating at best.
Across the street from Water Street Brewery is a bar called Bar Louie. Sometimes they have great beer on tap, and the happy hour is like 4-9PM with $3 pints of great beer. Or at least it was, the last time I was there. Lakefront IPA is my all-time favorite Milwaukee beer, so make sure you get some!
The Lakefront tour is great.
Discount Liquor (I think it's on Oklahoma, but please google it and check!) has great prices and selection. There's another place in Menominee Falls that Sause works at, but it's far from downtown so it depends on where you're staying.
For bottles, grab New Glarus Belgian Red, New Glarus Unplugged, or just about anything from New Glarus actually! Lakefront has some wonderful beers, and I love their Cream City pale ale, and of course the IPA. Capital Brewery's Maibock is very good and their Blonde Dopplebock is fantastic! (the rest of their beers are "meh"), Tyranea has some great beers.