Hi, and welcome!
I'm a bit new here, too -- just brewed my first batch today -- but I've been reading everything I can find, so I'll give it a shot.
(Oh, while I'm thinking about it, you may have a bit more luck asking in the beginners' board. Just a thought.)
The answer depends on a couple things. First, does your water taste okay? If you wouldn't drink it straight from the tap, then you don't want it in your beer. Second, are you brewing extract or all grain? Extract is more forgiving of water makeup, whereas all grain depends on certain minerals/ions being in the water to produce the type of beer you're wanting to brew. Third, your tap water has chlorine or chloramine in it; you'll want to boil it for a while to get that out (or filter through activated charcoal). Grocery store water is more expensive (Walmart sells it for 88¢/gal), but you don't have to boil it or treat it.
I bet if you do a forum search on the subject you'll come up with a bunch of stuff that's more intellegent than what I've given you.