Yes, allow the beer to stay in the primary fermenter for 3 to 4 weeks, the yeast will clean up some of their byproducts, and the beer will taste better.
Also, after you bottle (or keg) make sure you give it enough time to condition, most new brewers drink their beers far too young, and they have a green flavor, which is more than likely what you are describing.
Trust me I know from experinece, I am still a newer brewer, but after finding this forum, and with over 40 gallons under my belt, my beers have been improving greatly from batch to batch.
Patience is the key, I know waiting is the hardest thing to do, especially when you are very excited about your first beer, but giving it some extra time and TLC will benefit you!
Brew on!
