I'm not buying the infection thing either. I made a batch of apfelwein and used my Mr. Beer. I forgot about it and it's been sitting in there for over a year and isn't infected. If beer is infected, you are going to know it.
There are many things, besides infection, that can cause off flavors. One thing that no one has mentioned is the age of the extract. I noticed when I did Mr. Beer in college, that the extract can be quite old sometimes. It is always best to get the freshest extract you can. That's why so many people order from Midwest, AHB and the other big online stores. They go through so much extract, you are guaranteed to get fresh extract.
I think the age of the extract is a lot of it. My Mr. Beer batches always had at least a touch of this soy sauce flavor to them. Even Mr. Beer batches I made after I was making very drinkable five gallon extract batches still had the taste. I think getting a really fresh can is key. I have since used the Mr. Beer fermenter to make my one extract recipes and they turn out just fine.