I make my starters with DME and step them up with un-fermented wort I make in a 3 gallon batch. I'll save most of the wort wort in sterile mason jars. I sterilize them in the oven at a slow ramp to 200 degrees then pour in the boiling wort and cap them. After letting the wort settle I always get black/gray sediment in the un-fermented wort. Once it goes in the starter though, I get nice white yeast cakes when I turn of the stir plate and let the yeast settle. Every now and then I get a jar that starts fermenting on its own and causes pressure build up in the mason jar. I don't use those, even though they smell fine. They will have a yeast cake settled on the bottom of them. I bet it is from previous beers I have made. I tend to purposefully let my beer yeast "contaminate" my garage to keep out the bugs. All my beers are so far so good. My house ale stain was cultured from a sierra Nevada bottle. My house lager strain was cultured from a $1 expired american lager strain.
In short, don't worry about it unless it smells bad, and by all means, use temperature control during fermentation to boost the bug you want.
