Do you decant your starter before pitching into the fermenter? I do personally but didn't know if maybe that might be giving some of the "off" flavors you're experiencing? I ran an experiment with this before throwing a 2 liter starter (spent wort from the stir plate and all) of w34/70 into 5 gallons of a 12 ibu beer. I poured in all 2 liters and did not notice any off flavors. Pulled in some head brewers that verified this as well. Now this wasn't frozen yeast though so maybe that's the difference?
It's unclear to me if previously frozen yeast verses yeast from the fridge (or fresh from the manufacturer) makes a difference. But I try to decant as much as feasible. For lagers I always try to decant as much as
possible and still have enough liquid to resuspend the settled cake. That is to say, if I have a 300 ml cake I'll leave enough supernatant make a 500 ml slurry. For ales I'm not that persnickety, and often I won't wait for all the yeast to completely settle out and will pitch more liquid if I'm pressed for time. Complete settling of the yeast can take up to a couple of weeks, not unlike cold crashing, which is exactly what settling in the refrigerator is. Unless I've
really planned ahead I won't have that much time to wait.
The reason I decant (especially for lagers) is to avoid any off flavors from the spent wort. Somewhere in my reading on yeast propagation I came across a mention that if any more than about 5% of the total volume in the fermenter was supernatant from a propagation, it would begin to affect the flavor of the finished beer. That would be roughly 1L of liquid in a 5 gallon final batch (1/20th), or about 5%. For a 2L starter in a lager it would be 10%.
If you think about it, ales are fermented in the 62F-75F range, with higher temperatures resulting in more esters, phenols, and occasional fusels. They're sometimes favorable, but often are not and detract from the finished beer. Yeast propagation usually is performed on a stir plate at room temperature (say 75F) under conditions that already stress the yeast cells resulting in off flavors, or at least atypical flavors, in the spent wort. The delta between yeast propagation temperatures and fermentation temperatures for lagers is even greater, resulting in flavors that are generally unacceptable for a well-crafted lager, plus there's twice as much volume (2L) trying to spoil my perfect beer that I struggled mightily to perfect. The least I can do for my precious wort is to show it a little respect and decant the nasty supernatant!
As far as professional brewers not worrying about spent wort affecting their beer, consider the volumes they are using. Pitching 5 gallons from a yeast brink into a five barrel fermenter would yield a 31:1 ratio of yeast to wort, or less than 1/6 the threshold amount that allegedly starts to affect the finished beer. I guess it's all a matter of perception as well as perspective. If I can taste it (or even if I only
think that maybe, possibly I can taste it), it's worth it to me to take a few extra days to decant between step-ups in a starter. By the way, I basically do the same procedure for propagating yeast, regardless of whether it's a frozen sample from the freezer, a harvested sample stored in the fridge for a few months, or a fresh pitch from the LHBS that I'm building from scratch. The only difference with frozen samples is the slow thawing and matching the temperatures of the sample and the initial 125 ml viability starter wort before mixing and putting on the stir plate.