There's no doubt that perfectly drinkable beer can be made without a secondary. There's also no doubt, to me, that that same beer would have been better had it been racked to a secondary in a timely fashion.
Autolysis, the off-flavor generated by spent yeast essentially decomposing in the fermenter, is really not the biggie here, though it's often talked about. Most of us don't leave our beers in the fermenter long enough for that to matter. So, yeast contact isn't the issue to me.
Nor is lack of clarity caused by suspended particles and yeast. Sure, that stuff settles out in the end, and more importantly, it's not the primary cause of haze.
The biggger issue is prolonged contact with the trub in the primary. We all know that it's best to leave as much of the hot break as possible behind in the kettle. That's why people use Irish moss or whirlpools. The same is true of cold break and any of the protein trub in the wort.
The conversion of sugar to alcohol and CO2 happens pretty quickly in the first few days in the primary for most beers. Much of the later secondary fermentation involves the yeast metabolizing off flavors like diacetyl. Physically removing as much of the off-flavor producing proteins (trub) makes this phase of the yeast's job easier.
It's clear once you brew a few batches that batch aging the beer for a few weeks in the secondary (or tertiary) fermenter makes better tasting beer than if you bottle at the first possible moment. That's because of these secondary roles of the yeast in metabolizing off flavors generated as byproducts of fermentation. I'm by no means an expert on the chemistry of it all, but by removing the trub from the fermenting beer, you physically reduce the amount of off flavors the yeast must metabolize.
To me, it has little to do with clarity or suspended particles, though I do think you get clearer beer if you use a secondary. In any event, clarity of homebrew is usually dictated not by suspended particles, but by things like chill haze. You'll also reduce chill haze by removing the beer from the trub. But the flavor advantages come from complex chemical reactions that have to do with the later phases of yeast metabolism.
Personally, I can always taste the advantages of using a secondary and batch aging an appropriate amount of time. Racking to a secondary within a week is one of the top five things you can do to improve the flavor of your beer, IMO.
Cheers!
