Papazian specifically advises against milling your grain where your
fermenters are, not just where you brew. The OPs quote of Papazian is misleading. The rest what you say is correct; anything that touches the cooled wort must indeed be cleaned and sanitized before the wort contacts it.
To the OP: Perhaps I'm a rogue zymurgist, but I mill my grain and ferment in my basement brewshop and haven't had to contend with any wayward infections from Lactobacillus or other beer spoilage organisms. Remember as well, we aren't brewing in a sterile environment, merely a sanitary one. A little common sense goes a long way. I'm not milling grain while I'm racking to a fermenter, I cover my sanitized fermenters (a sanitized plastic sandwich baggie or bit of sanitized foil over the mouth of the carboy/BB) if I'm not ready to rack to them, etc. If you are diligent in your sanitation and exercise common sense, you needn't worry in my experience.
In the podcast referenced, a brewer recounts a visit to an old British brewery where the grain was being milled with no separation between the grain station and the open fermenters.
There are a couple of things at work here that makes it possible: *
(1) Large amounts of
vigorous,
healthy yeast. Brewer's yeast are voracious little buggers, and will definitely outcompete any wayward bacteria that might find their way in. Underpitching negates this.
(2) Relatively quick consumption of the ale. Most British ales are, by design, low-gravity session beers made for quick consumption. Brewed, packaged and served within a fortnight. Spoilage organisms take quite a while to assert themselves and make themselves known.
* Disclaimer: this is a theoretical discussion. If you attempt this, you do so at your own risk.