Good point, and one I have considered many times (also, very appropriate given your TB handle!).
Edit: feel free to take or leave this post-I'm happy with my results but I am far from a scientist.
Seems counter-intuitive, yes, but some very experienced homebrewers as well as people in the industry have explained it to me like this; when you do flame out and whirlpool additions, you are releasing a huge number of volatile oils into suspension. Since the hop oils do not go through isomerization, rapid chilling is important to lock the hop oils into the beer since they wont naturally bond. Sure, some hop oils probably lock onto the coagulated proteins that drop out, but plenty lock on to stuff that sticks around in the beer, too. My beers end up hazy as all hell, despite the efforts I put into cooling (I made a rib cage double chiller that I pump ice water through in a closed circuit). I was told by a professional that I really respect that using a heat-exchanger or something of the like to get temps down as quickly as possible after a nice whirlpool is important to lock in flavor.
Also, I am using oats, flaked wheat, Marris Otter and Carapils. There is plenty of sticky and rich stuff in there where I am not too worried about losing some haze.