My first (and only) brew had some off flavors (bananna) and I'm pretty sure it was caused I pitched my yeast too soon. I've read (may not be right) but just cooling it naturally is okay but I plan on using the big snow bank in my front yard to get to 65ish.
It does matter thought, but things I've read vary on how long it takes and what impact it makes, obviously the quicker the better to prevent other **** from gaining a hold in ur brew
Brewsume
Muntons Nut Brown - started December 15th, 2013, bottled January 3rd, 2014 - Mild off flavors suspected from high pitching temp, overall, a good first batch!
Muntons Irish Style Stout, brew day March 9th, 2014