Should be fine. I have done it before with no ill results.
I do this somewhat often actually when it's 100 degrees out and I forget to buy ice for the pre chiller. There's no way I can cool the wort with my immersion chiller to proper pitching temp when the water temp is close to 100. I just cool the wort to around 100 and throw the fermenter in my beer fridge at 65 degrees, then pitch in the morning. No problem.