I usually leave it out for 3-4 hours before I make my starters. I've never noticed a problem but I've also never done 24 hours so I don't really have a comparison. I would think you'd be safe going with the manufacturers suggestion though.
The thing is. I've already taken it out of the fridge and it's been at room temp for 4 hours now. Will it shock the yeast or anything if I put it back in the fridge and then follow pull it back out before brewing tomorrow?
I left one out for about 18 hours once (had one or two too many while brewing) it looked "clumpy" but was fine... as far as re-chilling I usually get 3-5 batches from each vial by refilling it from the starter, chilling and using that to pitch the next starter.