Yup, make a starter, and you will find out for sure. My bet is that its still okay. I read that liquid yeasts are able to survive up to 110 degrees. (Im at work so I cant show my math...lol) So your brief time unrefrigerated should be okay. Not preferred, but okay.
A cool idea (pardon the pun) if you buy yeast and do not go home right away, is to get a thermos that a tube of yeast will fit into easily, add an ice cube or two, add some water, and viola! You are set. Drop in the tube and you have plenty of time.