It'll be fine. Try to pitch the yeast when they are close to the same temperature, as to not shock the yeast. If they yeast is warmer from being at room temperature, or being rehydrated with warm water, you can add a spoonful of the wort to the starter/rehydrated yeast and stir, and do that over a period of time to "equalize" the temperature. Then when they are roughly the same temperature, you can add the yeast.
Broken Leg Brewery
Giving beer a leg to stand on since 2006