Put a sticky thermometer on the outside of the fermenter. They only cost $1-2. The brewing ones work great, but you can get ones for an aquarium that work about as well. Just make sure you get one that will read temps down to about 60 at least. The nice thing with these thermometers is you can easily check the temp without risking infection.
Yes its possible to get it too cold, but that will just give you a s-l-o-w start to the fermentation. Colder than optimal fermentation temps are better than too hot. To fix that, remove the ice from the cooler. You can take the fermenter out too, but you don't want rapid/dramatic temperature swings. After sitting out for a couple hours, the wort should be warmed up a bit and be fine to put back in the water.