^ I think you'll have the same issue with those Johnson Controls controllers as well, unless they're using PID logic rather than simple on/off. I don't think they're PID controllers.
I think the issue is probably where the probe is.
OP: the probe is dangling in air inside the chamber? It's not in any liquid or insulated in any way? What's happening is when the probe reads the correct temperature (your setpoint) it turns off the fridge. But the air that is closer to the coils is going to be a lot colder than the air around the probe, and after a little bit of time, the temperature will even out more, which means the air at the probe will get colder, even though the fridge is off. The air inside the fermentation chamber is going to swing a lot like this unless you install a fan to circulate air.
The better way to monitor the temperature is to try to monitor the temperature of your beer instead of the temperature of the air around the beer, as the huge amount of liquid is going to buffer out the temp swings more, and change temperature a lot less than the air around it does. What I do is tape the probe to the side of my fermenter with a pretty thick (probably 1.5 inches) layer of bubble wrap and packing foam on top of it. Tape it up nicely so air cant freely circulate around the probe, and you should be measuring the temperature of the outside of your fermenter, which should only vary by a couple degrees from the center (warmest) part of the beer.
*edit*
I keep my probe on the fermenter as described above, but I also like to use another probe to monitor the air temp inside the chamber, mostly for ****s 'n giggles. On a cooling cycle, I have seen the air drop to the 30's, yet the beer stays right around my setpoint. That much liquid is slow to change temperature and won't match the air temp at all. So don't worry much about the air temp.