That's why right there. Think about it: The temp rises in the fridge and your thermometer senses the air temp but the bulb is only sensing the water temp which takes time to equalize to the air temp...so your thermometer sees a big change in temp but the bulb only sees a small change. Finally when the bulb senses the high temp it kicks on the cooling. Now the fridge cools and cools and your thermometer sees a big temp drop but the bulb doesn't see that temp drop until some time has passed...meanwhile the thing just keeps cooling and your thermometer keeps dropping. Rinse and repeat.
I actually do it just the opposite of what you have...I put the thermometer probe in a jar of water and the controller bulb is exposed to the air. FWIW, I work in Chemical Lab and we keep lab standards in a tightly controlled fridge. There is a tight temp spec on these lab standards. Inside each of these fridges is a small jar of water with a calibrated thermometer in it.
EDIT: Plus by doing it the way I am...you don't see the big temp swings on your thermometer and imo you get a MUCH better idea of the temp your keg contents are at. I'll bet your kegs are not quite as cool as you think. If the air temp swings between say 40 F and 50 F...your kegs are not at 45 F...they're warmer.