The settings on your refrigerator usually dont correspond very well to temperature. We get used to turning it up in the summer and turning it down in the winter. You need something better to compensate for the relatively wide temperature swings in your garage.
Oh, and Im guessing your fan is pointed at the condenser, the warm coils on the back, not the compressor, the electric pump. If it isnt, it should be.