With most of the bi-metal dial thermometers, the calibration screw simply rotates the dial face relative to the needle indicator. It gets kind of tricky from that point on as you must place the probe into a liquid of a known temperature and that is usually ice slush at 32F or boiling water at 212F. Neither of these is in the mid range of 150F or so that we are most interested in for mashing purposes. This can lead to much frustration if any error is not constant throughout the temp range. IOW, it may be off at the low end and dead on at the high end or vice versa. Good luck trying to establish a liquid at 150F for a compromise. This can be done with a good glass lab type thermometer, but you may be dealing with a moving target so you have to be quick. By now, I am sure you are getting my drift. Boiling water can often fool you too as the water boils at the bottom of the kettle, but it may not be exactly 212F at the surface. Probably close enough though. The same with the ice slush. It will often be somewhere around 32-34F depending on how finely you crush the ice and how much water is in the mix. My expensive dial thermometer was considerably off, would not hold calibration well and it was inconsistent throughout the range. I finally just said "fuk it" as I so often do and bought a digital thermocouple thermometer from Cole Parmer. This is the one I have been using for quite awhile and I've been very happy with it:
http://www.coleparmer.com/catalog/product_view.asp?sku=9121045
It's a lab calibrated traceable model. Runs on a 9v battery and it's not overly pricey. I'm done with the dial thermometers forever.