The first question would be as to how old the meter is and how you have stored it, how much use you have had with it, whether you have put it into hot mash etc.
It is essential that a pH electrode be properly hydrated (per the manufacturer's instructions) that it be stored per the manufacturers instructions (usually in a particular storage fluid, most often a saturated solution of potassium chloride), that it be kept full of electrolyte (if it is a refillable electrode in which case it is also important that the fill port be open while the electrode is in use), that the reference junction be renewed as necessary (if it is a renewable junction electrode), that the sensing bulb be kept free of protein deposits and that it not be subject to high temperature. In a nutshell, follow the manufacturers directions to the letter.
The usual cause of drift is a plugged reference junction. This is typically a small frit which allows electrolyte from a gel or liquid filled into the body of the electrode to flow into the sample. If you look into the end of the electrode you will see three items: the glass bulb (the sensing element), a slightly smaller metal rod (the temperature sensing RTD) and a third item (the reference junction). If inspection shows that it is covered with gunk, mold, etc then you need to clean that off. A stream of DI water from a wash bottle is probably the easiest but won't get anything too stubborn. In brewing the best thing to use is an enzyme cleaner such as Zymit (go to
www.coleparmer.com) or an enzyme cleaner made especially for cleaning pH electrodes (ideally by the people that made the electrode). If anything is going to solve the problem this is most likely it.
In some electrodes you can pull out the frit with a tweezers and replace it. Obviously that is even better than cleaning but not many electrodes offer that feature.
Eventually, of course, gel filled electrodes run out of juice but they usually are loaded with enough to last a good long time (i.e. longer than the life of the bulb). If you store in the wrong storage solution that can draw electrolyte out of the electrode much faster than normal. In either case and electrode that is depleted needs to be replaced.
Finally, pH meters do drift quite a bit in solutions of low ionic strength like distilled or otherwise deionized water - perhaps even some RO waters. If your water is very pure it would not be unexpected that the electrode take a long time to come to equilibrium. If adding some table salt stabilizes the reading then low ionic strength was probably the cause.
Use of the mV mode with fresh buffers can be diagnostic. In pH 7 buffer the meter should read at most a few mV (positive or negative) and the reading should be stable even if the temperature isn't. In pH 4 buffer the reading should be close to +173 mV and should be stable if the temperature is stable but will drift with temperature. If you get drift in buffers then look at a plugged reference junction as the probably cause. Note that it is normal for it to take a couple of minutes for the reading to change from near 0 in 7 buffer to +173 in 4 buffer. As the electrode ages or as the bulb becomes coated with protein, this takes longer. Note that cleaning with Zymit not only cleans protein off the frit but the bulb as well.
If your meter tells you slope and offset as part of the calibration process it is a good idea to write these numbers down with the date of the calibration. An electrode has a finite useful life (can be a bit more than 2 years with proper care) and the history of change in slope (should be around 57 mV/pH but is sometimes given as a percentage) and offset (should be 0 but will increase in magnitude with age) will give you an indication as to how healthy your electrode is.