Your Regulator looks like a TapRite-Fassco Regulator...with Micromatic Gauges on it.
They make regulator bodies with the safety relief on the side...and on the back. Since your's is on the back...I found this schematic. It's for a Higher pressure regulator used in the Soft Drink business...but the parts may be the same.
Make sure the REGULATOR is not fastened to the CO2 bottle when you proceed!! You can get hurt taking apart a regulator if it's under pressure.
Refer to the schematic...and look at parts 9, 10, 11, 12... You can remove these from the back of the regulator. Make sure the seating surfaces are clean...and reassemble. Hopefully there aren't any parts missing from somebody else screwing around with it before you got it.
Anyway...that's where I would start...and if a cursory take apart, clean, reassemble doesn't fix the relief valve/or if it's missing parts...go back to your LHBS shop for a swap.
Hope that helps get your regulator fixed up...