Bummer! I've lost a couple tanks of CO2 this way also, relief valves and system leaks.
I've got several different styles of relief valves on my kegs. I've had to take a couple of them off and rebuild them with o-rings from an o-ring kit that I have. So I guess it's up to you, you could tear into it or just replace the whole valve. Not being there and not knowing which you have it's difficult to say.