Update...
So, after further inspection it turns out that rough surface I felt was actually very fine, hardened mold. It was on the liquid dip tube as well, and even on the inside after I checked it out with a flashlight. I took some #0000 steel wool to the inside of the keg and voila, mirror...