I'm disinclined to try it myself, but I'm having a hard time finding a reason why it'd be harmful, I mean so long as it remains under pressure. I've taken cold bottles all the time out of refrigeration and later cooled them back off with no ill effects. A keg is just a big bottle, right?
I'm going to say that it should be OK. But you try it first.
Tap 1:Traditional Bock
Tap 2:Robust Porter
Tap 3:California Common
Tap 4:Old Ale