The warning comes from glass not being a great thermal transmitter. If you have untempered glass, with a large heat sink of liquid inside keeping the inner layers cool, then a strong heater on the outside, you'll create stresses in the glass which could cause cracking. That's the general theory, and it's why you don't boil yeast starter wort in cheap unknown flasks directly on the stove. With carboys i'm sure there is a huge range of quality between different sources, so some are safer than others.
However I would say the risk with a heating belt is rather low, since the heat applied isn't at a huge temp delta to what is inside the carboy. If you're worried at all the towel idea between the belt and carboy is a good way to spread the heat more and prevent a violent thermal shock. Or used one of the heaters made for glass as mentioned above.