I actually had the same problem at first (maybe quite a bit more severe). Now I am just a little more careful with long-ish exposures to star san. If I am going to be working with it in direct contact with my skin, I just do what I need to do and then wash/rinse afterwards and seems to have solved the problem.
Initially, I had a real issue after bottling and dunking my hand into a little bucket where the caps were soaking in it. After doing that 50-ish times, I noticed about an hour after I was done, I was shedding the skin on my hand. It was pretty insane. Freaked me out for a second because all the skin on my thumb basically peeled away all at the same time.
My brother-in-law can't even use the stuff because it is so irritating to his skin specifically, his nose/sinuses, that he swore the stuff off. The one time he used it he started to get headaches, eye irritation, and stuffed up. Pretty nasty. As far as I know he wasn't snorting the stuff or using it as an aftershave either... so he claims...
So yeah, some folks are definitely more sensitive to it than others - lol. I think my brother-in-law is the extreme example.
It sounds like if you invest in some good brew gloves you should be all good though. Definitely worth it.