This time of year I mash in the basement and it hovers around 50 F. I always wear some kind of beer related shirt, then jeans, my leather Pilsner Urquell apron, and either shoes or leather slippers, and a hat (mostly to help keep warm). I change hats after I mash in to a different hat. I also wear gauntlets when doing any step that involves raw grain. I was surprised to notice just how much dust collected on my arms, glasses and head as a result of milling and mashing in - even with being careful. So now once I'm mashed in the gloves come off and the hat gets changed.
In the summer, I switch out jeans for shorts, and ditch the second hat.