Restarted with a weaponless thief/mage build. Got my sneak and destruction leveled up, the perk for +25% to fire spells, have the robes of destruction, and now I tiptoe in, lure them out one at a time and hit them with a firebolt where no one else can see.
And if I run up against a heavy magic...