I do.
I agree for the most part, but I've been a mod (and even an admin) on a number of sites in the past. And even still, mods here have on occasion actually talked about things like the time and the standard required of them. So I can certainly understand that one can't hold every perceived "failure" against them. I also know that being a good mod requires a certain standard of behavior that can often dramatically affect one's enjoyment of the site.
Having said that, while I personally wouldn't really mind being a mod (though I don't honestly expect that to happen anytime soon!), I imagine the vast majority of people might say the same, but I really think most people would reconsider once they fully realize what that responsibility entails, both in terms of the time and behavior required. The experience can be drastically different and most would likely find that it's no longer something they want to spend time doing. To relate it to something that at least AbG can understand, it's like going to an outdoor concert as an actual concert-goer as opposed to an officer working security. A good time can still be had, but you can't exactly just be yourself all the time, and sometimes you downright end up looking like the bad guy, despite being committed to doing the right thing. And I think, even moreso than most sites, that most of the mods here do a damn good job of it.