For half the guys on the field, stats are meaningless (unless they want to count something unofficial, like "number of times beaten for a sack"). For the Pats d-linemen, they're generally asked to tie up a couple of blockers so that the linebackers can come in and make the play, so you'll never see any of them put up Dwight Freeney-like sack numbers. For cornerbacks, interceptions and even tackles are pretty meaningless, since the best guys aren't thrown at. Really, the only positions where stats have any meaning in football are at running back, WR, and QB, and even then it's imperfect.
Now, the people who SHOULD know who the best defensive linemen are, or the best center, or whatever, are the guys on the field - but from all accounts, they tend to vote based on reputation as well.