I consider TOOL to be the most musically talented band in existence. Guitar, bass drums, vocals, lyrics and the entire musical compositions are simply 2nd to none.
That said, it's easy to understand why they aren't more popular: they just aren't as musically accessible as most bands. Their songs are in complex time signatures and have unusual compositional structures and phrasing. Extreme variations in tonality and dynamics along with lyrics that are often very uncomfortable to think about makes for less popularity among mainstteam fans.
Most importantly, in order to truly appreciate the music, a lot is required of the listener. Most people listen to music as background noise while they're engaged in something else and usually on a cheap sound system from mp3's. TOOL music doesn't come across that way. You must really pay attention while listening with no distractions and have a decent music player with a high quality source. IMO, the album 10,000 Days is worthless unless you can listen to it from begining to end at a relatively loud volume with no interuptions. Most people are unwilling or unable to invest that much effort.
My $0.02.
All true. This is why the best music generally actually seems to be meh (or even bad) at first and requires at least several playthroughs paying attention.
Great music requires effort not just on the part of the artist, but the listener as well.