Waits' music changes tremendously over the years. The early stuff (that many people like a lot more) is very much classic blues-oriented. Some very cool stuff that's very much influenced by beat poetry (don't get turned off by that, things like "Diamonds On My Windshield" are very cool). "Heart of Saturday Night," "The Piano Has Been Drinking," "I Hope That I Don't Fall In Love With You" - dozens and dozens of classics.
Over time, starting around Swordfishtrombones, it starts getting more.... eclectic. Still blues-based, but some stuff that really sounds like it's almost field hollers. Leadbelly-esque. A lot more organic sounding. More stuff that's very... indescribable.
A lot of stuff where he's getting some very beautiful music from very ugly sounds and from very ugly places.
Of the "newer" stuff, I LOVE the album "Mule Variations."
"Alice" is also a gorgeous album, it's music written for a play he did with Robert Wilson about the relationship between Lewis Carroll and Alice Liddell. It's very different from anything else he's done, but spectacularly gorgeous. I used to sing the album to my daughter when she was little. Melodies that are just etherial.
Just, keep an open mind. You probably want to start with some of the older stuff ("The Heart of Saturday Night" is very accessible). "Rain Dogs" is excellent for when he was beginning to transition some of his sound. Listen to the track "Singapore."
As you can tell, I'm a fan.