It really is easier than you think. And if you're on the fence, do it. I thought my extract / specialty grain beers were good, but dayum, AG is the way to go IMHO. Yes, there are a few more things to think about, but in return, you get much more control over your finished product. You can control the mouthfeel of your beer based on your mash temp, just to name something. And you start to understand a lot of the science behind brewing that you just don't need to know with extract.
That said, to each his own. Excellent beer can be made with extract also.