I think the jump to all-grain looks more daunting on paper than it actually is. The only major adaptation, is the more time it takes. I personally find the all-grain process more leisurely than extract. Once you start the boil for extract, you have to have all your stuff ready to go. Extracts open, liquid ones warmed up enough to flow, when to add them to the boil, etc.
All-grain, where it takes more time, allows some time to get things together as you are brewing, but with that leisure, comes more time. So that's the real trade off.
As has been mentioned here before, the key to making a good beer, really lies in fermentation. That's where most of your off flavors come from. So, either way you go, make sure you are keeping your fermemtations in the mid to lower range of the temperature for the yeast. I like to keep most of my basic Ales in the mid to lower 60s. So for some on their first brew, it might be better to go extract, if you are unsure of your fermentation control This way you haven't invested 6+ hours in making a wort that doesn't come out very good because you let it get up towards 80 degrees while fermenting.
If you are confident you can keep your temps steady, then personally, I don't see any thing that would keep me from going all grain. just do you research so you have a decent game plan set up when you do start.