I'll start by saying I do both.
That being said: all grain is a much purer beer, usually has better mouth feel, and gives you some sick satisfaction that you can brew like a champ.
Extract on the other hand is easy, quick, and with a bit of trial and error, can compete with some of the best all grain brews, as long as you add some specialty grains to your boil. (They release tannins and compounds that are lost in the extraction process). I do not mean to bash any all grain purists out there, but I have competed with my extract and partial mash brews against all grain and have won before.
I asked myself a long time ago whether to start all grain right away, or invest in a good home keggerator. the keggerator won (pics below).
some things to think about if you want to stay with extract. add anywhere from 1-2lbs of specialty grains to the boil, they make a WORLD of difference to the finished mouth feel and malty characters in the beer. Also, stay away from DME. I use only good quality, unhopped, fresh LME from my local brewpub or LHBS store. Use 3 litres (9.9lbs) as opposed to the 6.6lbs most recipes ask for and omit the sugar all together, it will treat you better in the long run and only costs an extra $3-$5. Use fresh pellet hops and make sure you get a good cold break after your boil and this should mitigate a lot of off flavors.
Another thing to consider is your yeast choice. A good yeast will make or break a great beer. To start off with, for ales try Nottingham ale yeast (if you can ferment at or just below 20C/69F). For wheat beer, use white labs heffeweizen IV ale yeast, and safeale US04 or US05 work fantastic for IPA's.
hope this helps. If I can answer any questions just give me a shout.
btw, when it comes to Kegging, you will never go back to bottling again. If you are even considdering it, go for that first.