It took me nine years before I went from Extract to All Grain. Alot of those years I did maybe one batch or less.
I regret not making the move earlier. It made me a better brewer, gave me the tools to be more creative in my brewing and I understood brewing SO much better.
Are you me? I started brewing in 2004, and did a total of 28 batches (including several meads and ciders) from 2004-2012. This year I discovered BIAB, and I've brewed 48 batches since January. I just built myself a "real" mashtun (toilet braid style) this month and I've enjoyed not having to lift and squeeze grain bags, and to be able to more easily do high-gravity beers.
For me I find the price difference between Extract and AG to be very important. I bought LME in bulk, but even so it's still much more expensive than buying grain in bulk, and SUCH A GIGANTIC PAIN.
My equipment outlays have been very modest - a 7.5-gallon kettle with a ball valve ($120), a Corona mill ($25), cooler + toilet braid + tubing + valve ($50). The money I've saved on ingredients has more than paid for the equipment.
There's nothing wrong with using extract, and you can certainly make very good beer with it (especially with some of the newer extracts out there). I find myself being more creative with AG, though, especially since I can brew on a whim when the inspiration strikes me, as I keep base malts, specialty malts, and bulk hops on hand.
I also find it much more relaxing - it probably only takes me an extra half-hour to an hour over extra + specialty grains, and I don't have to mess around with sticky extract.
Additionally, there are some kinds of beers you simply can't do with extract. You can't really do a proper dry stout since the flaked barley needs to be mashed. I brewed a Classic American Pilsner with pilsner malt and polenta, which came out about 2 SRM. (That one was not so relaxing, as stirring the cereal mash is a pain in the rear.)
Honestly, though, you know what keeps me coming back to AG? I absolutely LOOOOVE the smell of the mash, and the differences in the aroma between different malts are amazing. If somebody made mash-scented candles, I would buy them....