Extract is a great place to learn how to care for yeast. If you abuse yeast you can get really crappy beer. If you are getting off flavors in your extract beer, work on learning to care for the yeast first.
After that, brew-in-a-bag (BIAB) is a great next step and is easy and fun. You can do partial or entire mashes with BIAB (if your bag is big enough).
It really is a spectrum. If you're coming out of all-extract, maybe make your next brew something with extract and steeping grains. See what that adds. Get comfortable with calculators for hop additions; the differences between bitttering, flavor, and aroma hops. Gradually up your amount of grain and reduce your amount of extract. You'll learn about mashing along the way.
I'm a partial masher, mostly because I am still practicing and my biggest pot can only take about 7 lbs of grain, so I need some extract to round up the gravity. You can keep incrementing up the grain until the change to all-grain is natural.
It is possible for an excited newbie to go whole-hog and buy an all grain rig and churn out nothing but bad beer. I'd say work your way up in increments, but don't be a snob either. Good beer can be made with extracts and partial mashes; don't shun them because they aren't all grain.