If you want to get away from using kits, then you'll need to learn what the different grains, hops, and yeasts add for flavors and such. I would start by getting the gear you'll need to start brewing all grain (pick your method first) and then start making recipes. There are plenty of sources out there that will tell you what a specific grain will add for flavor, character, etc. to a brew. Same thing with yeasts (look to the lab's sites for best info) and hops. You can also start by looking up recipes of brews you already like and seeing how much of each grain is used in them.
Personally, I went from my first two batches (slightly modified kits from the LHBS) to the third heavily modified 'kit' from the LHBS to making my own recipes.
I would also heavily advise using software when you're putting together a recipe. I've been using BeerSmith for some time now (since before it went 2.0) and really like how I can adjust a recipe almost on the fly and see what the results will be (before crushing the grain).
Depending on which type of method you go with, you'll need to calibrate the results from said setup. I know a lot of people are lauding BIAB as the new coolest method. Personally, I see it as having far too many limitation, at least for me. With a three vessel setup (propane burners) I can easily mash about 30 pounds of grain without worrying about the grain bag ripping, or getting hurt lifting (and then holding) it up to drain. As I see it, BIAB is viable for 5 gallon batches, of moderate strength. Once you get to something stronger, or larger batches, it quickly becomes an issue. But, with how little you can spend initially with it, it's not a bad start. I used it for several batches before moving to a cooler based mash tun, then finally transitioning to a keg mash tun.
As for using ingredients you actually grow yourself... You can easily grow hops to use. Trying to use your own grain can present you with a entirely new set of issues. IMO, better to purchase the malted barley already ready for use. Get involved in a local HBC (HomeBrew Club) that does periodic group grain purchases. Or get into one of the group buys from HBT members that gets posted. Depending on how much you brew, you might be able to purchase most of your grain at one shot. Just get the grains you know you'll use enough of.