Hillman, there are a few steps you can take to move from extract to all grain brews. The first is to incorporate steeped grains in your extract brews. This can make a world of difference in the range of flavours, colours, and possibilities in your recipes. All you should need is a big pot (a 1 - 2 gal kitchen pot should do) and a nylon bag (a grain bag from your LHBS, or even a nylon paint strainer bag from Lowes will do). Steeping grains are specialty grains that have little to no starch in them: things like crystal/caramel malt, chocolate malt, black patent, or roasted barley. These grains can simply be steeped in hot (~150 degree) water to release the goodness.
You can also do partial mash brewing, which is a fancy steep where you use grains with starch and enzymes in them, typically a base malt (e.g., American 2-row, British pale malt, Munich malt, Vienna malt, etc.), and perhaps some specialty grains like the ones used for steeping. In this process, one adds hot water to the crushed grains to hit a particular temperature (generally between 148 and 156 degrees), at which enzymes inside the grain will convert the starches to sugars. Once this is done, you can drain the liquid and add it to your extract brew for an even broader array of flavour and possibility. To do this, you need a thermometer (and perhaps a bigger pot).
For complete control, all grain brewing (no extract) follows the same process. But to have complete control over the temperature of the mashed grains, you probably want to have a cooler outfitted with a lautering device (e.g., a stainless steel braid, manifold, or false bottom) to help separate the liquid from the grains. You might also need a much bigger pot (7 gal minimum for 5 gal batches of beer).
Anyways, those are the options in a nutshell. There are lots of great recipes out there if you look for those terms "extract with steeped grains", "partial mash" and "all grain".
Don't hesitate to ask more questions. There is a LOT of expertise floating around here.
Cheers