Do all the prep work the day before you plan on brewing and if you can, have your rig setup up to go the day before as well. That way all you have to do is fire up the strike water and you're in business. If you clean as you go, you can may be able to hit your ideal time.
This is great advice. Grind your grains the night before, measure your strike water ahead of time too if you can.
Aside from that, learn to clean as you go, and make use of the down time. It makes for a less relaxing brew day, as you'll be working almost the entire time, but it's much quicker.
-Setup your equipment as the strike water is heating.
-Weigh out your hop additions during the mash.
-Measure and heat your sparge water early-on during the mash.
-Clean and sanitize your fermenter (and anything that will touch the wort post boil) in your downtime during the boil.
-Get your chiller, and any chilling 'stuff' setup and ready before you need it.
Doing those things really helps me. I can consistently get a batch knocked out and have everything cleaned in 4:00 to 4:15 assuming it's a 60 minute boil. It makes for a less relaxing brew day though. If I have friends over, and we're shooting the s***, and I'm in a more relaxed mood, that brew day becomes 4:45 or so.
The more you brew on your setup, the more you'll find your stride. Try not to stress yourself out too much trying to speed up the brew day. Remember, it's still a hobby =)