There's a lot here to digest and I've tried to read it all but I'll just start with some general feedback.
There are thousands of homebrewers, including myself and almost all my friends, that brew beer that is at least as good as commercial beer. We should be fighting hard to erase the concept of inferior "homebrew". Sticking with extract brewing, here are the ways I'd immediately improve your process:
1. Start with Distilled or RO water.
2. After the grain steeping process, add more water to reduce the boil concentration. If your kettle is too small to be able to do this, a larger pot will help you.
3. If you must keep the boil smaller/concentrated, add half of the malt extract in the last 5 minutes.
4. Double check the packaging/best by dates on liquid malt extract if you're buying kits. If you buy ingredients ala carte based on recipes, switch to 100% DRY extract. It has better shelf life.
5. Use yeast pitch calculators to ensure you're using enough yeast.
6. Keep your hands off the fermenter until bottling day. Dumping trub out of a fermenter without the ability to back feed it with CO2 is counter productive. That is, whatever damage may occur from leaving trub in contact with the beer for a few extra days pales in comparison to pulling oxygen into the fermenter.
7. Keep your ales fermenting in the upper 60's for the first few days at least.
Even if you do a few of those, you should see improvement.