As the other poster said, sediment in the bottle is normal for bottle-conditioned (bottle carb'd) beers. Just don't pour the last 1/4" of beer from the bottle when you pour your pint.
I doubt that's your off-taste. Its probably your water.
I'm on well water too, and i've been chasing an off-flavor that appears in everything i've made except IPA's. I guess the hops mask it. I had been brewing with untreated well water, and even worse, i was using hot water for my steeping grains. (They say water from hot water heaters can add to off-tastes)
After much deliberation and research, i believe my flavor is from a very iron-rich water. (Insides of toilet bowl tanks are brown/rust colored, and water will leave brown/rust streaks on tubs/showers)
The batch I have in primary now was made entirely with bottled spring water. I'm fairly confident this will correct the problem.
I tried modifying my process using late/flameout additions of the LME, because at first I thought I was scorching my extract. The late additions did not correct the problem. Fermentation temps and sanitation were/are fine. I brew in a stainless stockpot, so not a problem with unoxidized aluminum.
If you like IPAs, maybe try brewing one of those. I've been extremely happy with both of mine, and couldn't detect the off-flavor.
Or better/cheaper yet, brew entirely with purchased water and see if that corrects the problem.