High pH usually leads to dull flavors rather than off flavor. Was the off taste a kind of sour bitterness? If so you can probably blame the magnesium though water problems are by no means responsible for all off tastes.
If you dilute 1:1 with RO (or DI) water you cut everything, including the alkalinity, approximately in half, 2:1 reduces everything to 1/3, 3:1 to 1/4 and so on. Brewers like alkalinity less than 50 generally speaking so a dilution of 6:1 or 7:1 is required. This lowers everything by a factor of 7 or 8 and calcium, chloride and sulfate (if you want it) will have to be supplemented. An obvious question when such heavy dilutions are contemplated is 'Why not use 100% RO water.' You can indeed do that thus attaining the ultimate in water control. There are also several other ways to remove the bicarbonate and magnesium none of which are as simple as using RO either in toto or as a dilutant.
You may find the Primer here helpful.