Yes, malt does impart calcium and magnesium into wort. However, the concentrations we aim for in our brewing water are largely based upon empirical results and including targeted concentrations in our brewing water produces improved beer. In addition, I contend that the calcium and magnesium bound in the wort compounds may not be as available to affect taste and flavor. Therefore, it is appropriate to add certain ions to brewing water to produce desired results. The levels of ions in the water will depend on the taste and flavor goals of the brewer and drinker.
In my experience, brewing with very low ionic content water produces bland beer. In the case of lighter and maltier styles, having low ionic can still be desirable.