One should never harden water with chalk. Chalk is highly alkaline and will send mash pH much too high. One of the main reasons for hardening water is to try to lower mash pH. An addition of chalk to a mash with 1 unit of potential to decrease pH has 3.5 units of potential to increase pH.
Sometimes the object of hardening is to get the calcium level high enough that it aids decarbonation. Obviously, calcium carbonate would be a poor choice for that.
Sometimes the object of hardening is simply to increase the calcium level for yeast health etc. In those cases the chloride or sulfate are preferred because they do not increase pH as the use of chalk would.
So when does one use chalk? When the mash pH is too low and needs to be raised. Another application is when one wants to brew with authentic water. He may go to the trouble of simulating the water of some city of brewing reknown as closely as he can, including the bicarbonate. This is almost always a waste of time and effort as, at mash pH, most of the carbonate and bicarbonate have been converted to carbon dioxide which will escape the hot mash.
Those who want to add chalk to their water must dissolve it in acid. Any acid will do (for dissolving it) but nature uses carbonic acid for that job. Brewers who are willing to take the trouble will sparge water to which the chalk has been added with CO2 or will put the chalk in a soda bottle or Cornelius keg, pressurize with CO2 and shake thus causing the CO2 to dissolve in the water. The resultant carbonic acid then dissolves the chalk.
As soon as the water is depressurized it is super saturated WRT CO2 and the object is to mix it with the target water before enough CO2 has the opportunity to escape that the chalk starts to precipitate back out.