The chlorine isn't actually listed in the water report. What's listed is chlor
ide, as in the ion, most likely from a small amount of naturally occurring salts. In any case, four parts per million is quite low, low enough that some additions of table salt or calcium chloride may be called for.
That having been said, you definitely want to filter the chlorine. More or less any city water treatment out there will involve either chlorine or chloramines as a disinfectant (only a handful use alternative such as oxygen); you'll want to remove them so that they don't affect the yeast growth, and filtering is a cheap and simple solution.
The good news is, your water is quite soft, with low levels of pretty much all the minerals tested for. This means you shouldn't have to bend over backwards to neutralize any of the mineral content. Your main concern will be adding enough minerals to get the right pH and yeast nutrient profile, which shouldn't be too difficult.
I would recommend downloading either
EZ-Water or
Bru'n Water and plug in the values for both the water and the type of beer you are making; I personally like Bru'n Water, but I know others prefer EZ-Water. Either one should give you a useful mineral additions profile.
