As a beginning brewer, especially if you are doing extract brewing, water does not play as big a part as long as your water tastes good. This is not to say it plays no role. There are certain minerals which will improve the taste perception of hops and malt. To the extent you want to play those up or not, water is important.
However, with beginning brewing there are really areas you need to pay SO much more attention to that water is NOT where you need to start worrying. Focus on controlling the temperature of your fermentation before almost anything else to improve the quality of your beer.
All that being said, once you get the basics down, including fermetation temp control, you will be ready to start delving into the more complicated, technical aspects of brewing to take your beer to the next level. Water is definitely an area you need to understand. Especially when you get into All-grain brewing (through BIAB, 3 vessel, cooler mash tun, whatever) water becomes critically important. You can do everything else well and your water can really hurt your beer.
So, know you can get away with doing very little to start, but that as you get deeper into the hobby there is more to know and it will make a difference. There are great stickies in the Brew Science forum which will give you the basics. Read them and profit.
Also, know that I have done what you are saying and have learned from experience just how much water can make a difference. I do minimal adjusting today to remove Chloramines and neutralize alkalinity in my tap water based on the recipe I am brewing. It is pretty basic and can be for you too.