Will it have a large impact on fermentation if I do so now vs. later?
Not really, and by waiting, you could adjust to taste.
Remember one unassailable statistic, as explained by the late, great George Carlin: "Just think of how stupid the average person is, and then realize half of them are even stupider!"
"I would like to die on Mars, just not on impact." Elon Musk