A higher gravity wort will almost always darken more than a lower gravity wort when boiling, all other things being equal, and sugar will increase that gravity just like malt, so adding sugar will most certainly darken the resulting wort more than not adding sugar. That fact is attibuted to combining sugars with amino acids and forming melanoidins (a process commonly called
caramelization even though it is actually a
Malliard reaction since it is heat and the amino acids in combination and not just the heat which also caramelizes some of the wort). Additionally, adding sugar pre-boil, during boil, and post-boil will all have that same reaction; darkening wort quite a bit, some, and basically none. Houblon's beer sort of makes that point very clear. There are some books that go into this subject; some oversimplify the subject, others go too in-depth, but the info is out there. Also, I am positive there are references online that will be just as good for purposes of homebrewing!
Additionally, worts brewed with 100% malt and no sugar will have the same relationships. If you brew a wort with, say, 10lbs of malt and one with 15lbs of malt, keeping the batch size, malt type, the boil time, etc equal, the wort with 15lbs of malt will be darker for the same reason as listed above.
On the other point, invert sugar, there are those that claim to taste the difference in finished beer between invert and "regular sucrose" and I won't go there. The process of boiling AND the acidic nature of wort does invert sugar.
Some of those bonds will break. How much is up for debate until someone does a study on it. I will say that a lot, if not most Belgian breweries use plain sugar in their worts and not invert sugar if they use sugar in the recipe; I've seen this with my own eyes. And clear candi rocks are pure sugar, not invert sugar, fyi. Most, if not all, commercially available invert sugars are liquid.
Personally, I'm more concerned about flavor contributions than color. If you're looking to make a more interesting beer and it can fairly dark, like a Dark Strong or even a Porter or something unexpected, I would add the sugars early. If you were making a Golden Strong or Blonde, I'd probably add them later. Again, that's my personal opinion so you may think differently.