I wouldn't use what's typically sold as 'brown sugar' since it's most often just white sugar that's been mixed with molasses to get the color and a bit of it's flavor added...
I used Turbinado (aka Dememera) sugar to prime my old ale and another brew (still carbonating)... No negative effects on the old ale at all. Since I used minimally processed cane sugar, I know it's closer to it's original form. More of a personal preference than anything else.
Personally, I'm not using corn sugar, or any super-processed (white) sugars in my brews at any stage. It costs me less than $0.50 (per 5 gallon batch) for the sugar I'm using to prime with (buying it in a 2# bag right now, might get a larger amount when this bag is gone), so cost isn't an issue.
You can carbonate with any sugar that will ferment. If you like the flavor of it, and think it will be a good thing if you get any of that in the brew, then go for it. If you're on the fence about the flavor, then don't. Even though the amount of flavor you could get should be minimal, if it's a lighter brew, it has more of a chance of showing itself.