What are you using to add the flavorings? I find that orange zest gives the freshest, most authentic flavor to a beer. It doesn't require a long contact time, generally I'll add it 24 hours before packaging. Depending on how orange-y you like something, one to two medium oranges will work.
As for chocolate I can recommend cacao nibs enough. In case you aren't familiar with them they are the crushed, fermented, and roasted pieces of the cacao bean. They give a superb chocolate flavor, without any of the sweetness you can get off of using other chocolate products. I usually let them sit in a secondary for a couple of weeks. If they aren't available to you I'd definitely go for a high quality chocolate that's on the dark end of things so that you get more of that rich flavor, without the candy sweetness - though as a caveat I've never actually used chocolate in a bar form in brewing before. But I imagine breaking it in to small pieces would allow a lot of the flavor to come through. I've worked with powdered chocolates in the past and not only found they made a bit of a mess, I just wasn't overall impressed with the flavor.
As for the brown sugar, I'm not sure that will do much for your beer other than make it sweeter. While I prefer a more dry chocolate beer, that's just a personal preference so your mileage may vary.