I think you have a very young mead with enough oranges and the "white junk" between the orange and the rind to give an inital bitter and funky taste.
When I make an orange mead, I take the outside rind, and the inside pulp only.
Don't worry, time will mellow out the flavors, and tone down the esters.
Just remember to give it a lot of time, the more the better.
In Primary: Belgium Chimay clones.
In Secondary: Braggot, pale ale, end of the world white.
Conditioning: Mead, Cider, braggot, Belgium Wheat.
On Tap: Clones, Chimay Blue, Red, Porter, malted cider.
Bottles: Far, far, too many to list.