You will need to provide additional information, like the recipe or at least the hop schedule, did it sit long before cooling allowing more hops to isomerize, to get better feed back. It would be helpful to know how many IBU's are in it and the gravity etc, it may be bitter but there's a difference if it's 45 IBU's bitter vs. 100 IBU's bitter and both will taste different if the FG is 1.007 vs 1.015, and honestly things like OG, yeast strain, and your process may help.
That said I have a few thoughts without that information. To start with yes the tangerine peel is likely to add bitterness but it shouldn't be a ton. Also in my experience beer in general will taste more bitter and less hoppy straight out of the fermenter. I believe this is because it is uncarbonated and the carbonation puts more molecules associated with flavor into the air giving the beer more aroma and taste, carbonation should bring out the hops AND the tangerine. Not only that but I think the carbonation blunts the bitter just a bit. If you did not add any hops towards the end then you may not get much hop flavor. Truth in advertising I've never done short boils so there may be some important differences there.
As far as fixing it goes that will be more or less difficult depending on how you re serving it. Time will help no matter what, racking it first is up to you, but other flavors will fade along with the bitterness. If you are kegging it I'd wait to hear a few more comments and ideas, especially after including your recipe, then keg it and see what it's like before messing with it, it may be just fine. If it is still too sweet just open up the keg and throw in some water with sugar or DME dissolved in it, that will raise the gravity and take some of the bite off, do this gradually so you don't over do it. If you are bottling it may be harder to deal with. You could try adding some sugar to it but this will likely just restart fermentation and you will not get much actual sugar left after the yeast pig out, dark DME is your best bet there but even that may not work as hoped, and you will have to wait until it's done refermenting or you will get bottle bombs. You could water it down and add sugar/DME, I know that's sacrilege to some, but that would help by raising the volume and there-by lowering the IBU's.
As I said posting your recipe will help further diagnosis