I haven't brewed this one but it sounds like you added the sweet orange peel into the fermenter and are currently fermenting with it in there.
I wouldn't worry too much about infection risk, you pitched way more yeast cells than any bad stuff on the peel. I don't know if they do anything to reduce bacteria on the dried peel at the factory but I'm pretty confident anything on there is in small quantity.
You may get slightly less orange flavor because you don't have the heat to draw out the oils. Some people think the pith (white parts) can add some bitterness but I've never picked up on that much. You could get more orange because the heat didn't drive off the aromatics. It's complicated. Personally I'd probably consider adding it in the secondary so fermentation doesn't blow off the aromatics. Take notes in case you brew this one again! I think it'll be good.
My main concern would be: Are you using a blowoff tube? If not, watch your airlock. The peel could clog a 1 or 3 piece airlock. If you aren't sure, lightly push down on the lid. if you see the airlock rise, this means some air is moving and you are ok. If it doesn't rise or feels like you're pushing on a steel wall, you may have a clogged airlock. if you think it's clogged get some paper towels and maybe some sanitizer mixed and ready; when you pull a clogged airlock, it's going to foam some and make a mess on the top of your bucket. PM me if you think you need more info on this. (As a beginner I added dry hops to early, clogged the airlock, got a call at 11pm saying my beer exploded)