If you are putting the peel/zest into primary, I would encourage most people to sanitize it somehow. Oranges often rot from the outside in, and it's almost always a mold(!) on the skin(!!), so... I just think the skin is a mold risk, and mold is probably the most dangerous thing that a beer can grow.
Also, unless you grew it yourself, it could have pesticide on it. Wash the skin, IMO.