the pith is the bitter part of the peel, sooo that is why you're experiencing the undrinkable bitterness.
but I completely agree with JLem, don't throw it out or anything. Give it time, I would think there is going to be a little extra bitterness in the beer because of the pith, but it...