You'll likely damage head formation/retention but apart from that you're not likely to do any real harm. My chocolate stout has never gotten any complaints about head retention from the folks who have tried it, mostly because it's too delicious to complain about. I say go for it.
I can't be arsed to keep up this list of what's in the fermenters, but hey, check out the cool brewery I own!
twitter.com/2kidsbrewing .. facebook.com/2kidsbrewing .. 2kidsbrewing.com