The only beer I've done with honey, I used 2lbs at flameout. I didn't skim anything. It didn't give me a hangover.
From what I do know about honey, if you boil it, it'll scrub off much of the flavor characteristics that it might lend. So if that doesn't bother you, then boil it, skim it, and see how it treats you.