Should you have washed it first? Probably. Will it be a disaster if you brew with that grain? Probably not. If it's food grade you know, at the very least, it won't hurt you. Worst case scenario I think it could possibly affect your head retention of the final product. And that is a VERY worst case. I can't imagine there being enough of it transferred to the grain to cause any detectable differences. Plus, between boiling it and racking it you're likely to leave much of it behind in the kettle and fermentation vessel.
I say go for it! Good luck!
