I just use a brush like the one I use to sweep dust off my table saw. I only do this so that it doesn't shed dust everywhere when I move it in and out of storage. There is so much crap in the grain you are buying that I wouldn't worry about trying to keep your grain mill spotless. Besides, using water will most likely result in rusty rollers.
I tap it out. If necessary now it out with compressed air. I had to recently take it apart and clean the bearing surfaces and end of the roller surfaces as a build up had started make the turning of the sympathetic roller sticky.
I was going to used compressed air to blow out my grain mill but have noticed the all the air cans I find now have a bittering agent to help prevent inhalant use.
Will the bittering agent flavor carry over to the mill and get on the grains?
The compress air cans can leave some residue in some situations. I once blew in a tube we used a can of air on, and though it was dry i wanted to remove my tongue from my mouth. Forcibly, with a spoon. It was horrible!