Does the rpm change based on how hard you squeeze the trigger? I experianced (using my milwaukee drill) that if i didnt squeeze the trigger near as hard, it would go slower, and while it took me longer to crush the grain, it came out way better.
The other option is to maybe double crush.
My mill is the barley crusher, and my best guess is that at such high speeds, the malt was "jumping" around behind the rollers and falling to the catcher, where as going slower prevents that. When i finally got around to building a table and motorizing my mill, i reduced the speed via pulleys down to about 180rpm, and i still get a few uncrushed grains, but not near as much, and its usually only when the hopper is very low in quatity that it happens.