Yes, it absolutely could. When I first started crushing my own grain I did not have the drill speed dialed in yet and my efficiency would vary. Once I got that right I started hitting my OG every time.
I experience varying efficiency in my system. BigMack, how did you dial in your drill speed and how would that make a difference? If your rollers are set, does the speed make that much of a difference?