It's possible that by not "sanitizing" the bottle caps, you allowed microbes on the bottle caps to provide an off-flavor to your cider. On the other hand, if your flavors were not "off" but merely different from batch to batch, the differences in flavors could be due to yeast type and viability, concentrations in sugar in your must, fermentation temperatures, different clearing and settling schedules, etc.
I make the mistake of saying "sterilize" instead of "sanitize" all the time - even though I know the difference in meaning between the terms. You would think that in this era of hand sanitizer I would begin to speak as if I know the difference.
The bottom line is to make sure that there are the least number of microbes on your bottle caps, bottles, etc. - and anything that touches them as well - so that the very small amount that may get into our fermenter are overwhelmed by other factors, such as competition from yeast, for example.
Soaking bottle caps in StarSan is a good way of sanitizing them. So is picking them up wearing sanitzed latex gloves, etc.