Yes, it happens and the white residue is hard to just rinse off. When I need to really clean out my bottles (when i have new ones with the labels still on for example) I soak them in Oxyclean overnight, empty them and rinse them out, then soak them in clean water, again overnight. I never have any residue issues that way. The trick also seems to not let the bottles dry too much after the Oxyclean soak. Once the residue dries up and becomes clearly visible, it's hard to get remove. A mild acid like Star San will get rid of it though.