I do the same cleaning as you, but I always give my bottles a good soak in diversol sometime before using. I've never used a bottle brush.
I do this because my bottles can collect dust, spiders, and other things along the way. Also, sometimes I get "gifted" bottles and they get mixed in.
My...