Depends on your dust situation I think. If it's just loose on there - like recently collected dust - I think you could get away with rinsing. But I have some dusty bottles that have sat in my garage for extended periods and with changes in humidity across seasons the dust will start to kind of stick. In that case I'd use a bottle brush . . . or just soak them in bleach water for a week (rinsing well after that, of course). I soak all bottles - clean or not - in bleach and that stuff will eat just about anything off the glass.