I always remove the labels simply because I want to. I'm gonna give the bottles an Oxyclean soak anyway, so I might as well go ahead and remove the labels. Plus when I move up to kegging I'll have a large stock of label free bottles to sell to new brewers :D