When I do bottle, I use guinness bottles, but I've taken the widget out. I've noticed when taking them out that there tends to be a small amount of brew in them. This small amount may affect your taste.
My recommendatin is to take them out, then sanitize the bottle. Very simple to do...