I doubt this will cause any problems at all. Wel all stick stuff in beer, wine thiefs for samples, auto siphons to rack to secondary etc. Sure all of this stuff is clean and sanitized, but how clean? You are also opening the beer up to microbes and bacteria everytime you do this. The beer has already gone through primary, so there is some alcohol in there to kill some nasties as well. If you are doing a secondary in a carboy, I wouldn't worry about it. If in a bucket, it wouldn't be too hard to get out with a sanitzed instrument.