If you're using StarSan, it won't do anything to your brew. Being a "no rinse" sanitizer, it is meant to be fine with your brew in this type of situation. Most of my carboys still have a bit of StarSan bubbles in it when I rack to a secondary. Never experienced any type of off flavors due to this...
"Brewers enjoy working to make beer as much as drinking beer instead of working."