No, it'll be fine. You won't want to strain or otherwise oxidize the beer. What I do is just start the siphon in the middle, and then lower it until it's just off the bottom. That way you don't suck up any trub or hops particles. I never use a hops bag, and I make a ton of hoppy beers.