You are fine, dont worry about it. No off flavors or extra IBUs.
The whole anti trub and hop matter sentiment on here comes from people who use chill plates. It has irrationally carried over into all chill methods.
I strain, but generally am not that worried about what gets in the fermenter. The only reason I strain is so I can fit more beer in my carboy, instead of wasting space on trub and hops.
Kegged and serving - American Wheat, Daves Porter, Outkast Kolsch, Nectar of Nuggets
Bottled: French Canadian Breakfast Stout, Edworts Apfelwein, Westminster Wit, Put that in your stout and smoke it
Fermenting: Apfelwein, Flanders Red, Skeeter Pee, Heady Topper attempt #1, Kumquat Berliner Weiss, Sabraton Accident Stout, Nelson RyePA
Upcoming: Session IPA, Yeti eqsue stout, Dark English Mild, ESB, CDA, Row 2 Hill 56, Zombiedust, Hoppy Amber, Geuze, 100% Brett Golden Ale