I normally pitch my entire starter, I recently built a stirplate, so I let them grow fo 18-24 hours and try to pitch at high krausen, if for some reason my efforts get derailed, I will let it sit on the stirplate for about 48 hours and then stick it in the fridge.
Pitching your entire starter causing off flavors is debateable, if your starter wort is ~1.040 you shouldn't stress the yeast too much, and if your temps are controlled (not too hot) I don't see where a considerable amount of off-flavors can come from.
As you stated, if you're pitching 1 liter into 18.5 Liters how much could it affect it. Plus the yeast are going to stay in their aerobic state and continue to reproduce until all available oxygen is gone, so they are basically doing what they were in the starter to begin with. Plus if you extend your primary, some of the off-flavors that may have devolped will be reduced. The yeasties "clean-up" after themselves to some extent.
I also brewed an APA last Sunday, and I will leave it in the primary for 21 days, then I will dry-hop (in the primary) for an addition 7 or so days. This will be my first APA.