I did aerate before and after, just sloshing the wort around in the carboy. I am now more concerned about a bacteria infection from the unsanitized bowl.
I wouldn't worry about it.
If there is no airlock activity, check to make sure your bucket/carboy is sealed properly, and there is enough water in the carboy (believe me, it happens, you thought you sealed the bucket, but there was once part not popped down all the way).
Also, even if the bowl was riddled with bacteria or wild yeast, you still pitched yeast. That yeast will still ferment, probably faster than its unwanted neighbors.
The only thing that it could be - 1. bad packet of yeast. 2. You killed it while hydrating (water over 90 degrees). If the yeast looked healthy (they foamed up, got kind of gunky), then it should be fine.
Mostly likely its relax, don't worry, have a homebrew, and it'll be fermenting before you know it. I've had beers take 48 hours to start fermenting with US-04. I've made other beers with US-04 in which fermentation was almost done after 48 hours. So its unpredictable sometimes.