The essential math in case of infection is that yeast take about 1 hour to double, whereas bacteria take 20 minutes. It takes exponentially less time to get 1M bac's as compared to 1M yeast.
0min./1bac, 20/2, 40/4, 60/8, 80/16, 100/32, 120/64, 140/128, 160/256, 180/512, 200/1024, 220/2048, 240/4096, 260/8192, 280/16384, 300/32768, 320/65536, 340/131072, 360/262144, 380/524288, 400/1048576 = 400 min.
0min./1yst, 60/2, 120/4, 180/8, 240/16, 300/32, 360/64, 420/128, 480/256, 540/512, 600/1024, 660/2048, 720/4096, 780/8192, 840/16384, 900/32768, 960/65536, 1020/131072, 1080/262144, 1140/524288, 1200/1048576 = 1200 min. (20 hours)
So, at the 7 hour mark, 1 yeast has multiplied ~100x, whereas the bacteria has multiplied about a million times.
It's even worse than that, too... because bacteria can split an unlimited number of times, but most yeast "scar" when they bud off a shmoo -- so they can only "give birth" about eight times before they go sterile and essentially die... whoops!
That's why you need to pitch so much yeast right away -- because they need the headstart. If they are in the millions to start, and if there are sufficient nutrients and enough oxygen, then they can compete with the bacteria -- but without enough of a headstart they don't stand a chance.