There is typically a lag phase in which the yeast reproduce until they reach an optimal concentration for the gravity of the wort.
Plus the majority of the stuff at the bottom ("trub" pronounced 'troob') is protein and whatnot from malts and hops.
In short, you'll be perfectly fine. It hasn't even been 24 hours yet. If you have some beer already ready to drink, I'd suggest you RDWHAHB (Relax, Don't Worry, Have A HomeBrew)