You'll be fine. If you are gone until Friday, the yeast will need time to floculate anyway, depending on the strain. I'd put it in the refrigerator when you get back overnight so that it's all dropped clear. Won't hurt anything to leave it over the next few days.
Kegged: Waldo Lake Amber, Notty as Helles, Vanilla Porter, Sweet Stout (nitro), NB Surly Furious Clone, Petite Saison D'ete, Le Seigle Belge Saison, BM Cream of 3 Crops, Edworts Apfelwein
In Process: Braggot
Upcoming Brews: Surley Furious Clone, Uintah Wyld