I suppose it's possible, but leaving your wort for two days with no yeast is begging for an infection. If you do it, make sure everything is super-sanitized, and seal the mouth of the carboy with a stopper and with sanitized saran wrap or foil.
Make a 2 quart starter. That should be fine!