You'll hear most people say 3 weeks minimum, but I've had beers taste great after 1 week in the bottle (I gave them a couple days in the fridge too). I've never tried as early a 4 days though. You'll probably get more yeast flavor and less carbonation than if you wait 2 or 3 weeks. I'd say wait as long as possible, maybe go grab some new beers to try and distract yourself.