If your skipping 'secondary' (which is totally acceptable for some styles, etc) so you can get to drinkable beer quicker - beware that nothing can speed along 'green' (i.e. not properly conditioned) beer. You can bottle now if your fermentation is done and have carbonation within a week or two, but that doesn't mean its not going to get much better with age.
Currently my practice is a 3 week or so primary and straight to kegs... Not eveyone uses a secondary, but the masses practice approximately 3-4 weeks prior to bottling/kegging so that the beer is appropriately aged after carbing for two weeks (so your looking at 5-6 weeks total on average). Even then a lot of beers get better with time...
I would suggest leaving it in the primary and not touching it for at least another week, I'd suggest two, check the gravity when ready, and then bottle or keg it.
EDIT: Note that I am speaking in generalities here. Some people have a 3.5%-4% "house beer" they can brew for 2 weeks and force carb in a keg in a week and have good drinkable beer in 3 weeks. However, for most beers (beers > 4% ABV, etc) most people practice some patience and have waited 6 weeks or so before drinking the fruits of their labor. Generally the bigger the beer, the longer the aging... So if you brew somethign at 7%, plan to wait 8-12 weeks, etc. You can drink as soon as a beer is carbed, but when you get to the last one 2-3 months later, you are going to regret it, b/c its usually a lot better 'matured' and/or 'mellowed' out.