Beer can ferment out quickly, especially the lower gravity ales and the color or darkness really doesn't have any bearing on how fast your beer will ferment.
What yeast did you use? That should really tell us why your beer dropped so fast... I used SafAle S-04 recently and it started and ended very fast.
I'm bottling an ESB today and it the OG was 1.054 and it went down to 1.012 in about a week. My general procedure is to brew, let it sit in primary for 2-3 weeks and then bottle. I don't usually bother checking the gravity before bottling any more, but this batch was an exception.
If your stout tastes good, I'm sure it's fine.