A lot of factors some into play. When did you harvest the yeast? Just because you washed it the day before, doesn't mean it's ready to go. I always make a starter with washed yeast unless the beer I brewed with it is less than a few weeks old and I just washed a couple days before. Of course, if that's the case, I don't even bother washing.
Again, though, that looks like a really small amount of yeast. If I were a betting man, I'd say you seriously under pitched.