I think Yuri got it right, you washed most of the yeast away and got just a small amount into your final starter. The cell counts are low and it will take time for the yeast to increase in numbers as well as any bacteria along for the ride.
Next time you do this, keep in mind that all you are trying to do is reduce the amount of trub you carry into your starter, not eliminate it. As homebrewers, we do not have the equipment to remove the trub as effectively as we would like.
I repitch my yeast regularly from my primary fermentations and they seem to grow fine in my starters. Here is what I do and, keep in mind, everything I add to yeast has been boiled for 10 -15 minutes and cooled in a covered stainless steel pot and all containers have been sanitized.
1. After removing my fermenting beer into a carboy, I leave a little beer in the bottom of the bucket so the yeast, trub, etc is the consistency of flowable mud. I pour this into a half gallon plastic container. This saves me from using sterile water. The container is the plastic containers my LHBS uses for selling liquid extract.
2. Often, I put this collected slug into the refrig until I am ready to make a starter which may be a week or two, no more.
3. To make a starter I add sterile water to the slug, about a quart or so, swirl well, wait about 1 -2 minutes (the big chunks will settle in that time) and then carefully pour about 2 inches of liquid into a one quart mason jar. This is my harvested yeast. To the mason jar I add my sterile wort, add an air lock and let them go overnight. I throw away the remaining container of slug. You will get fresh yeast once you new fermentation has completed primary fermentation.
I only reuse my yeast like this about 3 -5 times, then I buy new yeast.
I hope this helps.
Dr Malt