You should see some activity in the starter. There should be some foam or something on the yeast. You may want to add more yeast to the lesser active yeast starter or add some DME to the starter that is "slow"
I wouldn't say that's an accurate way to judge a starter. I look for activity in the first few days like (CO2) bubbles, and foam or krausen. Before you brew let it sit in the fridge overnight. You should see a nice layer of white to off-white yeast covering the bottom of your flask. Pour off 95% of the spent wort from the flask after it comes to room temperature and swirl the remainder into a slurry and pitch into your new brew.