Pitch that starter man! 20 hours is good enough. As long as you are pitching the entire thing then pitch now. If you are waiting to let it ferment out, then you should have started the yeast starter two days from brew day, giving it 24 hours to build, then chill overnight, decant and pitch...