I would have let it finish, just out of curiosity to see what happened. If the starter finished and the beer from it tasted clean, it might have been just fine. Or, it might have been nasty vinegar bug juice.
But for a few bucks of yeast, no big loss to start over, compared to the risk of a ruined batch.
Yeah I thought about throwing the yeast in anyways to see if any bugs that got in would actually make a difference. This was for a westvleteren 12 clone though so I didn't want to waste 3 months of patience for a vinegar quad
+1 on tossing it. You can half ass your brewing by pitching a possibly infected starter, or you can strive to make the best beer possible with proper sanitization. You opted for making the best beer possible. Good call.