How big is your starter? If it's only a liter or so, just pitch the whole thing. Much more than that and it may have an impact on the final product. All else being equal, I'd rather pitch an entire 2L starter than skipping a starter and pitching just a vial of yeast.
If it's a standard gravity stout, I'd just do a 1L starter and pitch the whole thing.