All a starter really is is the first two days of the yeast growth phase in a separate container. If you pitch a vial directly, the growth phase in the wort is longer. The total number of cells in your wort when fermentation starts will be about the same regardless, because that is largely determined by the amount of oxygen in the wort. Direct pitching means a slower shift to the fermentation phase, a longer fermentation overall and an increased chance of contamination.
Remember one unassailable statistic, as explained by the late, great George Carlin: "Just think of how stupid the average person is, and then realize half of them are even stupider!"
"I would like to die on Mars, just not on impact." Elon Musk