My Koelsch was the first time I used a starter, I also stuck it in the fridge for a day or two before brewing with no ill effects. Fermentation took off like a rocket. Can't be much different than pitching a vial of WL yeast I should think.
I guess some might say you left it open to infection in the refrigerator (but was it more open to infection than it was on the stir plate?) I think you are alright.