Yes, you run the risk of contaminating your wort. It is, however, a minimal risk, I think. It certainly sounds like fermentation was pretty far advanced by the time you switched out for the blowoff tube, and the alcohol should help resistance to infection. Watch it until you can verify, by SG readings, that fermentation is finished. Hope for the best.