No. As long as your follow the correct procedure while cooling and transferring wort to the fermenter, your chances of contaminating your beer are the same whether you brew inside or outside.
I use a semi-closed transfer system after the boil. I sterilize the chiller, tubing, and fermenter, and close off the boil kettle with a sterilized lid and the carboy with a sterilized carboy cap/tube (the wort is fed into a tube that goes down to the bottom of the carboy). I oxygenate the wort using an sterilized oxygen system and a similar sterilized carboy cap/tube device. I add yeast using a sterilized funnel, and then a sterilized carboy cap/blow-off device is put on the carboy before it goes into a fermenting chamber. Did I mention that I sterilize everything?
The only thing that I worry about when brewing outside are acorns in the fall (my patio is has several oak trees on the periphery). I brew inside my garage in the winter months, with the garage and back door open for ventilation, but it has to be pretty cold to drive me indoors.
"Skunking" beer, imho, has more to do with contaminating the beer with oxygen while it is fermenting and aging.