I leave mine uncovered, but I brew in a fairly "clean" environment. I don't have a lid for my kettle so its not an issue. If you have the ability to cut a notch in your lid to fit your chiller, go ahead and do it. It certainly can't hurt.
I've notched an old lid. fits pretty well and easy to slip on and off. Also have a 3000' roll of plastic/food wrap from Costco that I use to cover stuff after cleaning. Also wrap whatever was just sanatized. Not something I obsess over, but SWMBO wanted the smaller roll and now I own 3000'
I have an old screen window that I put over the top of the keggle during chilling in spring and early summer when there's a lot of crap in the air and in fall when the leaves are falling. Otherwise, I generally don't bother, or else a lay a jelly-roll pan over the top of the keggle (it mostly covers).
I think this falls within whatever your general level of infeciton paranoia is. I used to go on faith with an open top because I always had to fiddle with my chiller as beer4 does. Then I noticed that if you get your wort spinning (whirlpooling) just prior to running the chiller, you can cover it for the whole cooling process. At that point, I notched my lid so it's one less potential exposure.
Stir your wort to the point where it's all spinning really well (without frothing). Put the lid on and let the chiller run. I used to do the wiggle the chiller thing too to break up the zones but nothing chills as fast as whirlpooling. I used a cutoff wheel on my grinder to notch the lid. It took a few seconds and I followed up with a little sandpaper to dull the edges. Before I notched it, I'd just set it down a little off center. At least 80% of the opening is shielding wild yeast.
When I cooled outside I cut a notch in the lid. Too many gulls flying around at times and why take a chance on a contribution from one of them when a cover works fine.
Now I cool inside, so my paranoia is not justified.