If you're anything like the average man then you've most likely been kissing a lot of different women - maybe even on the same night

And you are very likely not the only man they've been with - maybe even on the same night... And it's definitely not unheard of, even for married people, to be getting some on the side and then come home to get busy with their partner - again, maybe even on the same night... You get my point, by proxy. Here, honey, swirl this listerine before I give you a kiss!
99.999% of the time, a weak immune system is what makes us sick; not exposure to bacteria and virii. The immune system can't strengthen itself in a sanitary environment; it needs small doses of this and that nastiness on a regular basis in order to improve it's general strength of immunoresponse. And then there are some things, like ebola, from which even doctors who practice rigorous sanitation still die - hand washing didn't and couldn't have helped them.
The average penis will harbor roughly 45 unique types of bacteria. This is very tiny - no pun intended - in the grand scheme of things. I'm more worried about touching the door handle of a public toilet on the way in, which can harbor over 100 thousand unique types of bacteria, and then touching my penis, but even that is not a big concern. Again, no pun intended.