+1 on cougar idea, I was going to suggest to buy a bigger dog.
I have had this very same problem though at my old house. I talked with my neighbor a few times and he didn't stop. My best solution was to just call animal control when I noticed it out and they took it away. He got fined for it not being on his property and un-leashed. And it never happened again. It sucks to have to do that with neighbors, but some people just aren't respectful.