I've only ever had one neighbor come over while I was brewing. It was a nice day and I was doing it on the back patio as opposed to my normal garage brew. He and his wife were out gardening and when I added the first hops, he came over to take a look. "MMMM, love that smell, when we first got married my wife and I lived down the street from the Budweiser Brewery. I loved it when the added the hops."
Other than that, I've had zero interest from anyone. I suppose I've always been a Black sheep in a way, but not because of my brewing. When my wife and I first moved into the neighborhood 15 years ago, we were really young compared to everyone else. Most of the people on the Block, we were closer to their kids' ages than to the parents. I'm always very helpful and friendly around the neighborhood, but I think there was just that disconnect because of age.
As a reference, I've actually become a little more social with another of our neighbors because his grand kids are the same age as my kids. So, when they are in town, the kids play together a lot.