My girlfriend has a tendency to leave milk around the house. A half drank glass here, a non rinsed out jug in the trash, etc. As a result, I am very sensitive to the sour milk smell. I have to follow it with my nose all around the house until I find and dispose of the culprit.
I also have had several people tell me that all milk stouts have a twinge of that aroma to them and ever since I've been living with her I've started to notice that as well. If its minor and it dissipates (or perhaps you're just getting used to it) over time then I would bet you're just highly sensitive to that aroma just like I am.