I like the work being done by the guys at Brulosophy and I'll continue to follow them. There is one thing I would criticize about they way they perform their tests though.
I understand the idea of a random sample group, but I don't think they should be using group of random people for their tests. When testing for a difference in the presence of DMS for example, since DMS can't be detected by everyone I think the testers should be screened first and only those who are sensitive to DMS should be included.
I don't care if I can do something wrong or less than ideal and still have a beer that lots of people like, I can do that now. Investigating the science of beer should have us striving for a more perfect beer that virtually everyone will like, not just a beer that is good enough that most people won't notice some of the minor flaws.
In my opinion they should go out of their way to make sure the people being used as taste testers are competent tasters with decent beer pallets. It would be great if they worked up a profile on a bunch of testers to determine what variables those testers can be used for and expected to give meaningful results for.
Since Brulosophy is trying to test a potential change in a single variable by using people as testers instead of lab equipment to simply read the amount of the variable present in the sample (which I think is an awesome way to get real world insight and test the significance of a change), they need to make sure their testers are "calibrated" to give input that is more than just noise.
I think this explains why so many experiments seem to show no significance when you would expect otherwise. The intentional DMS dosing experiment (
http://brulosophy.com/2016/09/05/off-flavor-series-pt-1-dimethyl-sulfide-dms-exbeeriment-results/) and the mash temp experiment resulting in a clearly measurable difference in FG (
http://brulosophy.com/2015/10/12/the-mash-high-vs-low-temperature-exbeeriment-results/) come to mind as examples where I suspect too many of the people used as taste testers were... less than ideal candidates.