I'll just throw a shortened version of my previous critiques of Brulosophy out here:
Triangle Tests: done properly (and the Brulosophy people don't) they are the gold standard in this type of testing analysis. We have no idea what the sample of testers generalize to, i.e., all beer drinker, just IPA lovers, casual drinkers, hipsters, who knows? We also don't know what they were eating and/or drinking prior to the test, which could damage palates and make people unable to perceive differences they might otherwise be able to perceive.
Done properly, you randomize the order of the samples and hopefully have a balanced design so every order is represented equally. In other words, if you are testing beers A and B, you have six orders: AAB, ABA, BAA, BBA, BAB, ABB. Testers should try them in the order listed, so if there's an ordering effect to the results, it hopefully balances out.
It's hard to do it right, in that it's a pain. You have to hope you can get people to do the tests before they drink or eat anything which can screw up their palates, and then hope they'll actually listen and try the beers in the order you give them.
I've done it. It's hard.
*****************
And here's the part that gets me--yeah, it may be true that, statistically, more than an expected random number could identify the odd-beer-out. But in most of the Brulosophy experiments, a majority of testers
could not tell a difference. And yet they're told to guess and, if they guess right, they then are asked for their opinions on the beers--after, by
guessing, they got lucky.
But assume for a moment that the ones who identified the proper odd-beer-out actually could perceive the difference. What we have is two beers the majority of beer drinkers (assuming the testers constitute a reasonable sample of the beer-drinking public)
cannot tell apart. After years of thinking about this, I'm still trying to wrap my head around it.
***************
I met Marshall at a BYO Brew Boot Camp about 20 months ago. Nice guy. And I think the experiments are, generally, well done. The message to everyone should be to take what you see at Brulosophy and then do your own experiments and see what you think.