Certainly a triangle test can be in error, but to reach statistical significance, it accounts for those who are guessing, that’s why you need to reach 11 out of 20 and not just 7 out of 20. Sure when something is really different the significance is more certain and less likely to be in error, but that’s the point of using widely agreed upon statistical standards. If significance is found, it is reliable that a difference is there even if subtle. I know that Marshall doesn’t feel like there is much meaning in the preference data, because what one person prefers is wildly different than another. From my perspective, it is important. I feel like the preference data is about as close to a good beer yardstick that we can give to a future brewer. What would be interesting is if we could somehow find how each of these flavors could be found to be distasteful in one context and preferred in another, much like a musical composition of notes. In that regard we could presumably achieve a beer opus of sorts.
To take this idea a step further, I propose a true robo-brew, set in a nitrogen purged room, ran by an AI. The AI will be fed a large number of malt types, hops, yeast and water salts and start churning out beers. Anyone that comes to Robo Brewery is given as much free Robo beer as they can drink, three at a time and then surveyed for each flight on difference and preference. The AI will then get the feedback and produce better and better beers. At some point it will find the penultimate beer. Plot twist, it tastes just like Bud Light.