I feel like this experiment is not receiving the attention it deserves. I hope it'll move to its own thread so more people can find the results
So in a first very simple-minded check on the preferential data I just looked, for every malt and every rating between 1 and 7, at the number of participants that ranked this malt at k or better. I apologize for introducing mathematical notation, but I find it hard to express references without using variables. So I will denote that number as pref
(M, k), where
M denotes a malt. Note that pref(M, 7) = 35 is the number of participants, for any malt M.
I will say that
the participants preferred malt M over malt N if the number pref(M,k) is greater-than-or-equal-to pref(N,k)
for every k and there
exists some k with pref(M,k) > pref(N,k). This is a rather strict notion of 'preference', and I suppose that several sensible weaker notions exist, but I think it's fair to say that malt M scored better than malt N in this scenario.
So, anyways, under this strict notion of preference we only have a few clear results:
- Best Malz Pils beat all other malts
except for Dingemans, as a total of 6 participants rated Best Malz at 6 or 7, whereas only 3 participants gave such a low score to Dingemans.
- All malts except for Barke and Sugar Creek were preferred over Mecca Grade Pelton.
The sequences pref(M, k) for the various malts are also quite interesting, I think, so here they are:
Best Malz Pils: [14, 23, 24, 27, 29, 33, 35]
Weyermann Barke: [2, 4, 14, 23, 28, 32, 35]
Avangard: [3, 8, 15, 19, 21, 28, 35]
Weyermann Floor Malted Bohemian: [5, 10, 12, 19, 26, 30, 35]
Sugar Creek: [4, 8, 14, 21, 25, 33, 35]
Dingemans Belgian: [4, 11, 19, 20, 32, 33, 35]
Mecca Grade Pelton: [3, 6, 7, 11, 14, 21, 35]
From this we can read off, for every malt M, the minimum rank r such that at least 50% of all participants ranked it r or better:
Best Malz Pils: 2
Weyermann Barke: 4
Avangard: 4
Weyermann Floor Malted Bohemian: 4
Sugar Creek: 4
Dingemans Belgian: 3
Mecca Grade Pelton: 6
This is sort of in-line with our result from experimenting with "preferred over": for Barke, Avangard, Bohemian and Sugar Creek, preferences are essentially split down the middle (r=4), Best Malz is notably better (r=2) and Dingemans in between (r=3), whereas Mecca had a notably worse result (r=6).
If, instead of seeking the approval of 50% of all participants, but of 80% of them, we get the following table:
Best Malz Pils: 5
Weyermann Barke: 5
Avangard: 6
Weyermann Floor Malted Bohemian: 6
Sugar Creek: 6
Dingemans Belgian: 5
Mecca Grade Pelton: 7
Now, we have in shared first place Best Malz, Barke and Dingemans. While Barke was only rated among the top 2 by 4 participants, many people felt it was at least somewhat good, and only 7 put it in the two bottom ranks.
For the BJCP table, the results are indeed a bit different:
- All malts except for Barke were preferred over Mecca
- Best Malz was preferred over Mecca, Sugar Creek and Avangard
- Weyermann Bohemian was preferred over Sugar Creek, Avangard and Barke
In fact, Best Malz and Weyermann Bohemian have quite similar results here, whereas Best Malz was clearly preferred by the large sample.
For more sophisticated analysis of the results (such as "80% of tasters preferred malt M over malt N") I'd need the 'raw' preference data of each participant.