Boil Length xBmt results: 30 vs 60 minute

[Brulosopher]

Since I started all grain brewing, I've always wondered if it was really necessary to boil for 60+ minutes. I finally put this one of the test and the results are in!

http://brulosophy.com/2015/03/11/the-impact-of-boil-length-ale-exbeeriment-results/

 

[ajdelange]

I had a quick look at the results. There are apparently some continuing misunderstandings of what triangle tests are and how their results are interpreted. Here you have two beers, A and B and two hypotheses:
H0: The beers are indistinguishable
H1: The beers are different

You can't prove H1, in fact you can't prove H0 either but you can gain support, or lack thereof, for it by asking people, given 3 samples, 2 of which are the same, if they can pick the odd one. Under H0 they can't and so must guess. With rolling dice as the basis for choice of the odd beer it's not likely you'll get a lot of correct answers. If you do get a lot of correct answers then it seems that there isn't much support for H0 and you conclude that H1 may be valid: that the beers are indeed distinguishable.

The numbers in the table are the probability that n out of m panelists will make correct identifications when they are rolling dice. For example, with 24 panelists as you had, the probability that 9 will be correct by rolling dice is 40.6%. That's almost even odds (for 8 correct the probability is greater than even at 57.6%). Thus with 8 correct there is quite a bit of support for the null hypothesis. There certainly isn't much for H1 and you conclude that the beers are indeed indistinguishable or really, and this is an important point, that they are indistinguishable by your panel. It is well to keep in mind that a triangle test is a test of the panel, not the beer.

The other thing that caught my eye is that you presented triplets with two of the long boiled beer and one of the short boiled. There are 4 possible triplets, AAB, ABA, BAB, and BBA. Panelists must be presented with a triplet chosen at random from this set. The math from which the probabilities are computed is based on that assumption.

 

[Brulosopher]

That sounds really interesting... and a bit too complicated for my simple mind. Either way, thanks for the info! Cheers!

 

[ajdelange]

Maybe explaining it in the language of the statistician confuses you but the test itself is very simple and at the same time powerful. If I understand what you set out to investigate it is the only way to answer your question, "Are two beers brewed with everything else but the boil length the same distinguishable?" If that's not the question then ignore what follows.

I propose the 'Triple Blind Triangle Test' with the 3rd level of blindness being that you can use it successfully by blindly following this prescription:

1. Empanel a group of tasters and find a reasonable place to conduct you tasting (not the fermentation room in a fish sauce factory).

2. Prepare three cups (ideally opaque or blue or something that obscures subtle color differences) numbered 1, 2 and 3, for each taster by flipping two quarters. If the first quarter lands heads then put beer A in cup 1 and B if it lands tails. If the second quarter lands heads put A in cup2 and if it lands tails put B. Fill cup 3 so that the set contains 2 of one beer and one of the other. E.G. if you got two heads you'd have A in 1 and 2 and so B should go into cup two. Obviously you need to record what you did.

3. Have someone other than the person who filled the cups (this is the second level of blindness) present them to the panelists and instruct the panelists to taste and smell them in any order and record the number of the cup that is different from the other 2. Panelists know that there are two the same in each set but obviously they don't know if they have two A's or two B's or which cups they are in. This is the first level of blindness.

4. Score the results and enter the number of correct identifications and panel size into the ASBC table or my spreadsheet or a formula that calculates the corresponding probability.

5. Make the statement "A double blind triangle test showed the beers to be distinguishable at x% confidence level." where x% is the probability from the table or spreadsheet.

Now you may really not understand what that means (but I'll bet that if you think about it for a bit you will) but it is meaningful to other people who do and there are quite a few of us out there. Where I get upset is when someone says "We did a triangle test" and then, upon examination of what they did, find out that it wasn't really a triangle test. That's what I'm hoping to eliminate here. It's not a difficult test but you do have to follow the procedure.

Finally we can make a couple of comments on how to interpret the results blindly. The statement "A double blind triangle test showed the beers to be distinguishable at 40.6% confidence level." (9/24) isn't very exciting. It means that you'd get the results you got about half the time using 24 monkeys with 3 buttons a press of any one of which gives a banana. OTOH had 13 panelists chosen the correct beer your statement would be "A double blind triangle test showed the beers to be distinguishable at 2.8% confidence level." This means that you monkey panel could only do this in 3 out of 100 tests so the odds are pretty good that your panel was better than monkeys and, of course, that the beers were distinguishable because if they aren't even 24 Morton Mielgaards couldn't tell them apart. So you blindly accept than any probability under 5% as 'significant' (at the 5% level) and that any probability over 5% isn't.

So don't give up on Triangle testing. Just do it right. It's not that hard. But if you do decide to give it up don't call what you do in its stead Triangle.

 

[DMosPickles]

So do you have to use 2 different quarters or can you just use one twice :fro:

Just kidding. Kick ass breakdown of triangle studies by the way. I didn't expect to get that kind of detailed statistical info on HBT! I often wondered how to properly interpret the results.

 

[brettwasbtd]

Aj, What is the significance of having a second person present the three beers to the testee?

Also, and maybe I read incorrectly, what happens if you get head on he first, tails on the second, do you flip again for the third glass?

 

[ajdelange]

So do you have to use 2 different quarters or can you just use one twice :fro:

If you only have one you could borrow the other from one of the panelists. Or you could use a nickel or half dollar for the other coin.

 

[ajdelange]

Aj, What is the significance of having a second person present the three beers to the testee?

If the person presenting the beers is blind (doesn't know what's in the cups) then he can't give any cues to the panelists. Thus one person should fill the cups off somewhere by himself and then give them to 'stewards' to present. He should not know which beer is A and which is B. If he does then the tests should not be scored by him but rather by someone who only knows the triplet pattern given to each panelist. Anyone with knowledge of which beers are in which cups should stay out of the room where the panelists are located. I guess I didn't make that clear. The panelists are blind, anyone they come in contact with is blind and the person counting up the number of right answers is blind so the test is double blind.

Also, and maybe I read incorrectly, what happens if you get head on he first, tails on the second, do you flip again for the third glass?

No need for a third flip. Here are the four possible outcomes from two tosses of a coin and the corresponding triplets:

HH: AAB
HT: ABA
TH: BAB
TT: BBA

 

[masaba]

I have one comment and a question:

Comment: From reading the Wikipedia page on the triangle test, it looks like it is also important that the panelists cannot speak to each other about the test while they are taking it, and that the panelists cannot observe each other taking the test.

Question: Couldn't one person pour the beers and 'present' them to the panelists as long as this procedure is blind? Ie. the presenter goes into a room by himself, pours the beer, sets it in a triangle on the table, and leaves the room. Then, the panelist walks into the empty room with the cups on the tables, tastes them, and then puts his response on a piece of paper.

 

[wilcode190]

Don't forget to add in a sensory deprivation chamber! Only way for conclusive results.

 

[ajdelange]

Comment: From reading the Wikipedia page on the triangle test, it looks like it is also important that the panelists cannot speak to each other about the test while they are taking it, and that the panelists cannot observe each other taking the test.

The probabilities are calculated based on the assumption that the panelists choices are completely independent of one another. Clearly complete independence requires complete isolation but this would not be practical. In most laboratories set up to do this in breweries my understanding is that each panelist sits in a booth separated from the other booths by partitions and facing a common wall equipped with little doors through which the samples are presented.

Question: Couldn't one person pour the beers and 'present' them to the panelists as long as this procedure is blind? Ie. the presenter goes into a room by himself, pours the beer, sets it in a triangle on the table, and leaves the room. Then, the panelist walks into the empty room with the cups on the tables, tastes them, and then puts his response on a piece of paper.

There are lots of possible ways to do this. As long as the taster is blind, everyone he interacts with is blind and the scorer is blind you are OK.

 

[Hamaki]

So...short of repeating all the various brews and running new tests as described which would you say is a more accurate representation?

The tests run by Brulosopher do not strictly adhere to the parameters for a triangle test but are close enough to have meaning

or

The results cannot be relied on at all because the deviation from accepted standards is too great

 

[Oginme]

While the triangle testing that Brulosopher conducts may not be strictly controlled, it goes pretty far above what other people do to determine the effects of altering a single parameter on the product than most of what I have seen published. For this, we should all applaud his efforts to bring some type of order to the evaluation of experiments.

Most of the comments on brewing 'facts' that I have told or read about since I began brewing several years ago have been completely anecdotal and uncontrolled comparisons.

Remember also, as Marshall states in his blogs, this is only one test result. He encourages additional follow-up experiments to verify his results.

 

[ajdelange]

So...short of repeating all the various brews and running new tests as described which would you say is a more accurate representation?

The tests run by Brulosopher do not strictly adhere to the parameters for a triangle test but are close enough to have meaning

or

The results cannot be relied on at all because the deviation from accepted standards is too great

Let's assume that the deviations made no difference IOW that these results are from a valid triangle test. What do they really tell us? That there is about an even chance that a group of monkeys could have turned in the same score. That says that there isn't much support for either hypothesis which certainly damns the hypothesis that longer boiled beers are better by faint praise. With a different panel one might have gotten a different result. That's why I keep emphasizing that you are testing the panel as well as the beer.

Now if 13 had shown preference (probability that monkeys could do this of < 5%) then we'd conclude that there is indeed something about the beers and the panelists that made them able to tell them apart and we would then start to focus a jaundiced eye on the deviations from the procedure. Were biases introduced by failure to fully randomize? Were servers, scorers and panelists adequately isolated? But at the 40-50% level (the level of minimum information from the test) it would be hard to level any such accusations.

When I say the 50% level is the level of minimum information gain it makes it sound as if even a completely by the book triangle test would be a waste of time. That's not so. Are the longer brewed beers better? Maybe. Again we note that this is pretty faint support for the idea that the longer brewed beers are better.

So in this case I say the former. All this has convinced me that the extra half hour boil didn't make an appreciable difference in this case. But I also know that boiling takes energy and that energy costs money and that no brewery will spend money it doesn't have to so I'm guessing that out there in the industry there are lots of triangle tests comparing shorter boiled to longer boiled beers that show longer boiling is of benefit.

 

[ajdelange]

While the triangle testing that Brulosopher conducts may not be strictly controlled, it goes pretty far above what other people do to determine the effects of altering a single parameter on the product than most of what I have seen published. For this, we should all applaud his efforts to bring some type of order to the evaluation of experiments.

And I do. The purpose of my post was to point out a few shortcomings in the procedure. These, as stated in earlier posts, are very easily resolved such that future tests can indeed be completely valid triangle tests.

 

[misterkidd]

Thanks for sharing, Brulosopher! Informative, interesting and entertaining as usual! ^_^

 

[jdubdvdt]

I didn't read too much into the procedure, but did you mash one batch and then split it into two for the boil? Or were they two completely separate beers using the same recipe? I see where you have two refractometer readings, so I presume you made two completely separate beers.

 

[Brulosopher]

Hey everyone!

First off, AJ is a genius and a legend, I can't argue with his recommendations, they're beyond on point... he obviously knows what he's talking about and I absolutely appreciate his suggestions.

That said...

My thing is "sloppy citizen science," as Drew Beechum once referred to what it is I do. I'm not necessarily trying to "prove" anything, but rather practically test a few of the many "facts" us homebrewers have been told about since starting. I try to make a point to say in each article that my results are but a single point of data and that they shouldn't be used to justify sh*tty brewing techniques. If ever I become a professor in a brewing school, which is highly doubtful, I'll certainly utilize a more controlled approach.

Also... cheers... I'm on pint #4 of my delicious Vienna lager

 

[madscientist451]

Putting all the statistical arguments aside for the moment, the question that comes to my mind is this: Have the modern, computer controlled malting methods (Briess in this experiment) made the standard 60 minute boil obsolete for some beer styles? Or was it never really necessary to begin with? There's already a thread about a 15 minute cascade ale made with extract and the hop bursting technique. Can an all grain brew be shaved down to 60 minutes? 30 min. full volume BIAB then 30 minute boil?