Judging the beer that wasn't there.

[sudbuster]

The AHA annual NHC. It has been three years since I have been able to get an entry into that circus. This year I got in with one entry I thought was worthy. As so often happens in life, other duties trumped it, and I was not able to meet the shipping window when we returned, so the entry was not shipped. That did not incumber the judging staff at the event in any way. They judged it and sent me score sheets.
I try to enter my beers in at least two sometimes three different contests in order to gain a wider consensus of its worth. The scores returned usually vary widely, the worst being from 27 to 40. Before the above incident, I blamed the judges for such disparity. Now I am not so sure. Could such wide range of scores be the result of receiving someone else’s score sheets? How often does this happen? How would we even know?
Needless to say, my confidence in the system is at a very low point at this time.

 

[Teromous]

This is so hilarious. I'm wondering what happened. There's so much speculation that can go along with this, but needless to say it is very bad.

 

[duboman]

You should email Janis


Sent from the Commune

 

[machfive55]

I have wondered this very thing myself. I have entered comps and have seen scores vary from 20's to 40's on the same beer, same batch, etc. Now, I do understand that I could have shipped a bad bottle, or luck of the draw and the one I sent and they chose was oxidized or contaminated...but, by reading the descriptors I couldn't help but think they were not drinking my beer. So, yeah, a big contest, lots of beers, volunteers who may be inexperienced...it's not beyond thinking it could happen.

 

[beeber]

From the contests I have judged I would say that it is very unlikely that you would get some else's score sheets. The variance is a different story. And the order things are judged is a factor too.

 

[passedpawn]

Hit up Martin Brungard (mabrungard) on this forum. He's on the AHA governing committee now. Probably not as direct as going to Janis of course, but I think this would be a great first job for him in the new office

 

[Teromous]

From the contests I have judged I would say that it is very unlikely that you would get some else's score sheets. The variance is a different story. And the order things are judged is a factor too.

If not, then the implied alternative is far worse:

Judge 1: "Where's the score sheet for this guy?"
Judge 2: "I don't know, did we already sample it?"
Judge 1: "I don't remember..."
Judge 2: "Just fill out a score sheet and put it in the stack."

 

[mwilcox]

"No head. Very light color, with extreme clarity. Invisible, really"

seriously though, I don't get how this happens. Maybe if the pull sheet had all the entries including those that didn't arrive, but I would still expect the judges to copy down the judge number from the bottle, not just copy the next number from the pull sheet and assume the steward was handing them the correct bottle. I've never seen judges NOT look at the bottle. Weird.

afterthought: what was the score? if top 3, uh, congrats?

 

[Wynne-R]

You should return the sheets to the competition so they can fix it. Somebody didn’t get their sheets.

Mistakes happen, but they’re pretty rare. In your case, it was probably somebody with the same name, who entered the same style. Careless data entry, they didn’t check the address.

 

[GilaMinumBeer]

Best beer they never had.

 

[passedpawn]

Tastes great. Less filling!

 

[sudbuster]

Thanks for the replies. Some of them were a hoot . I thought about sending the sheets back, but then thought that might cause even more confusion. It's just a game anyway.

 

[passedpawn]

It's just a game anyway.

Wait... what?

 

[sweetcell]

Tastes great (or whatever). Less filling (in of paperwork)!

fixed that for you.

 

[MuddyCreek]

My very favorite beer judging competition is the one when I slide a pint glass over to a person who's never tasted one of my beers and their face lights up when they taste it.

They don't even give me score sheets.

Now admittedly, sometimes they also take a sip and politely put the glass down and start visiting with their friends. I can accept that too.

 

[kharper6]

Contests are actually just a bunch of guys that don't want to pay for beer and they typed out some generic responses to certain styles etc.

 

[Hello]

Contests are actually just a bunch of guys that don't want to pay for beer and they typed out some generic responses to certain styles etc.

I've entered two competitions and judged none, so my experience is limited, but this statement suggests that you really have no respect for anyone who truly tries to put forth the effort of judging a beer. What about BJCP certified judges? Are those just guys looking for free beer too?

OP: Not sure what happened but someone else mentioned in another thread that one or two of his 3 entires never made it despite all entires being shipped in the same box. They mentioned they would just judge a different beer that may not have any identifying owner, or something like that. That alone would make me lose confidence in what seemed to be a pretty big competition.

 

[Bsquared]

Needless to say, my confidence in the system is at a very low point at this time.

You should get involved, it's easier than ever to get going, you can take the online test, and start Judging as a provisional judge, you can volunteer and be a steward. You can do as little or as much as you want, but BJCP sanctioned contests, let alone NHC contests are crazy these days, and the people doing it are volunteers, and they are almost always short qualified judges.

As for scoring, there should not be more than a seven point spread between judges. As well, a beer entered in the wrong category should not get more than a 29 score, even if its would have been a 40+ beer in the correct category.

 

[chocotaco]

I propose that it was an ironic statement on the part of the judges about the state of "beer fandom" - where the beers that are hardest to get are the most highly prized.

 

[kharper6]

I've entered two competitions and judged none, so my experience is limited, but this statement suggests that you really have no respect for anyone who truly tries to put forth the effort of judging a beer. What about BJCP certified judges? Are those just guys looking for free beer too?

OP: Not sure what happened but someone else mentioned in another thread that one or two of his 3 entires never made it despite all entires being shipped in the same box. They mentioned they would just judge a different beer that may not have any identifying owner, or something like that. That alone would make me lose confidence in what seemed to be a pretty big competition.

sarcasm

 

[sudbuster]

You should get involved, it's easier than ever to get going, you can take the online test, and start Judging as a provisional judge, you can volunteer and be a steward. You can do as little or as much as you want, but BJCP sanctioned contests, let alone NHC contests are crazy these days, and the people doing it are volunteers, and they are almost always short qualified judges.

As for scoring, there should not be more than a seven point spread between judges. As well, a beer entered in the wrong category should not get more than a 29 score, even if its would have been a 40+ beer in the correct category.

I think I'll pass on that invitation. Judging is an expensive, thankless hobby. What's more, I'm too old to be dragging my buttt all over the country. The OP was to just share an experience with the community and it's effect on my viewpoint.

 

[JohnSand]

"Brews the Force, Luke. Feel the beer..."

 

[elkshadow]

I don't get competitions. I never did. Call me a hater but I never had any interest whatsoever in entering a competition. The whole idea of judging beer and giving out awards is just silly to me. It's like giving out awards for the best pie or the best BBQ or whatever. I don't lend much credibility to anything that's judged by people with a score sheet actually. I guess I don't see the point. If I like what I make, that's all that matters and I don't care if someone who has spent years tasting and studying a thousand different beers thought my beer was "buttery" or "too phenolic for the style" or whatever. Is the beer that wins the NHC comp best in show really the "best" homebrew in the country? Or does it win because it matches all the characteristics of the style that some judges have as check boxes on a score sheet? BLAH

The more I hear about the shortcomings of the AHA and the NHC (which is often) the less interested I become in the bureaucracy of homebrewing.

FWIW, I got a good laugh out of the OP.

 

[Darwin18]

OP - I would at least email Janis and let her know that you received score sheets for an entry that was never shipped. Someone out there is probably wondering where their score sheet went....

 

[Braufessor]

I guess I don't see the point. If I like what I make, that's all that matters and I don't care if someone who has spent years tasting and studying a thousand different beers thought my beer was "buttery" or "too phenolic for the style" or whatever.

I understand what you are saying, and I don't totally disagree with you. One of the obvious conditions of BJCP comps. is that beers are brewed "to style" which can, indeed, be limiting in some ways. And, you are also right in regard to the concept that there can be GREAT beers that don't do well, simply because they don't "fit" in the boxes right.

However, I disagree in some ways too. To say that just because"someone spent year tasting and studying beers and they 'thought' my beer was too buttery or phenolic"....... It is not that a quality, experienced judge "thinks" it is phenolic or buttery - it very like "actually is" buttery or phenolic. Most brewers (myself included) are biased toward our own beers. Most of our friends and family are nice, and error on the side of caution when giving us feedback.

I am not saying everyone should, or has to enter competitions. But, there is "a point" to doing it. My beers have improved more in 2 years of entering competitions and getting good feedback, from strangers, than they had improved in the 15years before that. Every judge, every competition is not great. All feedback is not perfect - but, taken together, there is a lot to be gained.

Saying "If I like what I make, that's all that matters" sort of reminds me of "The Allegory of the Cave." If the shadows on the wall are my reality, then why would I want someone to show me something more?

I am not saying that folks can't make great beer without entering comps.... they can. However, I will say this - I have tasted a fair amt. of homebrew made by people who do consistently well in competitions - I have never, ever gotten a beer from them that was "bad." They are almost all good-great. On the other hand, i have drank homebrew on many occasions from people who think there beer was awesome, and it is "good to them" - and it was somewhere between mediocre and horrid. I have been given beer that is "really, really good" and the reality is, it was "really, really infected and plastic flavored." I did not "think" it was - IT WAS.

Obviously I have no way of knowing how good your beer is. It may be awesome. But, one of the reasons people enter comps is to get feedback to sort through - some of which will help them see things in their beer, or explain flavors they could not pinpoint on their own. Good judges really can offer some great insight to brewers looking to learn more about improving their beers.

 

[cosmo]

A few days before the judging in PA, I got an email stating that they couldn't find my entries. I e-mailed back the tracking number and they said they found one that appears to be right, but lost two of my three beers. Not sure how that happened since all 6 bottles were labeled with the standard competition entry labels and were shipped in the same box. They asked me what the bottles looked like. Really? They look just like all the other bottles in the comp except they had my labels on them. Mybe they judged mine and sent the score sheets to you. Later, they said they found some bottles that they "think" are the right ones. Aren't the labels supposed to clearly identify what they are? After spending well over $100 on membership, entry fees and shipping, I have no confidence that they even judged the right beers.

 

[Wynne-R]

Wow. That is wrong on so many levels. It sounds like those guys can’t do anything right. The way it’s supposed to work is your bottles are given unique numbers which are correlated to your entry. That information is verified before the entry labels are pulled.

If you’re going to fix a problem, it has to be done when the label is still on the bottle, otherwise you’re screwed. It sounds like these clowns are guessing.

It doesn’t add up. If they lost the box, how would they know? If they lost the labels, they couldn’t fix it.

 

[agrazela]

I propose that it was an ironic statement on the part of the judges about the state of "beer fandom" - where the beers that are hardest to get are the most highly prized.

That would be true if he'd WON the contest with a beer that never showed up

 

[sweetcell]

It sounds like those guys can’t do anything right.

no it doesn't like that. let's keep a sense of proportion here, folks.

a judging site receives thousands - thousands - of bottles. that they screw up a few is unfortunate, it's not what we expect, but given the number and the fact that it's done by volunteers we shouldn't be completely shocked. and for every entry that is eff'ed up, what about the other 999 that aren't? you only hear about the exceptions, because 999 people chiming in "my entries went through fine" doesn't hold the sensationalism of "OMG they got 0.1% of entries wrong!!!!!!!11111".

 

[Wynne-R]

I was talking about cosmo’s entries, where apparently they screwed up 6/6 in ways I don’t understand. Same thing with the op, except he wouldn’t have known they screwed up if he had actually sent his entry.

Looking at the big picture, what is the acceptable error rate? It’s zero. Bluebonnet had 1300 x 3 bottles this year without those kinds of problems. NHC apparently can’t pull off 900 x 2 in at least two of it’s regionals.

Mistakes will be made, you have to have a process that catches the errors while they’re still fixable.

 

[TheZymurgist]

I was talking about cosmo’s entries, where apparently they screwed up 6/6 in ways I don’t understand. Same thing with the op, except he wouldn’t have known they screwed up if he had actually sent his entry.

Looking at the big picture, what is the acceptable error rate? It’s zero. Bluebonnet had 1300 x 3 bottles this year without those kinds of problems. NHC apparently can’t pull off 900 x 2 in at least two of it’s regionals.

Mistakes will be made, you have to have a process that catches the errors while they’re still fixable.

Do you know for a fact that no errors were made in the Bluebonnet competition? I think it's unreasonable to expect an error rate of zero. It sucks when things get messed up, but it's not like anyone is actually losing anything except their entry fee. If there was more at stake, then the expectation could be higher, but I think people should be aware of the possible risks when they enter a competition run by volunteers.

 

[Sir-Brews-Alot]

However, I disagree in some ways too. To say that just because"someone spent year tasting and studying beers and they 'thought' my beer was too buttery or phenolic"....... It is not that a quality, experienced judge "thinks" it is phenolic or buttery - it very like "actually is" buttery or phenolic. Most brewers (myself included) are biased toward our own beers. Most of our friends and family are nice, and error on the side of caution when giving us feedback.

I am not saying everyone should, or has to enter competitions. But, there is "a point" to doing it. My beers have improved more in 2 years of entering competitions and getting good feedback, from strangers, than they had improved in the 15years before that. Every judge, every competition is not great. All feedback is not perfect - but, taken together, there is a lot to be gained.

Saying "If I like what I make, that's all that matters" sort of reminds me of "The Allegory of the Cave." If the shadows on the wall are my reality, then why would I want someone to show me something more?

I am not saying that folks can't make great beer without entering comps.... they can. However, I will say this - I have tasted a fair amt. of homebrew made by people who do consistently well in competitions - I have never, ever gotten a beer from them that was "bad." They are almost all good-great. On the other hand, i have drank homebrew on many occasions from people who think there beer was awesome, and it is "good to them" - and it was somewhere between mediocre and horrid. I have been given beer that is "really, really good" and the reality is, it was "really, really infected and plastic flavored." I did not "think" it was - IT WAS.

Obviously I have no way of knowing how good your beer is. It may be awesome. But, one of the reasons people enter comps is to get feedback to sort through - some of which will help them see things in their beer, or explain flavors they could not pinpoint on their own. Good judges really can offer some great insight to brewers looking to learn more about improving their beers.

Exactly. That was the main reason I started entering comps. To improve my beer. I can't judge the quality of my beer based on family and friends opinions or even my own. Even if they didn't like a particular beer for whatever reason, they could never tell me WHY, or what might be wrong with the beer. I'd say 90% of the time when a judge noticed an off flavor in a beer I entered, I would re-taste my beer and get the same flavor that I may not have noticed initially. Or a few times I knew there was something wrong with a beer, but didn't have the knowledge to pinpoint the problem, so I'd enter it in a few competitions and almost always find the solution to my off-flavor based on the judges comments.

And I agree I have had so many homebrews that the brewer thought was so awesome, only to find that it had major flaws, and I'm not just talking stylistic flaws, but infections or off-flavors. This will probably be a controversial statement, but in my experience, brewers who enter comps are more flexible and capable of adjusting their process to improve their beers and by and large brew better beer overall than those who "like their beer and don't care what anyone else thinks." You need to at least take it to your homebrew club and get constructive feedback that way. Of course many people are plenty happy with mediocre beer and have no desire to strive for perfection with their homebrewing, which is fine too, I suppose. That's the beaut of homebrewing....you can do what you want.


All that said, seeing mistakes like these is definitely disheartening. I had one comp where my kolsch had gone on to the BoS round, but they broke my second bottle. Unfortunately, by the time they contacted me for another bottle I had consumed the whole batch, so my beer never got judged for BoS. That sucked!

 

[Wynne-R]

Do you know for a fact that no errors were made in the Bluebonnet competition?

I never said that, I said we didn’t have any of the egregious errors described on this thread. I was talking specifically about check-in. At the end of the day we had 3 x 54 ½ cases of beer that passed QC. I am highly confident we did it right. This was the 28th year for it, we’ve gotten pretty good at it. Nothing is lost or broken. Everything is checked and double checked and checked again before the labels are stripped.

Getting a scoresheet with no bottles could be a simple address mix-up, but cosmo’s deal involves multiple mistakes. How do you lose a box, then find part of it and guess the rest ? Is there a big old pile of unknown beers that they randomly assign to beers they lost? That’s crazy.

 

[elkshadow]

Looking at the big picture, what is the acceptable error rate? It’s zero.

Like I said earlier I don't really have a dog in this fight but pretty much this.^^^^

 

[elkshadow]

I can't judge the quality of my beer based on family and friends opinions or even my own.

You can't judge the quality of your beer based on your own opinion?

This will probably be a controversial statement, but in my experience, brewers who enter comps are more flexible and capable of adjusting their process to improve their beers and by and large brew better beer overall than those who "like their beer and don't care what anyone else thinks."

It's not that I don't care what other people think. It's that I'm the ultimate judge of my own beer. I've been drinking good beer for more than 20 years. I know when a beer is "good" or not. Paying an entry fee so some third party can verify what I already know or think is a waste of my time.

Believe me I get the whole "my beer got better after I read my score sheets and adjusted my process" argument. I've seen it a million times on brewing forums.

Competitions help some brewers get better and that's great. They are just not worth the time and effort for me.

 

[hunter_le five]

Looking at the big picture, what is the acceptable error rate? It’s zero.

You must lead a very disappointing life.

An error rate of zero is a statistically unrealistic expectation to have for anything involving this many people.

 

[Flipadelphia]

You can't judge the quality of your beer based on your own opinion?



It's not that I don't care what other people think. It's that I'm the ultimate judge of my own beer. I've been drinking good beer for more than 20 years. I know when a beer is "good" or not. Paying an entry fee so some third party can verify what I already know or think is a waste of my time.

Believe me I get the whole "my beer got better after I read my score sheets and adjusted my process" argument. I've seen it a million times on brewing forums.

Competitions help some brewers get better and that's great. They are just not worth the time and effort for me.

Amen. I like my beer, and am always learning something new and attempting to get better, but I don't need to pay some judge to tell me that. I'm not a professional brewer, so I don't need that kind of critique.

I think people are kidding themselves if they don't admit that one of the main reasons people get in these competitions is because men enjoy defeating other men in competition. Women too.

I just have no interest in that. It's just beer.

 

[passedpawn]

Amen. I like my beer, and am always learning something new and attempting to get better, but I don't need to pay some judge to tell me that. I'm not a professional brewer, so I don't need that kind of critique.

I think people are kidding themselves if they don't admit that one of the main reasons people get in these competitions is because men enjoy defeating other men in competition. Women too.

I just have no interest in that. It's just beer.

Why go into a thread talking about... let's say baseball, and announce to everyone that you find the activity useless.

 

[Flipadelphia]

Why go into a thread talking about... let's say baseball, and announce to everyone that you find the activity useless.

I don't know. I hate the Reds.

 

[Spartan1979]

Contests are actually just a bunch of guys that don't want to pay for beer and they typed out some generic responses to certain styles etc.

You've obviously never judged or organized a homebrew competition.

As far as judging goes, for every good beer you judge, there are 2 or 3 that aren't that good. I want to spend my day judging beer so I can get 3 oz of a good beer.

Organizing a competition is a lot of work. I've done two now with about 300 entries each and it certainly wouldn't be worth it for free beer. I probably put in at least 50 hours of my time this year.