Beer Judging Rant

[jacksonbrown]

Is it just me or are competition judges going off completely arbitrary guidelines? I enter competitions mostly for feedback on ways to improve, although accolades are always a nice side note. I recently got a 3rd place ribbon for my Saison and was very excited to get the score cards back to see how I can improve on it. One of the cards was completely illegible due to horrid hand writing. But some of the notes the other two judges left just didn't add up. "Very good beer. Refreshing and to style" 3/10. The notes lead me to believe they enjoyed it, the points tell a different story. While another judge found it to be "an interesting beer that lacks the refreshing quality of the style." Actually, most of the comments between those two judges are contradictory and make me wonder if they were scoring two different beers.
My other entry, a Pumpkin Spice Ale which was submitted in the Spice/Herb/Veg category (21A) got such comments as "not what I was expecting in the style." It's a pretty broad, vague category, could you be more specific please?

That's just from my most recent entries. I've gotten back comments from judges that specify things they're looking for even if those characteristics aren't to style. As thought they have their own expectations of a style rather than BJCP guidelines (which as a BJCP function I would think they should have to follow).

Anyway, that's my rant. Am I alone here, or does anyone else get back these fairly unhelpful comment cards?

 

[McKBrew]

Never done a competition. I'd be pretty mad if I received feedback like that though. What a waste.

 

[Tonedef131]

Below their name on the scoresheet there should be a judge number if they are actually BJCP. If they aren't BJCP, well...there is your answer. If they are I would contact the competition organizer and let them know that you couldn't even read your scoresheets. They should be within 5 points of eachother, but some competitions allow up to 7 points of difference if they simply can't agree.

 

[smizak]

I'm beginning to wonder if the supply of qualified judges is way short of the demand for competitions these days. I've noticed a significant increase in the amount of entries from three or four years ago compared to today.

I plan to resolve this situation for my part by taking the BJCP exam in March.

 

[weirdboy]

Yeah that is why I have pretty much written off BJCP competitions for a while.

It is very frustrating to get back such crappy feedback, especially after waiting weeks and sometimes months for your scoresheets. If I want people to give me useless feedback I can do it for less than $7 + shipping.

Also I disagree philosophically with the way BJCP competitions are scored. It is painful to see someone scored low because their excellent, BOS-quality beer was accidentally placed in the wrong style category, or see people enter a fantastic beer that is just slightly out of style and get dinged for it, and also fail in the catch-all category 23 because it wasn't experimental enough.

 

[chefmike]

I plan to resolve this situation for my part by taking the BJCP exam in March.

If more people took this attitude, the world would be a better place.

Get involved. Its a hobby, this is how to make it work better.

And what a rough way to spend time for anyone who likes beer...:rockin:

 

[smizak]

If they aren't BJCP, well...there is your answer.

That is SOOOOOO evident on the score sheets I've have received. I'll get one sentence comments in a judging category like, "appropriate to style", or "possibly under-attenuated", and that's it. I'll take a look at the judge credentials, non-BJCP.

 

[bubbachunk]

At least you got something? If they are not BJCP its a complete toss up. I had a friend get a completely blank score sheet, only numbers. If the non BJCP judge(s) is unsure they should consult the others with them. I have done a few and realize I am only semi competent, that said I worked with my BJCP judge and learned a ton.

 

[david_42]

When you consider 80% of homebrewers are hopheads, you can understand why they might have trouble judging the other 200 styles.

Seriously, it is very easy to judge a beer to the style guide. Even styles you don't like. But when you get to the 11th, or 15th beer in a flight or are judging a second flight, tasting anything can be a real problem. The last HoV competition had almost 300 beers. Attempts were made to have at least one BJCP judge and one experienced non-BJCP judge on each flight, but it was a stretch and the stewards were shanghaied into assisting. And this is a region that has an major concentration of homebrewers and certified judges. We even had judges coming down from Seattle.

 

[DBbrewing]

Is it just me or are competition judges going off completely arbitrary guidelines? I enter competitions mostly for feedback on ways to improve, although accolades are always a nice side note. I recently got a 3rd place ribbon for my Saison and was very excited to get the score cards back to see how I can improve on it. One of the cards was completely illegible due to horrid hand writing. But some of the notes the other two judges left just didn't add up. "Very good beer. Refreshing and to style" 3/10. The notes lead me to believe they enjoyed it, the points tell a different story. While another judge found it to be "an interesting beer that lacks the refreshing quality of the style." Actually, most of the comments between those two judges are contradictory and make me wonder if they were scoring two different beers.
My other entry, a Pumpkin Spice Ale which was submitted in the Spice/Herb/Veg category (21A) got such comments as "not what I was expecting in the style." It's a pretty broad, vague category, could you be more specific please?

That's just from my most recent entries. I've gotten back comments from judges that specify things they're looking for even if those characteristics aren't to style. As thought they have their own expectations of a style rather than BJCP guidelines (which as a BJCP function I would think they should have to follow).

Anyway, that's my rant. Am I alone here, or does anyone else get back these fairly unhelpful comment cards?

I agree 100%. The last comp I sent beer into I was very dissapointed with the feedback that I received. I sent in a Christmas Ale and I thought it was the best beer that I have ever made and everyone in my homebrew club loved it and our newest member liked it so much that he brewed this recipe for his 1st brew, anyway this is what the judges said, it scored a final score of 16.3 WTF? One judge (novice judge) said Hard to drink due to Astringentcy, one said (a non BJCP judge) Solventy, nail polish remover comes to mind, and the last judge (a National BJCP) said slight phenol/plastic aroma. This beer is 10.1% and I tried it while reading the score sheets and can't taste or smell any of the things they are saying, so I think I am going to have to take the exam so I can atleast be able to pick up these flavors and aromas. No one that has tried this beer can pick up any of the things the judges are saying.

 

[smizak]

Also I disagree philosophically with the way BJCP competitions are scored. It is painful to see someone scored low because their excellent, BOS-quality beer was accidentally placed in the wrong style category, or see people enter a fantastic beer that is just slightly out of style and get dinged for it, and also fail in the catch-all category 23 because it wasn't experimental enough.

I hear this a lot. I think what has to be understood is that they are judging your brewing prowess based on your INTENTION. That's why it is stressed to print beer style categories and numbers, and also to explain what makes a beer a specialty/spice/vegetable/etc... If the beer is slightly out of style, it may make sense to place it in 23 and explain exactly why, or even enter the same beer in different categories if you're unsure.

That being said, we all get goofy score sheets sometimes, all the exams in the world are not going to perfect a system that in the end is based on subjective decisions.

 

[KYB]

Yea I feel like it's hit or miss. My Foreign Extra Stout only had 1 BJCP Certified judge, who gave me a 43. Another had a label with his name (gave me a 40), the other just wrote his name down (gave me a 35). BJCP guy said it was medium bodied, label guy said it was full bodied, 35 guy said it was light bodied. Gotta love contradicting feedback! The certified judge actually works at one of the stores I work at, but different location, so I may try to get a hold of him and see if he can judge mine from time to time.

 

[Jim Karr]

A lot of time, people who are judging something may be at a lack of words to describe what they taste or perceive. Then, they compensate by just pulling some "beer term" out of their butt, whether it applies or not.

They get by just by saying SOMETHING.

 

[ericd]

A lot of time, people who are judging something may be at a lack of words to describe what they taste or perceive. Then, they compensate by just pulling some "beer term" out of their butt, whether it applies or not.

They get by just by saying SOMETHING.

So true. I have a friend like this. So annoying.

 

[bierhaus15]

I enter a lot of competitions and have come to realize that:

A) Most judges will score an out of style, hoppy beer better than an in-style malty beer.

B) Some judges shouldn't have passed the BJCP exam - like the dude that said that my witbier needed more hoppiness and suggested a late addition of centennial. (I actually protested that one).

C) If your beer doesn't employ the "beat them unconscious with the flavor hammer" approach, don't enter it. This also applies to well balanced beers.

Though in all seriousness, I have found most judges to be relatively impartial when judging and most provide good advice on how to improve your beer. However, I am a firm believer that a beer on the high end of the style guideline will do MUCH better than one on the low end. Now when I enter competitions, I brew beer specifically for that comp. I try to exaggerate flavors and aromas that the judges are looking for and minimize fruity esters to keep the beer very clean and straightforward. So far its been working...

 

[DBbrewing]

Yea I feel like it's hit or miss. My Foreign Extra Stout only had 1 BJCP Certified judge, who gave me a 43. Another had a label with his name (gave me a 40), the other just wrote his name down (gave me a 35). BJCP guy said it was medium bodied, label guy said it was full bodied, 35 guy said it was light bodied. Gotta love contradicting feedback! The certified judge actually works at one of the stores I work at, but different location, so I may try to get a hold of him and see if he can judge mine from time to time.

A 43!!! By a certified judge, that must be one hell of a good beer, congrats!!!

 

[McKBrew]

If it is an issue with judges having too many flights and not enough judges then maybe the competitions need to restrict the number of entries.

 

[overflow]

When you consider 80% of homebrewers are hopheads, you can understand why they might have trouble judging the other 200 styles.

Seriously, it is very easy to judge a beer to the style guide. Even styles you don't like. But when you get to the 11th, or 15th beer in a flight or are judging a second flight, tasting anything can be a real problem. The last HoV competition had almost 300 beers. Attempts were made to have at least one BJCP judge and one experienced non-BJCP judge on each flight, but it was a stretch and the stewards were shanghaied into assisting. And this is a region that has an major concentration of homebrewers and certified judges. We even had judges coming down from Seattle.

+1. In my experience entering competitions with non BJCP judges/inexperienced brewers/general populous, a friggin IPA _ALWAYS_ wins. I guess it's because it will stand out on the taste buds, even after drinking 15 other types of beers.

 

[Revvy]

Yeah I've had my share of "doozies" in contests. I primarily enter them for the judges feedback, not expecting to win anything. Most of the Detroit area contests are huge, the Michigan State fair alone has over 800 beers entered. I usually enter between 3 and 6 beers in any contest depending on which beers I think are the best OR because I hope to get that "great" feedback which will help me in tweaking a recipe.

And although my scores come out fairly good...High 30s or above, usually only on one beer out of all I enter will I actually get some constructive feedback from a judge that will help me re-consider a recipe.

Heck half the time I can't read the writing anyway. I guess after awhile alcohol and clear printing don't mix.

But the real fun one was this year.

I ask you, if you are judging a style like 21A. Spice, Herb, or Vegetable Beer then shouldn't you at least appreciate the style???

I got an honorable mention at the Michigan Renn Fest, bjcp sponsored Brew Barons Beer Brawl, for my Chocolate Mole Porter (or as entered it in the contest as "Chocolate Chile Porter")

And this was one of the final comments on the judging sheet;

"I don't know understand what prompts folks to put strange ingredients in otherwise good beer. Too Strange in weird spiciness for me, but ...I'm too traditional. Drinkable if you like peppery beer."

Then why the heck are they judging the category of beer called "Spiced, herb and vegetable beer, if that's their attitude? I also wonder if he wrote it on all the entrants sheets.

I wonder if the beer would have even ranked higher had he been more "open" to the style. Honestly I wouldn't judge a style I wasn't a fan of, like wheats for instance. I don't get any pleasure from them, so how can I not be biased as a judge?

I still scratch my head thinking about it. I mean he did still give me a good enough score that I got a certificate. And the other judge did say that if they hadn't ended up folding 21 A with 21b (holiday spice) I would have medaled in the category.

But still I don't get that first judge's comment.

 

[KYB]

A 43!!! By a certified judge, that must be one hell of a good beer, congrats!!!

Hah, thanks man. It was . I'm trying to make it even better. My goal is 45+. The recipe is in my dropdown if you're interested.

 

[Sigafoos]

If you get bad feedback that's not the system, it's the judge.

I think Buffalo is very BJCP-heavy... the last AWOG had over 400 entries and I'm all but positive every judge was at least Recognized. I'm spoiled, because I certainly wouldn't want someone as eager but unqualified as me scoring my beer.

 

[jacksonbrown]

Glad to see this generated a lot of talk, and that I'm not the only seeing these things. All my judges were BJCP certified, so that obviously doesn't always mean anything. Someone had mentioned this earlier, but perhaps I'd be best off taking the exam myself to perhaps balance things out a bit.
On a psudo side note, I did steward a competition once and did get to see first hand how arbitrary some of the decisions are and how much influence one judeg can have on another. I was at the IPA table and there was one clear winner, but when deciding on second, third, and honorable mention it was literally just point and pick. In terms of the competition, no it doesn't matter, that one beer will advance. But to the other three brewers it does.

 

[TexLaw]

There is very little official policing of homebrew judging, and there just isn't going to be much more anytime soon. The bottom line is that brewing and judging are hobbies, judges are volunteers and human, and good judges are can be fairly scarce if you aren't in the right area or environment. What policing there is typically is done by fellow judges during the competition, and that's a bit of "catch-as-catch-can." The Head Judge (a.k.a., Judge Coordinator) or competition organizer should review scoresheets to ensure legible and sensible comments, but I can tell you right now that virtually never happens. Unless you are dealing with a very small competition, they just don't have the time.

Competitions also have exploded in the last few years or so. There are many, many, many more competitions out there than there were five or ten years ago. That stretches the judging pool a bit thin and leads to judging fatigue. No one likes to judge three flights in one day. However, sometimes, you have to do it to get through the entries. There just aren't enough folks willing to spend the time and come out to judge competitions.

I hope you do go ahead and get into the BJCP program. We need more willing, skilled, and conscientious judges out there. It helps the hobby as a whole. In the meantime (and I mean this in the most amiable manner and direct it to everyone), be a little careful about ranting on judges when you are not willing to come out and judge a competition, yourself, or study up for and take the BJCP exam. It's bad form to rant on a volunteer doing the job you would not. Such criticism tends to fall on deaf ears. If you want to fix flaws in the homebrew competition judging you see, it's best to fix them from the inside.


TL

 

[arturo7]

Well said, TL.

 

[khiddy]

I have to think that studying for and taking the BJCP will also make for a better brewer, as well. Familiarizing oneself with the categories and styles, seeing the differences, can only help a brewer as s/he is developing a recipe. The best BJCP judges are those that are also brewers: by knowing the process of brewing inside and out, you can provide better feedback as you're judging. It's a symbiotic relationship!

 

[chumprock]

Guess I could have posted my HBT confusion about the judges comments here too.

https://www.homebrewtalk.com/f20/hbt-bjcp-competition-results-141012/index23.html#post1708091

 

[Beerrific]

As a person who has enter a fair number of beers in comps and is a BJCP judge, I have seen both sides of this. I have gotten sheets back like that as well. And, from my experience, being BJCP does not necessarily mean you will get good feedback either, but it helps. I have gotten sheets back from non-BJCP judges that had some really nice, insightful comments. I have also gotten some back from BJCP judges that should embarrass the BJCP.

My thoughts (IMO of course)....

First, if you get a score sheet back that:
1) Is illegible or
2) Is complete crap.
I would at a minimum contact the event organizer. Or maybe e-mail the judge directly. You are essentially paying for this service, it is in the club/organization's best interest to give you good feedback. If you are getting crap back, then why enter? A good organizer will realize that they are selling a service and if they want to grow the event they need to make sure the judges aren't crap by acting on feedback from people like you or even checking score sheets as they are completed. If one judge sucks, confront them and/or don't invite them back the next year.

Second, as far as judges judging categories they don't like, that falls somewhere between a poorly run event or the specific judge being an idiot. Every event I have judged in has you fill out a judge form a head of time and asks "What catagories would you not like to judge?" If you don't like certain categories, answer that question. It is not hard. I was judging Fruit Beers with a guy that hated fruit beers. I know if he had said he did not want to judge that category, there were a dozen guys there that could. "It is like a hefeweizen with fruit added....25" WTF? I don't love fruit beers, but I can give objective feedback. Now, smoke beers I can't do, but I know that and request not to judge them. Not a big deal.

Third, as far as the judges reporting things that you can't taste. Well, that is human nature. What judges taste in a beer is a function of what they have had to eat and drink that day, how cold the room is, some guy's cologne, etc. And don't forget, the ability to taste and interpret flavors is highly depended from person to person. For example, I am VERY sensitive to sulfur (like in lagers, etc.). I was judging with a Grand Master judge once, he wrote "clean, well brewed lager," but I know I tasted/smelled sulfur in there. So I wrote it down. Might have confused the person when they read the comments, but two people tasted it, you should expect (somewhat) different comments.

 

[ohiobrewtus]

If your score sheets were that poor, you should contact Janis Gross or someone else at the AHA so that the competition organizer and/or judges can be notified.

 

[weirdboy]

If you get bad feedback that's not the system, it's the judge.

It's a poor system that allows judges like this to turn in crappy scoresheets.

There's also the fact that you can become a BJCP Certified judge despite completely failing the tasting portion of the BJCP exam, and you can even miss 10% of the written questions as well and still be a recognized judge.

Frankly, I think the tasting portion should be worth more like 50% or more of the exam, since the primary purpose is to provide good feedback after tasting beers in competition, not to discourse about the history of a style to strangers in a brewpub.

 

[olllllo]

There can be poor service providers and there can also be poor customers in any customer service relationship.

If you do not clearly articulate to the service provider the nature of any shortfall in your expected level of service then you do not provide an opportunity to have that problem fixed.

If you routinely have this problem and do not choose an appropriate medium to address the problem, then perhaps you are not a good customer.