Competition results useless?

Homebrew Talk

Help Support Homebrew Talk:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

TheMan

Well-Known Member
Joined
May 8, 2009
Messages
2,957
Reaction score
341
Location
Illinois
I have found competition results/scorecards to be completely useless to better my brewing. There are 2 or 3 scorecards from most competitions and I find that the judges critiques (in the same competition) can differ by a large amount. So much that their input opposes each other. Obviously the score numbers are in the same range, but the written critiques are far from the same.

My last competition entry one judge said it had astringency. The other judge said it had no astringency...What am I to take from this? I either have a problem or I don't. I also have had an entry where one judge says it lacked aroma and I needed to dry hop the beer more while the other judge said it had wonderful hop aroma.

I understand that each individual has their own taste perception, but I would think that two judges sitting side by side, tasting and discussing a beer would write up somewhat agreeable critiques.

I've had beers win and had beers lose. When I win I get a medal/ribbon and prize with a critique that says it was wonderful and gives me small pointers that could bump it up a few extra points. And thus I know the beer was good...however, when I don't even place it's a bummer, but in my mind I'm thinking, "At least I'll get to find out what I did wrong and make it better next time". But this rarely happens. And the critique usually leaves me disappointed.

Do any of the judges here have any input on this type of thing? Should I be interpretting the results differently? Anyone else see similar results in their own competition critiques?
 
a lot of beer judging comments are completely useless. sometimes the judges are people with no training whatsoever. i personally witnessed a beer judge enlist the help of his wife for a competition in kansas city. his wife had never been officially trained yet was scoring beers and writing comments.

also, the training to become a judge is barely enough to even be considered an introductory course in beer tasting in my opinion. so yeah... I wouldn't put much stock in their comments
 
I have absolutely seen this on my scoresheets, and for that reason I hardly ever enter competitions.

Usually they pair less experienced judges with more experienced ones on the same flight. Take the higher-ranked judge's comments more seriously.
 
I made a soured saison once. Clearly in the style guidelines that a saison can have some sourness, it also had some earthiness and spiciness.... also in the style guidelines. It was a bad ass beer...

"Sour. Not to style. Need to watch your sanitation. Also getting some spice and earthiness that shouldn't be in there."

I don't enter many competitions anymore.
 
I have absolutely seen this on my scoresheets, and for that reason I hardly ever enter competitions.

Usually they pair less experienced judges with more experienced ones on the same flight. Take the higher-ranked judge's comments more seriously.

This is basically what I've been doing, but at the same time I consider what the other said because it is still an unbiased judge of my beer.

I feel I have a good taste for which of my beers will win, so I am just sending in beers that I know should win. The prizes are cool sometimes. But sending in a beer that I know isn't great just to get a critique and make it better is useless.
 
TheMan said:
I have found competition results/scorecards to be completely useless to better my brewing. There are 2 or 3 scorecards from most competitions and I find that the judges critiques (in the same competition) can differ by a large amount. So much that their input opposes each other. Obviously the score numbers are in the same range, but the written critiques are far from the same.

My last competition entry one judge said it had astringency. The other judge said it had no astringency...What am I to take from this? I either have a problem or I don't. I also have had an entry where one judge says it lacked aroma and I needed to dry hop the beer more while the other judge said it had wonderful hop aroma.

I understand that each individual has their own taste perception, but I would think that two judges sitting side by side, tasting and discussing a beer would write up somewhat agreeable critiques.

I've had beers win and had beers lose. When I win I get a medal/ribbon and prize with a critique that says it was wonderful and gives me small pointers that could bump it up a few extra points. And thus I know the beer was good...however, when I don't even place it's a bummer, but in my mind I'm thinking, "At least I'll get to find out what I did wrong and make it better next time". But this rarely happens. And the critique usually leaves me disappointed.

Do any of the judges here have any input on this type of thing? Should I be interpretting the results differently? Anyone else see similar results in their own competition critiques?

I reckon there must be very few knowledgeable judges with a sharp palate out there. Most of them in the hundreds of competitions country wide are hopeless in giving any real useful advice.

I think the national AHA competition is respectable, but I heard the first round is no different from other local competitions in terms of judge quality, in other words, an excellent beer may unfairly not pass to second round and vice-versa.

I'm starting to submit my first brews to competitions now but if I start getting inconsistencies among judges in the same brew, I will just stop wasting my otherwise wonderful beer with them, without mentioning the entry cost, which is often kind of expensive!
 
Most of the comps I enter have experienced judges paired with inexperienced judges and I follow the advise of the senior judge most of the time. The thing is that it takes a lot of experience to properly judge a beer and a persons own perferences may sway the marks. But to look at it another way, a judge has to start somewhere, and the only real way to gain experience is to jump in and judge in comps.
 
also, the training to become a judge is barely enough to even be considered an introductory course in beer tasting in my opinion. so yeah... I wouldn't put much stock in their comments

Are you talking about BJCP training? I hope you have a 90+ on the exam before you dismiss it's difficulty. If you do, congratulations there are about 100 of us but be respectful of the many people who find it very challenging to even score at one of the lower levels.
 
I reckon there must be very few knowledgeable judges with a sharp palate out there. Most of them in the hundreds of competitions country wide are hopeless in giving any real useful advice.

I think the national AHA competition is respectable, but I heard the first round is no different from other local competitions in terms of judge quality, in other words, an excellent beer may unfairly not pass to second round and vice-versa.

I'm starting to submit my first brews to competitions now but if I start getting inconsistencies among judges in the same brew, I will just stop wasting my otherwise wonderful beer with them, without mentioning the entry cost, which is often kind of expensive!

Running your mouth about things you have no experience with is one thing when you are giving advice, it is another thing when you are calling volunteers useless when you have never entered.
 
remilard said:
Running your mouth about things you have no experience with is one thing when you are giving advice, it is another thing when you are calling volunteers useless when you have never entered.

Wow, are you OK? Bad day?

I was just saying what I think based on the bad experiences friends and the others in this very thread (hello?) are having with judge inconsistencies! Oh my!
 
Remilard,
Since your a judge I'm sure you can answer a few questions I have. Do the judges not look at each other's score sheets to see critiques? I know there is a time limit, but I was under the impression that the beers were discussed when scoring.

And in my example of one judge claiming astringency and the other saying no astringency. What should I take from something like that? It's almost like they discussed it and the judges disagreed and just wrote what they thought.

Is it sound advice to just take more seriously the more experienced judge?
 
Remilard,
Since your a judge I'm sure you can answer a few questions I have. Do the judges not look at each other's score sheets to see critiques? I know there is a time limit, but I was under the impression that the beers were discussed when scoring.

Generally the judges discuss after filling out the scoresheet. In practice if they discussed while filling them out a lot of the time the more dominant personality would control and while you would get the same comments, it wouldn't necessarily be because the judges agreed.

With an experienced judge that is not a bully and a new judge some discussion throughout is helpful but I'll tend to use the Socratic method and ask questions rather than telling them what they should be tasting or smelling.

If both judges have some experience my preference is to write up, discuss briefly at the end (unless the scores are more than a few points off) and move on.

And in my example of one judge claiming astringency and the other saying no astringency. What should I take from something like that? It's almost like they discussed it and the judges disagreed and just wrote what they thought.

I think a lot of people make mistakes on astringency. It is a mouthfeel characteristic and people apply to flavor characteristics like roast. If you get someone saying in the flavor section that the beer is astringent, I don't think you should take it seriously. Also sometimes people get something pointed out to them or learn a new word and they kinda go crazy on that thing for a while and look for it everywhere. So I think you see this one too much.

On the other hand if you see diacetyl on one and not the other, that may be that some people have much higher thresholds for diacetyl or also that people tend to never notice diacetyl until they have done an off flavor tasting and the lightbulb goes off. Oxidation is similar where people are more likely to miss it then fabricate it.

Is it sound advice to just take more seriously the more experienced judge?

Yes, although I would use the quality of the score sheet and not just their rank to determine how good you think they are.

At the end of the day scoresheets take some parsing based on what is there, who wrote it, what their rank is, what you know about them etc. You'll get better at using the feedback over time. I think it helps a lot to enter the same beer over and over. You often find 2 scoresheets to be wildly different but if you can get 10 scoresheets you will see patterns.
 
Wow, are you OK? Bad day?

I was just saying what I think based on the bad experiences friends and the others in this very thread (hello?) are having with judge inconsistencies! Oh my!

There were more tactful ways to say that than to say that most people who volunteer to judge are useless.

You are welcome to bring your superior palate to bear on this problem.
 
remilard said:
There were more tactful ways to say that than to say that most people who volunteer to judge are useless.

You are welcome to bring your superior palate to bear on this problem.

OK, no problem. Let's keep the arguments away. This is a interesting thread.
 
Thanks for the info Remilard. I was really curious how to deal with such dissenting opinions on my scorecards. It's extremely frustrating when you see two writeups opposing each other like that. Especially when you paid money to enter and ship your beer there. It really put me off from entering competitions for a little while.

I'm sure other people will agree, even if your beer gets a low score it's still worth it as long as you get proper information from the judges to correct the problem.

I have come to find that regardless of score, the best beer usually wins (category and overall). So I don't find competitions completely useless. I'll continue to send in beers that I am fairly certain will win. But I will still be hesitant to send in any beers that I find questionable. I think it would be cheaper to just rebrew a beer I think is below par than to send it to multiple competitions.

Thanks again Remilard, much appreciated
 
I had a similar complete contradiction:

Sweet stout with 1lb of lactose added at the end of the boil. I found it to be noticeable, but being a malt/sweet person, I actually wanted more.

Judge #1: "Flavor presents a mix of roast, hop bitterness, and residual (added) sweetness."

Judge #2: "A single roasted malt flavor with hop astringency in the finish. No detection of sweetness one would expect from this style."

These were both in the flavor section, and BOTH were BJCP Certified level judges.

What the hell is hop astringency anyway?

EDIT: Just wanted to add that I copied those quotes off of the score sheets just now.

This turned me off to contests completely. I am entering the upcoming HBT contest, but I doubt I'll ever enter any other contests again. That isn't the only time this happened to me, but the most demonstrative and it was also the final straw.
 
I've had pretty poor experiences with the two competitions I've submitted to thus far.

The most recent competition was supposed to be a very loose competition, not strict to style, very loose categories. One of my beers, a double IPA with buckwheat honey, got reamed by a "certified judge" about not being to style, while the other two judges in the category gave it fairly standard remarks.

Another I entered to this competition, a very simple pale ale, was rated very highly by two professional brewers, but then a third homebrewer judge for the style competely trashed me in scores while giving very positive written remarks (as well as no negative remarks). The homebrewer scored me so low that it lowered my score enough to remove me from the final round.

Needless to say, I likely won't waste my time, money, or beer on any more competitions after this.
 
I enter competitions quite often and have come to the conclusion that 75% of the judges comments will either be completely worthless or vague. However, there are always a few judges that score properly and take the time to give constructive feedback - and that usually makes up for all the other shoddy judging.

Though, I should note that I am not afraid to e-mail the judges with my opinions on their lack of judging skills... here is what one "certified judge" put down on one of my score sheets and you can bet he got a very 'friendly' email from me. Never responded back.

Aroma: Dark grain
Apperance: Good
Flavor: Roasted grain flavor
Mouthfeel: Slightly astringent
Overall Impression: Good
 
...many people who find it very challenging to even score at one of the lower levels.

I think this gets to the point. Let's face it, most people are lazy. Anyone who's read any of remilard's posts knows that remilard is a thorough and precise person. And even remilard admits it's challenging to do a good job judging beer at a competition.

I haven't entered a competition, basically because I'm a stingy miser who wants to keep all his beer. So I am extrapolating from what I see at work, which I assume somewhat reflects the real world.

There are a few people at work who really do a good job and care about what they're doing, no matter what the job title or task at hand, from a machine operator to an engineering manager. But many of the people at work are just looking forward to going home at the end of the day. I bet it's similar with judges. There are some who really take the competition seriously and try to provide useful feedback, and there are probably some who just want to drink a bunch of different beers and feel smart for scoring them.
 
I'll also say that it matters when your beer happens to be pulled in the flight. I can tell you that my level of detail, quality of suggestions, and even handwriting vary greatly from beer #1 to beer #10. That's completely luck of the draw.

Also, in my experience, while some of the feedback may not be super helpful, I do think the competitions get the winners right more times than not. For instance, when you are judging 10-12 American IPAs you can usually pick the top 2-3 fairly easily. Then with the mini best-of-show, the top beers are directly compared to each other to arrive at the top 3.

So, if you're scoring in the mid or low-range, your beer probably didn't fit the style guidelines as well as the others. The suggestions/feedback may not be the best, but you can feel confident that your beer wasn't the best.

One final note....judges don't pick the BEST beer. Just the one that fits the style guidelines the best. So, you may have a low scoring beer that is GREAT. More power to you. Don't worry so much about the scores if you enjoy your beer.
:mug:
 
What the hell is hop astringency anyway?

Seems to be quite a bit of info about it...

http://www.google.com/search?q=hop+...ie=UTF-8&oe=UTF-8&sourceid=ie7&rlz=1I7GGIE_en

This turned me off to contests completely.

What are you expecting out of contests? If everyone had the exact same taste buds, this world would be a boring place...

Contests are about getting some people who have studied up on styles to give their impression of how your beer relates to a published style.....
 
Y'all can help improve beer evaluations by training yourself to help out with the homebrew competitions.

In the meantime do you have a better way to get anonymous constructive critiques?
 
"Most Beer Judges Are Boneheads" was the name of a Silver Medal winning Pilsner at GABF in 2005. This was one of seven award winning brews that year made at the SandLot Brewery at Coors Field. They also won small brewery of the year. What is interesting about the name of the beer is that the brewer, Tom Hail, is a National level BJCP judge.

I was part of the first group to take the BJCP exam in Colorado back in the '80s. I have judged a few competitions over the years and I do have to say the level of judging has improved considerably lately. There are more people interested in becoming judges than ever before. With all that said, Most Beer Judges ARE Boneheads. Take a look at the statistics published by the BJCP

BJCP Demographics

The exam HAS gotten tougher over the years. More people are taking it and more people are not doing as well. The newer judges ARE being taught to fill out scoresheets with more information and feedback helpful to the brewers. This is stressed during most judges meetings before the competition begins. The reality is that there are a lot more competitions being offered than there are experienced judges to fill.

How many of you are willing to give up a day from your weekend to travel 100 miles or more to critically assess many beers and write fantastic feedback to make everyone happy? A judge has two hours to to go through 12 to 14 beers. Then another style needs to be judged before you get lunch. The judge has to do it all over again in the afternoon in entirely different categories. If your are an experienced Certified level or National, you are then asked to judge Best of Show. This IS a lot of work.

What does the judge get out of it? A semi mediocre buffet lunch and a point to add to your experience level. I do have to stress this: judges are not paid! We are volunteers who do this to help other brewers become better brewers. Yes, there are some judges out there who are just going through the motions. Thankfully they are becoming rarer. The newer judges, like remilard and KellyK are very adamant about helping people improve their beer.

As an old fart, I do try to steer younger judges to add comments to help entrants improve their beer. Unfortunately, a lot of the newer judges are also new in their brewing experiences. Like a lot of the people on this forum, they haven't made a lot of beer. They haven't tried every grain or hop known to man.

Most brewers tend to gravitate to just a few styles they like to brew and drink. A judge has to be ready to critically evaluate every style. Realistically, this is not possible. While you are tested on many styles on your BJCP exam, it may be some while ago that you spent time studying up on the subtle differences involved in making a Saison.

Some competition organizers will ask if you have a preference for styles to judge. Other times you are just assigned to a category where there is a need. This may end up with a judge who is not really familiar with the style judging your beer. Every table usually does have a copy of the style guidelines to fall back on.

Getting back to my original theme, Most Beer Judges ARE Boneheads. But the BJCP IS trying to improve things. If you find that the person who judged your beer did not give you the feedback you wanted, please E-mail them and let them know. E-mail the organizer to suggest certain areas that you think could be improved. No one gets better without help. That is what competition is all about.
 
Running your mouth about things you have no experience with is one thing when you are giving advice, it is another thing when you are calling volunteers useless when you have never entered.

OK, no problem. Let's keep the arguments away. This is a interesting thread.

Remilard is right, although maybe not as tactful as he could be maybe. Judging is HARD work. The BJCP test is HARD. There is no pay, and many of the beers suck beyond belief. Band-aids, infected brews, astringent beers, gushers, etc. Many brewers suffer from what I call the 'ugly baby syndrome'. They love their beers and may be accustomed to their house flavors but I've had some beers that have turned my stomach.

But judging is a great way to help fellow brewers. As was mentioned, some of us have different taste thresholds. I seem to be a "super taster" and can pick up diacetyl easily while some others can't. I also can taste early oxidation on the sides of my tongue way before it's sherry-like or cardboard-like.

All of the judges I've been with have gone about their duties with eagerness, seriousness, and with a goal of providing good feedback. We talk among ourselves and discuss the beer. The best competitions will always do this.

If your comments on your scoresheet don't make sense, then contact the judge! They provide their information, including email addresses. Ask what they meant by "astringency". Now, they may not remember THAT particular beer but often they can when reminded of their comments. If you really have a problem with the comments, let the organizer know! That's so important.

Organizing a BJCP competition is hard work, and takes many hours. If there is a problem, trust me- we want to know!
 
If you really hate beer judges, do what I do: make the most DISGUSTING batch of beer you can possibly make, and enter it into every single category!


No... I don't really do that, but I'd be lying if I said I've never considered it!
 
Organizing a BJCP competition is hard work, and takes many hours. If there is a problem, trust me- we want to know!

Yeah, compared to running the competition, judging is cake. You mean all I have to do is drive across the state, write constantly for several hours, and drive back? Sign me up.
 
I have to say, I understand that being a judge is hard work. But claiming it's exhausting and you're giving up your time for no pay is no reason for giving a lackluster scoresheet. Any volunteer activity is the same way. No one is forcing these people to do this. In my mind, if you want to volunteer to help out, then you should do it right and to the best of your ability.

It sounds like you judges that are on here do a great job. However, it sucks when you get a scoresheet from one of these guys that are just going through the motions, as Wayne admitted they exist. I would rather get a low score with helpful criticism than get a high scoring beer and not have any useable information from the judge.

Also, I did not intend this to be a judge bashing thread. And I am in no way saying anything negative about the people that gladly donate their time to help out homebrewers. I just wanted to find out if other people have had the same results from competitions and if I should be interpretting the scoresheets differently.

I have not emailed any judge or competition coordinator about any of these instances. I am going to take Yoopers advice though. I'll start emailing them when something like this occurs again though. In reality, they can't improve upon this if they don't know. This is something everyone should do. If a competition coordinator gets enough complaints about their judging not making sense to us then they will be more likely to look into getting it fixed. Perhaps raising the standards for judges. And if you email the judge telling him you don't understand why his criticism was completely opposite the other judge, it's more likely he will ask other judges to find out if he is misdiagnosing a beer.
 
Yeah, compared to running the competition, judging is cake. You mean all I have to do is drive across the state, write constantly for several hours, and drive back? Sign me up.

Well, if you really want to, you can always help us out in Nebraska! :D

I'm not saying organizing is harder- I'm just saying that NO one wants to run a poor competition, Not the BJCP certainly, the organizer, nor the judges themselves. Everyone I've ever seen at a comp is very serious and does the best they can.

If someone's experience with a BJCP comp is sub-par, that reflects badly on all competitions and my point was that it's not usually that way.
 
I have to say, I understand that being a judge is hard work. But claiming it's exhausting and you're giving up your time for no pay is no reason for giving a lackluster scoresheet. Any volunteer activity is the same way. No one is forcing these people to do this. In my mind, if you want to volunteer to help out, then you should do it right and to the best of your ability.

It sounds like you judges that are on here do a great job. However, it sucks when you get a scoresheet from one of these guys that are just going through the motions, as Wayne admitted they exist. I would rather get a low score with helpful criticism than get a high scoring beer and not have any useable information from the judge.

Also, I did not intend this to be a judge bashing thread. And I am in no way saying anything negative about the people that gladly donate their time to help out homebrewers. I just wanted to find out if other people have had the same results from competitions and if I should be interpretting the scoresheets differently.

I have not emailed any judge or competition coordinator about any of these instances. I am going to take Yoopers advice though. I'll start emailing them when something like this occurs again though. In reality, they can't improve upon this if they don't know. This is something everyone should do. If a competition coordinator gets enough complaints about their judging not making sense to us then they will be more likely to look into getting it fixed. Perhaps raising the standards for judges. And if you email the judge telling him you don't understand why his criticism was completely opposite the other judge, it's more likely he will ask other judges to find out if he is misdiagnosing a beer.

There is a difference between a scoresheet that is bad but the judge tried (hopefully they are getting better) and one where the judge didn't try. If you get a famous:

Malt and Hops
Good
Malt
Light body
Good
33

Scoresheet then emailing the judge probably won't be productive. I would scan it, including the judges name, and email it to the competition organizer and your regional BJCP representative (if the person has a BJCP number) and calmly and briefly express your dissapointment.

If you get a scoresheet where you have questions or think the judge made a mistake (say they marked down your straight lambic for being uncarbonated or something) then I think a direct email to the judge can be productive.
 
Well, if you really want to, you can always help us out in Nebraska! :D

Hmmmm

I'll think about it but that is inconveniently the weekend before the NHC first round deadline. I'd probably be wise to save that weekend for bottling and packing.
 
There is a difference between a scoresheet that is bad but the judge tried (hopefully they are getting better) and one where the judge didn't try. If you get a famous:

Malt and Hops
Good
Malt
Light body
Good
33

I got a couple of those recently. About half of the sheets the judge didn't even bother to put his/her name on.
 
Well since we have a couple judges in the thread, we should really push now for some handwriting workshops to be included. What's the point of showing up and judging, providing feedback, and then presenting it illegibly?

I have had at least one illegible score sheet in each of the past 2 comps I have entered.
 
There is a difference between a scoresheet that is bad but the judge tried (hopefully they are getting better) and one where the judge didn't try. If you get a famous:

Malt and Hops
Good
Malt
Light body
Good
33

Scoresheet then emailing the judge probably won't be productive. I would scan it, including the judges name, and email it to the competition organizer and your regional BJCP representative (if the person has a BJCP number) and calmly and briefly express your dissapointment.

If you get a scoresheet where you have questions or think the judge made a mistake (say they marked down your straight lambic for being uncarbonated or something) then I think a direct email to the judge can be productive.

I like those ideas. I'll use them when I get the results back from future competitions if needed.
 
Hmmmm

I'll think about it but that is inconveniently the weekend before the NHC first round deadline. I'd probably be wise to save that weekend for bottling and packing.
No, you should judge. Screw that NHC thing.

March is going to be bottling/packing and jockeying kegs in/out of the keezer hell for me.

Do judges ever comment to their 'paired' judge (or the head judge) that they are not writing enough or illegibly or anything? Is it always upon the entrant to out the lazy judges?
 
Do judges ever comment to their 'paired' judge (or the head judge) that they are not writing enough or illegibly or anything? Is it always upon the entrant to out the lazy judges?

It should be up to the steward to make sure the scoresheets are properly filled out.

In my experience, the judges usually don't comment much until they are finished with the sample. They discuss their scores. If they are close, they move on to the next sample. If there is a difference of opinion, they discuss their thoughts and usually reach a consensus. I do not recall anyone mentioning they are not writing enough.

The feedback IS stressed very highly in BJCP training. It is also very important to provide the brewer with ways to get in touch with the judge. Quite a few competitions provide the judges with small labels with their names, BJCP rank and e-mail address on them. The BJCP has a template on their website where this can be found. I always print off a sheet or two before I go to a competition. It saves me writing the same thing 25 or more times.

I am wondering just what competitions you are entering. Are they BJCP registered competitions? Are they in an area where there is a good base of BJCP judges to draw from? Here in Colorado we are fortunate to have quite a few National level judges and many experienced Certified. A core group does seem to be able to travel the front range from Wyoming to Colorado Springs to participate in each competition. I believe there are 142 active BJCP judges in the state.

I can see it might be harder to find experienced judges in states like Delaware (6) or New Hampshire (9) or even Nebraska, which has only 32 judges for the entire state.

You might want to consider entering competitions in states that have a greater number of judges to draw upon. Here is the breakdown of judges state by state.
 
You might want to consider entering competitions in states that have a greater number of judges to draw upon. Here is the breakdown of judges state by state.

I agree with this. The best judging is in the Twin Cities, San Diego, Bay Area, and Denver. That doesn't mean other competitions are bad per se because some do a better job than others of attracting out of town judges.

MCAB qualifiers are usually good bets too.

The best judging is at MCAB and second round NHC, but you have to win your way into those. Unfortunately they both also use the check box score sheets so you get less written feedback.
 
Back
Top