Competition results useless?

[Yooper]

Yeah, compared to running the competition, judging is cake. You mean all I have to do is drive across the state, write constantly for several hours, and drive back? Sign me up.

Well, if you really want to, you can always help us out in Nebraska!

I'm not saying organizing is harder- I'm just saying that NO one wants to run a poor competition, Not the BJCP certainly, the organizer, nor the judges themselves. Everyone I've ever seen at a comp is very serious and does the best they can.

If someone's experience with a BJCP comp is sub-par, that reflects badly on all competitions and my point was that it's not usually that way.

 

[remilard]

I have to say, I understand that being a judge is hard work. But claiming it's exhausting and you're giving up your time for no pay is no reason for giving a lackluster scoresheet. Any volunteer activity is the same way. No one is forcing these people to do this. In my mind, if you want to volunteer to help out, then you should do it right and to the best of your ability.

It sounds like you judges that are on here do a great job. However, it sucks when you get a scoresheet from one of these guys that are just going through the motions, as Wayne admitted they exist. I would rather get a low score with helpful criticism than get a high scoring beer and not have any useable information from the judge.

Also, I did not intend this to be a judge bashing thread. And I am in no way saying anything negative about the people that gladly donate their time to help out homebrewers. I just wanted to find out if other people have had the same results from competitions and if I should be interpretting the scoresheets differently.

I have not emailed any judge or competition coordinator about any of these instances. I am going to take Yoopers advice though. I'll start emailing them when something like this occurs again though. In reality, they can't improve upon this if they don't know. This is something everyone should do. If a competition coordinator gets enough complaints about their judging not making sense to us then they will be more likely to look into getting it fixed. Perhaps raising the standards for judges. And if you email the judge telling him you don't understand why his criticism was completely opposite the other judge, it's more likely he will ask other judges to find out if he is misdiagnosing a beer.

There is a difference between a scoresheet that is bad but the judge tried (hopefully they are getting better) and one where the judge didn't try. If you get a famous:

Malt and Hops
Good
Malt
Light body
Good
33

Scoresheet then emailing the judge probably won't be productive. I would scan it, including the judges name, and email it to the competition organizer and your regional BJCP representative (if the person has a BJCP number) and calmly and briefly express your dissapointment.

If you get a scoresheet where you have questions or think the judge made a mistake (say they marked down your straight lambic for being uncarbonated or something) then I think a direct email to the judge can be productive.

 

[remilard]

Well, if you really want to, you can always help us out in Nebraska!

Hmmmm

I'll think about it but that is inconveniently the weekend before the NHC first round deadline. I'd probably be wise to save that weekend for bottling and packing.

 

[xerotime]

There is a difference between a scoresheet that is bad but the judge tried (hopefully they are getting better) and one where the judge didn't try. If you get a famous:

Malt and Hops
Good
Malt
Light body
Good
33

I got a couple of those recently. About half of the sheets the judge didn't even bother to put his/her name on.

 

[Randar]

Well since we have a couple judges in the thread, we should really push now for some handwriting workshops to be included. What's the point of showing up and judging, providing feedback, and then presenting it illegibly?

I have had at least one illegible score sheet in each of the past 2 comps I have entered.

 

[TheMan]

There is a difference between a scoresheet that is bad but the judge tried (hopefully they are getting better) and one where the judge didn't try. If you get a famous:

Malt and Hops
Good
Malt
Light body
Good
33

Scoresheet then emailing the judge probably won't be productive. I would scan it, including the judges name, and email it to the competition organizer and your regional BJCP representative (if the person has a BJCP number) and calmly and briefly express your dissapointment.

If you get a scoresheet where you have questions or think the judge made a mistake (say they marked down your straight lambic for being uncarbonated or something) then I think a direct email to the judge can be productive.

I like those ideas. I'll use them when I get the results back from future competitions if needed.

 

[Randar]

I like those ideas. I'll use them when I get the results back from future competitions if needed.

Ditto, thank remmy

 

[SpanishCastleAle]

Hmmmm

I'll think about it but that is inconveniently the weekend before the NHC first round deadline. I'd probably be wise to save that weekend for bottling and packing.

No, you should judge. Screw that NHC thing.

March is going to be bottling/packing and jockeying kegs in/out of the keezer hell for me.

Do judges ever comment to their 'paired' judge (or the head judge) that they are not writing enough or illegibly or anything? Is it always upon the entrant to out the lazy judges?

 

[Wayne1]

Do judges ever comment to their 'paired' judge (or the head judge) that they are not writing enough or illegibly or anything? Is it always upon the entrant to out the lazy judges?

It should be up to the steward to make sure the scoresheets are properly filled out.

In my experience, the judges usually don't comment much until they are finished with the sample. They discuss their scores. If they are close, they move on to the next sample. If there is a difference of opinion, they discuss their thoughts and usually reach a consensus. I do not recall anyone mentioning they are not writing enough.

The feedback IS stressed very highly in BJCP training. It is also very important to provide the brewer with ways to get in touch with the judge. Quite a few competitions provide the judges with small labels with their names, BJCP rank and e-mail address on them. The BJCP has a template on their website where this can be found. I always print off a sheet or two before I go to a competition. It saves me writing the same thing 25 or more times.

I am wondering just what competitions you are entering. Are they BJCP registered competitions? Are they in an area where there is a good base of BJCP judges to draw from? Here in Colorado we are fortunate to have quite a few National level judges and many experienced Certified. A core group does seem to be able to travel the front range from Wyoming to Colorado Springs to participate in each competition. I believe there are 142 active BJCP judges in the state.

I can see it might be harder to find experienced judges in states like Delaware (6) or New Hampshire (9) or even Nebraska, which has only 32 judges for the entire state.

You might want to consider entering competitions in states that have a greater number of judges to draw upon. Here is the breakdown of judges state by state.

 

[remilard]

You might want to consider entering competitions in states that have a greater number of judges to draw upon. Here is the breakdown of judges state by state.

I agree with this. The best judging is in the Twin Cities, San Diego, Bay Area, and Denver. That doesn't mean other competitions are bad per se because some do a better job than others of attracting out of town judges.

MCAB qualifiers are usually good bets too.

The best judging is at MCAB and second round NHC, but you have to win your way into those. Unfortunately they both also use the check box score sheets so you get less written feedback.

 

[SpanishCastleAle]

I am wondering just what competitions you are entering. Are they BJCP registered competitions? Are they in an area where there is a good base of BJCP judges to draw from?

I haven't had many problems in the few comps I've entered; NHC twice, Sunshine Challenge twice, and the HBT comp last year (and MCAB this year). I actually got the least feedback from the NHC but it was still decent overall.

I was cellarmaster for the last Sunshine Challenge and we had quite a few out-of-towners judging and the scoresheets were done very well imo.

 

[jkarp]

One thing I'd like to see in BJCP events is a little random 360 style testing. Stewards would slip commercial examples into the flight and if the judges incorrectly score them they'd be given a warning or set up for remedial training.

It really should be reasonable for a hombrewer to get objective and useful feedback from BJCP certified judges for their hard work.

 

[remilard]

One thing I'd like to see in BJCP events is a little random 360 style testing. Stewards would slip commercial examples into the flight and if the judges incorrectly score them they'd be given a warning or set up for remedial training.

The problem is, what is the correct score?

George Fix advocated a system whereby the highest ranking several judges at a competition would score a beer and everyone else would as well.

When the actual judging occurred, people's scores were weighted by how well calibrated they were with the high ranking judges.

I don't think that is a bad approach but based on my experience, the 99% of judges that aren't National or Master would see that as elitist (I'm wrong if I don't agree with these three guys?)

I've thought about organizing a local competition and using very high quality judging which would require me to limit entries to maybe 50-60. The problem is if people realize I have very good judging everyone might want to enter. How do I choose who to exclude?

This all gets down to the basic problem which is that the number of people who want to enter is very high compared to the number of experienced and thoughtful judges. The only solution ultimately would be to turn entrants away.

It really should be reasonable for a hombrewer to get objective and useful feedback from BJCP certified judges for their hard work.

That would be ideal. I think if you got involved in the competition organizer/judge director side you would have an appreciation for how hard people try to run entrant friendly competitions (good scoresheets being the largest part of that) and how big of a challenge it is.

I'm not denying that scoresheets are sometimes bad, I get bad ones too. I think ultimately you should prepare yourself for the fact that you will get a lot of okay sheets, a few crap ones and a few great ones. Yes I know that makes it really expensive to get those few great ones. Many people have access to a club with some pretty good judges in it where you can get some feedback on your beer (maybe not blind but let them know you want the unvarnished truth) so that is another source of feedback.

Along with the great increase of entrants there is an increase of people training to become judges. The problem is you can enter a beer tomorrow but it takes probably around a year to find an opening in a BJCP exam and take it and then another half year to get your score back. Depending on how often you judge it takes several years to get good and experienced. Hopefully 5 years from now the pool of good judges will be vastly larger and the increase of entrants will have slowed and people will see the quality of judging increase. I do what I can to help including judging a lot, grading exams, giving an exam and teaching a class later this year etc. Hundreds of other people do the same, it's just a big problem and progress will be slow.

 

[Wayne1]

Judges usually don't "incorrectly" score beers. At some competitions there is a calibration beer given before the start of the round. This is identified as a commercial example of the style. The beer is then judged and scoresheets filled out for it. A discussion is then held about that particular sample.

Just because it is commercially made, does not mean it will get a high score. The bottle may be old or abused. I have had many a badly oxidized commercial calibrator.

Why don't you come out to the The Falling Rock next weekend to see what is fully involved in judging? The Reggale and Dredhop competition will be judged there.

dredhop_2011

Get in touch with Deborah Lee about volunteering to steward. You will be able to observe up close just what judges do and help with the process.

 

[jkarp]

I think if you got involved in the competition organizer/judge director side you would have an appreciation for how hard people try to run entrant friendly competitions (good scoresheets being the largest part of that) and how big of a challenge it is.

An errant assumption on your part. I have been involved and it's hard as hell.

I'd still like to see some sort of periodic / random re-cert of a BJCP judge's skills however.

 

[Rhoobarb]

About half of the sheets the judge didn't even bother to put his/her name on.

I'm just now starting to read over the BJCP Exam Study Guide and plan to take the course as soon as I can. One judge gave our club a good tip: he has been printing out Avery labels w/ his name, ID# and email on them for years. The current score sheets now provide an area that specifically states that you can use Avery label #5160.

 

[jkarp]

Just because it is commercially made, does not mean it will get a high score. The bottle may be old or abused. I have had many a badly oxidized commercial calibrator.

Of course, but the style guidelines *do* list commercial examples. I've never seen a "calibration beer" in the competitions I've worked. It's a good idea. Even better done blind, IMO.

Get in touch with Deborah Lee about volunteering to steward. You will be able to observe up close just what judges do and help with the process.

Were you talking to me? I've judged, stewarded, and organized, in the past. Got a pretty good handle on the headaches involved in competitions which is why I rarely do them anymore.

 

[Rev2010]

A judge has two hours to to go through 12 to 14 beers.

Just curious, how do they cleanse their palate between tasting each beer? I know certain beers can taste like utter crap when drank after a different beer.


Rev.

 

[Wayne1]

I've never seen a "calibration beer" in the competitions I've worked. It's a good idea. Even better done blind, IMO.

This is from the AHA/BJCP Sanctioned Competition Handbook

"It is a good idea to start the judging with a “calibration” beer. The
calibration beer serves to give the judges a chance to get a feel
for each other’s scoring range and judging styles, as well as
to warm up their palates. Ideally you will have a commercial
example of each style being judged, but having only one style
will suffice."

Sanctioned Competition Handbook

 

[SpanishCastleAle]

Just curious, how do they cleanse their palate between tasting each beer? I know certain beers can taste like utter crap when drank after a different beer.


Rev.

Only comp I've been a part of had lots of water and oyster crackers (the little hexagon-shaped ones).

 

[Wayne1]

Just curious, how do they cleanse their palate between tasting each beer? I know certain beers can taste like utter crap when drank after a different beer.


Rev.

It depends on the competition organizer. Usually a crusty french bread. In one competition, matzo is used.

 

[SpanishCastleAle]

It depends on the competition organizer. Usually a crusty french bread. In one competition, matzo is used.

I was wondering if oyster crackers have too much salt? Salt seems to affect one's perception of sweetness so I would imagine it should be low. Maybe they used 'reduced sodium' oyster crackers, I didn't look at the box.

 

[Rev2010]

It depends on the competition organizer. Usually a crusty french bread. In one competition, matzo is used.

See, I still personally don't think that's enough to clear a palate. I've read some wine connoisseurs use lemon and water, not sure if that's accurate though. But either way anything you taste after is going to taste different because of the "foreflavors". There was a time when I was beer connoisseuring and had a notebook with my personal descriptions and notes about each beer I'd tasted. Very soon after starting it I'd had a second beer from one I'd already sampled and had a very different opinion on it that simply did not compare to the book. The reason? Because I'd tried 2 or so other beers before defaulting back to it for the night. I know it's completely impossible to expect any human being to have a completely clean palate when having to sample a number of beers. But in reality, it seems to me the system is flawed and what one judge may rate as poor one day might judge as excellent another if their palate is completely clean.

For example... if I have a Paulaner Hefeweizen then have a Franziskaner Hefeweizen right after I get this weird, as I call it dog hair taste, in my mouth. So in that instance, if I'd been judging something, it would not be fair to the second beer.

Of course, I don't have a solution. The human taste system is extremely complex. Even just having the onset of a cold you don't even know is coming on will screw up your tastes. But I can imagine it's hard to truly fairly judge beer when one is having several in a row.


Rev.

 

[olllllo]

If people gave up on brewing beer like they do on entering competitions, no one would be here.

 

[bob1852]

If you enter a comp that ollllllo judges, you will get good, constructive, legible feedback. At least I have.

 

[TheMan]

If you enter a comp that ollllllo judges, you will get good, constructive, legible feedback. At least I have.

...any competitions you know you'll be judging ollllllo?

 

[remilard]

I was wondering if oyster crackers have too much salt? Salt seems to affect one's perception of sweetness so I would imagine it should be low. Maybe they used 'reduced sodium' oyster crackers, I didn't look at the box.

I've had some oyster crackers that weren't very salty. Bought some for a competition this month and they were too salty, oops. I'll do some taste tests next year. I find bread too messy (crumbs everywhere).

Ultimately I think water is the best palate cleanser and try to use the crackers/bread as needed only. Crackers and bread have some of the same flavors as beer so if you use them as a cleanser I think it is important to come behind it with lots of water before the next sample, and this helped a lot with the too salty crackers too.

Maybe not appropriate for a homebrew comp but I find a blander cheese to work very well.

 

[Airborneguy]

What are you expecting out of contests?

Some level of consistency in the feedback. In my example, both judges were certified. One couldn't taste a full pound of lactose in a beer. That seems strange to me when the other pointed it out easily. If you can't taste lactose, judge IPA's or don't judge at all.

 

[Airborneguy]

If people gave up on brewing beer like they do on entering competitions, no one would be here.

So does one have to enter competitions to be a good homebrewer? I'm generally against them because I don't see much of a point.

Based on my experiences, I haven't gotten feedback that is better than handing my beers to certain friends who I know will give me an honest critique and know beer styles. I'm not saying this because I'm pissed at a few beers that got ripped, but at the contradictions. Two of my past entries were pretty well panned, but those sheets were the best critiques, and the ones I appreciate the most. I'm not looking for medals or awards, so that part means nothing, and the prizes rarely amount to anything anyway. So really, what's the point? I entered the HBT one because I am on this forum so much and some of the prizes seem worth a shot, but I seriously think this is the only contest I'll ever enter again (yearly if Yoop keeps organizing it).

 

[Indyking]

A judge has two hours to to go through 12 to 14 beers. Then another style needs to be judged before you get lunch. The judge has to do it all over again in the afternoon in entirely different categories.

I'm just curious. How many ounces per beer a judge have to drink? 12-14 beers in 2 hours, which seems to be just warming up for the day ahead, appears to be a lot! Maybe there is no correlation, but I wonder how much the alcohol can affect a conscious and fair judgment in the last ones tasted. Homebrews tend to have high ABVs as you all know.