I think I am done with contests

[brokebucket]

I just said I was struggling with the concept. I was not attempting to discredit judges and their ability nor their desire to do their best which your post implies I was doing.

Perhaps the existing process is the best there is. I've never entered a competition, but I've gotten to the point where I'd like to get some feedback from experienced palettes. Neither me nor my friends are there at this point, so your earlier suggestion wouldn't work for me.

Isn't the point of the guidelines to attempt to add some level of "calibration" so that there is, using your terms, something approaching a machine-like objectivity? I realize there a number of factors outside of an entrant's or judge's control, but given the same factors, such as those within a certain competition, I would expect the guidelines to steer the judges to a similar score. If they don't, then I would expect a bad, good, better, best system to be just as good and less confusing to those entering a competition.

Maybe I'm looking at this the wrong way, but if I got a score of 26 on a particular beer, I would take away a very different message than I would on one that received a 36.

Hey, I just wanted to comment on what you wrote, as I am probably right around where you are in brewing. I have entered a couple contests, and have done well and not so well....and have got good and not so good feedback on my beers. I personally enter them for fun, and take whatever feedback I get back with a grain of salt, because I dont know the people giving me the feedback.

With that being said, if you REALLY want good feedback, find somebody knowledgeable about beer that is willing to taste it and is willing to give it to you straight. Have a conversation about the beer as you both taste it. Your LHBS or local club is the place for this if you dont already know somebody.

Waiting around for a month to get a scoresheet back is not a great way for a new brewer to improve...there is just too much lag. That's my opinion.

 

[Brickman]

If you really want frustration enter a KCBS judged BBQ contest. They have no specific guidelines of how to judge and have taught 3 different scoring systems over the 11 years I have competed. They give mechanics of judging and that is about it.

But one thing I have noticed is the good cooks consistently rise to the top. However you are judged by humans and we all have different tastes. I had a pork entry that was amazing to me but my wife hated it. They used maple syrup in the glaze. My wife hates maple syrup.

Comparing the two systems the BJCP is much more thought out. It gives strict style guidelines. It is much easier for me to brew to it that it is for me to cook to KCBS guidelines. And as with BBQ the more you compete in BJCP the better understanding you will have of your brewing ability with regards to the style you are trying to hit. I can make amazing BBQ at home but with regards to KCBS competition guidelines it may only be good. Same with BJCP.

I am going to make it a point to enter each beer in at least 2 and hopefully 3 different contest. I feel that is the only way I will get a good feeling of how that beer is in taste and style.

 

[Darwin18]

Perhaps the existing process is the best there is. I've never entered a competition, but I've gotten to the point where I'd like to get some feedback from experienced palettes. Neither me nor my friends are there at this point, so your earlier suggestion wouldn't work for me.

If you want quality feedback on a beer bring it a local homebrew club and ask for them to evaluate it with you. Bring your recipe as well. You can discuss the pros, cons, and where to improve it. In a competition a judge doesn't have your recipe in front of them. All the judge knows about your beer is the category that it has been placed in. You're not going to get the feedback you're looking for in this scenario.

Isn't the point of the guidelines to attempt to add some level of "calibration" so that there is, using your terms, something approaching a machine-like objectivity? I realize there a number of factors outside of an entrant's or judge's control, but given the same factors, such as those within a certain competition, I would expect the guidelines to steer the judges to a similar score. If they don't, then I would expect a bad, good, better, best system to be just as good and less confusing to those entering a competition.

The point of the style guidelines is provide the entrants with requirements for that style to brew to and for judges to hold each entry against in that category. In every competition that I've judged the judging pairs are asked to fall within 3 points of each other. This is usually not an issue.

Maybe I'm looking at this the wrong way, but if I got a score of 26 on a particular beer, I would take away a very different message than I would on one that received a 36.

Yes, and you should. A 26 is a score that isn't going to place. A 36 is a score that is generally to at least advance to mini-BOS, and in a smaller flight will place. That's the difference between a 26 and 36. A 26 has off-flavors or is missing the style; a 36 is a well brewed beer, no off-flavors, and maybe could use minor refining to exemplify the highlights of that particular style.

 

[Darwin18]

They're two years overdue as we speak. I do think that goes to credibility for the BJCP. The current guidelines are 6 years old. A lot has happened in the beer world since then.

The 2014 style guidelines are up on the BJCP forums for judges to check out.

 

[Tiber_Brew]

There are some judges who are definitely judging beers on their own personal preferences rather than judging the beer on its own merits. I often enter the American Pale Ale category because I prefer to hop my pale ales in less than mega-doses. I'm talking around 35-40 IBU (Tinseth), which really isn't all that wimpy. In the comments section under "overall impression", one of the judges apologized for lowering my score, saying that I was a victim of "style creep". He said that the judges now expect hop levels in the APA category which used to be considered in the IPA range.

HUH??!! I was certainly a victim of some kind of "creep"!

Another valid point. I fell victim to this kind of judging once before, too. I submitted my German Pils that had a color of 3.0 SRM (2-5 is allowable per BJCP) and one judge knocked me down significant points because the "color was too dark."

 

[mdbrewer1]

I'm not sure what you mean by "the implicit quality determination of a 50 point scale?"

But I think the points system is designed more for common application of the style criteria then it is for consistency. The adherence to the style criteria itself is what should drive consistency.

What I meant was that using a 50 point scale implies that a beer that scores a certain number is of a certain quality. i.e., a beer that scores a 36 is quite a bit beer than one that scores a 26.

I'm confused by your other statement. If the points system is supposed to be a common application of the style criteria, and adherence to the style criteria lends itself to consistency, doesn't that imply that the points system lends itself to consistency? There might be a nuance there that I'm missing, but that's what it sounds like you're saying in taking your two statements together.

 

[mdbrewer1]

With that being said, if you REALLY want good feedback, find somebody knowledgeable about beer that is willing to taste it and is willing to give it to you straight. Have a conversation about the beer as you both taste it. Your LHBS or local club is the place for this if you dont already know somebody.

Waiting around for a month to get a scoresheet back is not a great way for a new brewer to improve...there is just too much lag. That's my opinion.

If you want quality feedback on a beer bring it a local homebrew club and ask for them to evaluate it with you. Bring your recipe as well. You can discuss the pros, cons, and where to improve it. In a competition a judge doesn't have your recipe in front of them. All the judge knows about your beer is the category that it has been placed in. You're not going to get the feedback you're looking for in this scenario.

Thank you both for your replies. Unfortunately, my schedule does not lend itself to me attending the local club's meetings. I may have to see if my LHBS would be willing to try a couple of my beers and see what they think. I don't know what the laws are in Maryland regarding that.

 

[Darwin18]

Another alternative would be ask around HBT to see if any BJCP judges live near you. You could also ship to a judge or two, provide them your recipe, and ask them to score and evaluate it.

 

[Darwin18]

Another valid point. I fell victim to this kind of judging once before, too. I submitted my German Pils that had a color of 3.0 SRM (2-5 is allowable per BJCP) and one judge knocked me down significant points because the "color was too dark."

You lost a point on appearance? Generally that's an automatic 2/3 or 3/3 points for everyone unless your beer has the color and consistency of motor oil.

 

[Tiber_Brew]

They're two years overdue as we speak. I do think that goes to credibility for the BJCP. The current guidelines are 6 years old. A lot has happened in the beer world since then.

I've seen the draft for the new BJCP Guidelines. I gotta say, it's drastically different, and has additions for black IPA, Kentucky Common, etc. So, yes, the beer world did/does change a lot!

 

[Tiber_Brew]

You lost a point on appearance? Generally that's an automatic 2/3 or 3/3 points for everyone unless your beer has the color and consistency of motor oil.

1/3 on appearance, and 6/10 on overall impression with yet more mention (complaint) about the color as reason. Also said something like "go easy on adjunct grains or leave out." I'm still not sure he was judging my beer that day.

It was brilliantly clear and bottled from the keg, too.

 

[jbaysurfer]

What I meant was that using a 50 point scale implies that a beer that scores a certain number is of a certain quality. i.e., a beer that scores a 36 is quite a bit beer than one that scores a 26.

I'm confused by your other statement. If the points system is supposed to be a common application of the style criteria, and adherence to the style criteria lends itself to consistency, doesn't that imply that the points system lends itself to consistency? There might be a nuance there that I'm missing, but that's what it sounds like you're saying in taking your two statements together.

You'd be better served referring to Darwin and Simps' posts the rest of the way. Gave it my best addressing your question about 26 vs. 36.

Our dialogue is going down a road too arcane and pedantic for me to have any desire to continue it. You've been respectful and I appreciate it, but I don't think I'm gonna be your huckleberry here.

 

[Darwin18]

Dude, that's a terrible break and not acceptable in my opinion. You should have forwarded your scoresheet to the competition organizer and asked them their thoughts on it....

 

[jbaysurfer]

The 2014 style guidelines are up on the BJCP forums for judges to check out.

I didn't know this! Been caught up with GABF

Thanks for the update. Edit: Thanks to Tiber brew too.

 

[Tiber_Brew]

Dude, that's a terrible break and not acceptable in my opinion. You should have forwarded your scoresheet to the competition organizer and asked them their thoughts on it....

I did. Got little-to-no sympathy. Not entering that comp anymore of course.

 

[mdbrewer1]

You'd be better served referring to Darwin and Simps' posts the rest of the way. Gave it my best addressing your question about 26 vs. 36.

Our dialogue is going down a road too arcane and pedantic for me to have any desire to continue it. You've been respectful and I appreciate it, but I don't think I'm gonna be your huckleberry here.

It takes some restraint to be respectful when others are not.....

 

[kombat]

I submitted my German Pils that had a color of 3.0 SRM (2-5 is allowable per BJCP) and one judge knocked me down significant points because the "color was too dark."

How could he/she knock you down "significant" points if "Appearance" only counts for 3 points in the overall evaluation? The worst they could have done is given you 0/3. Out of the grand scheme of things (50 possible points), that doesn't seem that significant, no?

Appearance is the thing I worry least about when submitting my beers (colour, clarity, lacing). Flavour is the big one (12 points, I think?), so I focus the majority of my efforts on minimizing off flavours by doing a good boil, quick chill, controlling my fermentation temperatures, and giving the yeast ample time to clean up.

 

[Tiber_Brew]

How could he/she knock you down "significant" points if "Appearance" only counts for 3 points in the overall evaluation? The worst they could have done is given you 0/3. Out of the grand scheme of things (50 possible points), that doesn't seem that significant, no?

Appearance is the thing I worry least about when submitting my beers (colour, clarity, lacing). Flavour is the big one (12 points, I think?), so I focus the majority of my efforts on minimizing off flavours by doing a good boil, quick chill, controlling my fermentation temperatures, and giving the yeast ample time to clean up.

To quote myself:

1/3 on appearance, and 6/10 on overall impression with yet more mention (complaint) about the color as reason. Also said something like "go easy on adjunct grains or leave out." I'm still not sure he was judging my beer that day.

It was brilliantly clear and bottled from the keg, too.

Flavor is 20 points possible. The flavor of my Pils submission was fantastic. It did end up winning 1st place, afterall, despite that one judge's comments and scoring method. I was still irked, but I'm not losing sleep. I was just adding my example of "subjective comments/scoring by a BJCP judge" to the thread.

 

[jbaysurfer]

It takes some restraint to be respectful when others are not.....

I'm gonna decide this is cryptic rather then backhanded. If I've been disrespectful to you in any way I apologize. But I rest very easy knowing I edited each post in this thread in an effort to be respectful, and any disrespect you perceived was 100% unintentional.

 

[Talgrath]

I think most people have had bad experiences with competition judges before. One thing that is hard to keep in mind is that they are just as human, mortal and to a certain extent, amateurs just like you. BJCP does have certifications and guidelines, but it's not necessarily a guarantee of good judging experiences or good judges, ultimately all of the judges are volunteers doing their best. If you have a bad experience with a competition, maybe just skip that competition; there's usually others in your area that you can enter.

 

[mdbrewer1]

I'm gonna decide this is cryptic rather then backhanded. If I've been disrespectful to you in any way I apologize. But I rest very easy knowing I edited each post in this thread in an effort to be respectful, and any disrespect you perceived was 100% unintentional.

Your last post sounded like you decided that this thread was beneath you and you couldn't be bothered to continue the discussion. The use of arcane doesn't make much sense and I feel that most of the time people call things pedantic when they have run out of ways to contribute to a discussion. I had to look up the huckleberry line and it further reinforces my thought that you are thumbing your nose at the thread and decided to let someone else stoop down to my level to continue to educate me.

My take is that you like the current system and that's fine. People tend to want to stick to what they are familiar with, even if it has its flaws, and can get defensive when it is challenged. I can be that same way on certain topics.

The constructive responses have been very informative for me though. It has changed my opinion on the usefulness of the numerical judging results and raised questions for me on the implementation of a points based system in general. As it currently stands, it sounds to me that one would need to see each scoresheet within a category to gain much information and that the numbers by themselves hold little to no value from competition to competition.

That said, an imperfect system might be the best realistic solution for what is ultimately a subjective assessment. Thank you to those who have shed some light on something I knew little about.

 

[TheZymurgist]

It's amazing how these threads always deteriorate into such pointless crap...

 

[jbaysurfer]

Your last post sounded like you decided that this thread was beneath you and you couldn't be bothered to continue the discussion. The use of arcane doesn't make much sense and I feel that most of the time people call things pedantic when they have run out of ways to contribute to a discussion. I had to look up the huckleberry line and it further reinforces my thought that you are thumbing your nose at the thread and decided to let someone else stoop down to my level to continue to educate me.

My take is that you like the current system and that's fine. People tend to want to stick to what they are familiar with, even if it has its flaws, and can get defensive when it is challenged. I can be that same way on certain topics.

The constructive responses have been very informative for me though. It has changed my opinion on the usefulness of the numerical judging results and raised questions for me on the implementation of a points based system in general. As it currently stands, it sounds to me that one would need to see each scoresheet within a category to gain much information and that the numbers by themselves hold little to no value from competition to competition.

That said, an imperfect system might be the best realistic solution for what is ultimately a subjective assessment. Thank you to those who have shed some light on something I knew little about.

All I can say is that you have misread me very badly. RDWHAHB.

 

[Sparge]

After reading this thread, I don't think I ever want to participate in a contest.

 

[jbaysurfer]

^ More medals up for grabs! Hopefully next year NHC will have fewer then 9000 entries to compete against.

 

[Minky]

The funniest comment I've ever gotten on a score sheet was for an APA I entered in the Ohio State Fair. In the "overall impression" category the judge said (and this is exactly as written): "Keep trying - It look great! It taste bad, but it's the best we've had today!"

Wow, you'd like to think the judges would use a little more tact than that, but I scored a 31 and got a 2nd out of 44 entries so I'll take it!!

 

[myelo]

I would be interested if there are any actual studies on the reliability of beer tasting competitions. There's a fair amount of published research on wine testing and the bottom line is it's not reliable.

Like, at all.

One study analyzed judge performance at a major wine competition from 2005 to 2008 using replicate samples and found only about 10% of the judges were able to replicate their score within a single medal group. (Hodgson Journal of Wine Economics 2008)

In a study, of over 4000 wines entered in 13 competitions, the probability of winning a Gold medal at one competition was stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone. (Hodgson Journal of Wine Economics 2009)

I would not be surprised if beer tasting was just as unreliable.

 

[podz]

The funniest comment I've ever gotten on a score sheet was for an APA I entered in the Ohio State Fair. In the "overall impression" category the judge said (and this is exactly as written): "Keep trying - It look great! It taste bad, but it's the best we've had today!"

I'm not usually a grammar nazi, but you would also like to think that judges would possess the skills to properly conjugate two of the most common verbs in the language which they are using to describe the beer evaluation. After all, it's their job to write evaluations.

Wow, you'd like to think the judges would use a little more tact than that, but I scored a 31 and got a 2nd out of 44 entries so I'll take it!!

So a beer that tasted worse than yours won first place?

 

[Minky]

I'm not usually a grammar nazi, but you would also like to think that judges would possess the skills to properly conjugate two of the most common verbs in the language which they are using to describe the beer evaluation. After all, it's their job to write evaluations.

I was also a little surprised to see such a mangling of the language!

So a beer that tasted worse than yours won first place?

Evidently, my entry wasn't the last one they judged, huh?

BTW, sorry for messing up your quote. Here's the results from the state fair competition, and I could include a scan of the score sheet, if you need it.

http://www.ohiostatefair.com/index.php/news-a-media-48/item/526-homebrew-results-available-now

 

[rawlus]

among those certified as judges, I'm going to expect a minor few to be truly at the top of their game. and most being real beer lovers and loving what they do as judges and supporting the hobby and just all around great, well-intentioned people.

pro-level judging is exceedingly difficult to do. I've taken courses to try and get there in wine judging and ended up not continuing due to the demands of doing so at a high confidence level.

on top of this, flaw training takes the right tools and lots of practice and reinforcement. I saw a beer flaw kit in BYO mag I think recently, this would be a great training aid for anyone interests in improving their ability to detect flaws and train their brain to instinctively identify those flaws without having to think about it. in wine appreciation I had a similar kit from le nez du vin which was 50+ flaws and other characteristics, that really helped to create the necessary brain/scent/taste associations to speak somewhat intelligently about wine.
I think to be able to speak the same way about beer characteristics you really need a broad exposure to something like this...

I think in smaller events the judges might tend to be really great people who really like supporting the hobby, take their role seriously and really try their best to do the best and most honest job possible. it's sort of a thankless job but I think most of these se participants do the best they can with what they have, the experience they have, the trianing they've had and so forth.

the reality however is that this is a skill that requires a great deal of exposure to beer in a judging/tasting environment and practice and reinforcement of the articulation and recognition of characteristics of taste, appearance, and so forth. people who have livelihoods which afford this exposure and reinforcement can do very well and have very advanced skills in both recognition and articulation, people who do this as a hobby, to support the hobby and fellow Brewers, are at a natural disadvantage.

try not to take it personally if something you feel is your best work doesn't get the score you think it deserved. there are so many variables in play. I equate it to judging pies at the local fair. it's fun to participate in the contest and it's fun to win or place, but it's still a hobby and nylon don't want to let the seriousness of judging and competing ruin your enjoyment of your hobby.