I think I am done with contests

[Tiber_Brew]

You lost a point on appearance? Generally that's an automatic 2/3 or 3/3 points for everyone unless your beer has the color and consistency of motor oil.

1/3 on appearance, and 6/10 on overall impression with yet more mention (complaint) about the color as reason. Also said something like "go easy on adjunct grains or leave out." I'm still not sure he was judging my beer that day.

It was brilliantly clear and bottled from the keg, too.

 

[jbaysurfer]

What I meant was that using a 50 point scale implies that a beer that scores a certain number is of a certain quality. i.e., a beer that scores a 36 is quite a bit beer than one that scores a 26.

I'm confused by your other statement. If the points system is supposed to be a common application of the style criteria, and adherence to the style criteria lends itself to consistency, doesn't that imply that the points system lends itself to consistency? There might be a nuance there that I'm missing, but that's what it sounds like you're saying in taking your two statements together.

You'd be better served referring to Darwin and Simps' posts the rest of the way. Gave it my best addressing your question about 26 vs. 36.

Our dialogue is going down a road too arcane and pedantic for me to have any desire to continue it. You've been respectful and I appreciate it, but I don't think I'm gonna be your huckleberry here.

 

[Darwin18]

Dude, that's a terrible break and not acceptable in my opinion. You should have forwarded your scoresheet to the competition organizer and asked them their thoughts on it....

 

[jbaysurfer]

The 2014 style guidelines are up on the BJCP forums for judges to check out.

I didn't know this! Been caught up with GABF

Thanks for the update. Edit: Thanks to Tiber brew too.

 

[Tiber_Brew]

Dude, that's a terrible break and not acceptable in my opinion. You should have forwarded your scoresheet to the competition organizer and asked them their thoughts on it....

I did. Got little-to-no sympathy. Not entering that comp anymore of course.

 

[mdbrewer1]

You'd be better served referring to Darwin and Simps' posts the rest of the way. Gave it my best addressing your question about 26 vs. 36.

Our dialogue is going down a road too arcane and pedantic for me to have any desire to continue it. You've been respectful and I appreciate it, but I don't think I'm gonna be your huckleberry here.

It takes some restraint to be respectful when others are not.....

 

[kombat]

I submitted my German Pils that had a color of 3.0 SRM (2-5 is allowable per BJCP) and one judge knocked me down significant points because the "color was too dark."

How could he/she knock you down "significant" points if "Appearance" only counts for 3 points in the overall evaluation? The worst they could have done is given you 0/3. Out of the grand scheme of things (50 possible points), that doesn't seem that significant, no?

Appearance is the thing I worry least about when submitting my beers (colour, clarity, lacing). Flavour is the big one (12 points, I think?), so I focus the majority of my efforts on minimizing off flavours by doing a good boil, quick chill, controlling my fermentation temperatures, and giving the yeast ample time to clean up.

 

[Tiber_Brew]

How could he/she knock you down "significant" points if "Appearance" only counts for 3 points in the overall evaluation? The worst they could have done is given you 0/3. Out of the grand scheme of things (50 possible points), that doesn't seem that significant, no?

Appearance is the thing I worry least about when submitting my beers (colour, clarity, lacing). Flavour is the big one (12 points, I think?), so I focus the majority of my efforts on minimizing off flavours by doing a good boil, quick chill, controlling my fermentation temperatures, and giving the yeast ample time to clean up.

To quote myself:

1/3 on appearance, and 6/10 on overall impression with yet more mention (complaint) about the color as reason. Also said something like "go easy on adjunct grains or leave out." I'm still not sure he was judging my beer that day.

It was brilliantly clear and bottled from the keg, too.

Flavor is 20 points possible. The flavor of my Pils submission was fantastic. It did end up winning 1st place, afterall, despite that one judge's comments and scoring method. I was still irked, but I'm not losing sleep. I was just adding my example of "subjective comments/scoring by a BJCP judge" to the thread.

 

[jbaysurfer]

It takes some restraint to be respectful when others are not.....

I'm gonna decide this is cryptic rather then backhanded. If I've been disrespectful to you in any way I apologize. But I rest very easy knowing I edited each post in this thread in an effort to be respectful, and any disrespect you perceived was 100% unintentional.

 

[Talgrath]

I think most people have had bad experiences with competition judges before. One thing that is hard to keep in mind is that they are just as human, mortal and to a certain extent, amateurs just like you. BJCP does have certifications and guidelines, but it's not necessarily a guarantee of good judging experiences or good judges, ultimately all of the judges are volunteers doing their best. If you have a bad experience with a competition, maybe just skip that competition; there's usually others in your area that you can enter.

 

[mdbrewer1]

I'm gonna decide this is cryptic rather then backhanded. If I've been disrespectful to you in any way I apologize. But I rest very easy knowing I edited each post in this thread in an effort to be respectful, and any disrespect you perceived was 100% unintentional.

Your last post sounded like you decided that this thread was beneath you and you couldn't be bothered to continue the discussion. The use of arcane doesn't make much sense and I feel that most of the time people call things pedantic when they have run out of ways to contribute to a discussion. I had to look up the huckleberry line and it further reinforces my thought that you are thumbing your nose at the thread and decided to let someone else stoop down to my level to continue to educate me.

My take is that you like the current system and that's fine. People tend to want to stick to what they are familiar with, even if it has its flaws, and can get defensive when it is challenged. I can be that same way on certain topics.

The constructive responses have been very informative for me though. It has changed my opinion on the usefulness of the numerical judging results and raised questions for me on the implementation of a points based system in general. As it currently stands, it sounds to me that one would need to see each scoresheet within a category to gain much information and that the numbers by themselves hold little to no value from competition to competition.

That said, an imperfect system might be the best realistic solution for what is ultimately a subjective assessment. Thank you to those who have shed some light on something I knew little about.

 

[TheZymurgist]

It's amazing how these threads always deteriorate into such pointless crap...

 

[jbaysurfer]

Your last post sounded like you decided that this thread was beneath you and you couldn't be bothered to continue the discussion. The use of arcane doesn't make much sense and I feel that most of the time people call things pedantic when they have run out of ways to contribute to a discussion. I had to look up the huckleberry line and it further reinforces my thought that you are thumbing your nose at the thread and decided to let someone else stoop down to my level to continue to educate me.

My take is that you like the current system and that's fine. People tend to want to stick to what they are familiar with, even if it has its flaws, and can get defensive when it is challenged. I can be that same way on certain topics.

The constructive responses have been very informative for me though. It has changed my opinion on the usefulness of the numerical judging results and raised questions for me on the implementation of a points based system in general. As it currently stands, it sounds to me that one would need to see each scoresheet within a category to gain much information and that the numbers by themselves hold little to no value from competition to competition.

That said, an imperfect system might be the best realistic solution for what is ultimately a subjective assessment. Thank you to those who have shed some light on something I knew little about.

All I can say is that you have misread me very badly. RDWHAHB.

 

[Sparge]

After reading this thread, I don't think I ever want to participate in a contest.

 

[jbaysurfer]

^ More medals up for grabs! Hopefully next year NHC will have fewer then 9000 entries to compete against.

 

[Minky]

The funniest comment I've ever gotten on a score sheet was for an APA I entered in the Ohio State Fair. In the "overall impression" category the judge said (and this is exactly as written): "Keep trying - It look great! It taste bad, but it's the best we've had today!"

Wow, you'd like to think the judges would use a little more tact than that, but I scored a 31 and got a 2nd out of 44 entries so I'll take it!!

 

[myelo]

I would be interested if there are any actual studies on the reliability of beer tasting competitions. There's a fair amount of published research on wine testing and the bottom line is it's not reliable.

Like, at all.

One study analyzed judge performance at a major wine competition from 2005 to 2008 using replicate samples and found only about 10% of the judges were able to replicate their score within a single medal group. (Hodgson Journal of Wine Economics 2008)

In a study, of over 4000 wines entered in 13 competitions, the probability of winning a Gold medal at one competition was stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone. (Hodgson Journal of Wine Economics 2009)

I would not be surprised if beer tasting was just as unreliable.

 

[podz]

The funniest comment I've ever gotten on a score sheet was for an APA I entered in the Ohio State Fair. In the "overall impression" category the judge said (and this is exactly as written): "Keep trying - It look great! It taste bad, but it's the best we've had today!"

I'm not usually a grammar nazi, but you would also like to think that judges would possess the skills to properly conjugate two of the most common verbs in the language which they are using to describe the beer evaluation. After all, it's their job to write evaluations.

Wow, you'd like to think the judges would use a little more tact than that, but I scored a 31 and got a 2nd out of 44 entries so I'll take it!!

So a beer that tasted worse than yours won first place?

 

[Minky]

I'm not usually a grammar nazi, but you would also like to think that judges would possess the skills to properly conjugate two of the most common verbs in the language which they are using to describe the beer evaluation. After all, it's their job to write evaluations.

I was also a little surprised to see such a mangling of the language!

So a beer that tasted worse than yours won first place?

Evidently, my entry wasn't the last one they judged, huh?

BTW, sorry for messing up your quote. Here's the results from the state fair competition, and I could include a scan of the score sheet, if you need it.

http://www.ohiostatefair.com/index.php/news-a-media-48/item/526-homebrew-results-available-now

 

[rawlus]

among those certified as judges, I'm going to expect a minor few to be truly at the top of their game. and most being real beer lovers and loving what they do as judges and supporting the hobby and just all around great, well-intentioned people.

pro-level judging is exceedingly difficult to do. I've taken courses to try and get there in wine judging and ended up not continuing due to the demands of doing so at a high confidence level.

on top of this, flaw training takes the right tools and lots of practice and reinforcement. I saw a beer flaw kit in BYO mag I think recently, this would be a great training aid for anyone interests in improving their ability to detect flaws and train their brain to instinctively identify those flaws without having to think about it. in wine appreciation I had a similar kit from le nez du vin which was 50+ flaws and other characteristics, that really helped to create the necessary brain/scent/taste associations to speak somewhat intelligently about wine.
I think to be able to speak the same way about beer characteristics you really need a broad exposure to something like this...

I think in smaller events the judges might tend to be really great people who really like supporting the hobby, take their role seriously and really try their best to do the best and most honest job possible. it's sort of a thankless job but I think most of these se participants do the best they can with what they have, the experience they have, the trianing they've had and so forth.

the reality however is that this is a skill that requires a great deal of exposure to beer in a judging/tasting environment and practice and reinforcement of the articulation and recognition of characteristics of taste, appearance, and so forth. people who have livelihoods which afford this exposure and reinforcement can do very well and have very advanced skills in both recognition and articulation, people who do this as a hobby, to support the hobby and fellow Brewers, are at a natural disadvantage.

try not to take it personally if something you feel is your best work doesn't get the score you think it deserved. there are so many variables in play. I equate it to judging pies at the local fair. it's fun to participate in the contest and it's fun to win or place, but it's still a hobby and nylon don't want to let the seriousness of judging and competing ruin your enjoyment of your hobby.

 

[rawlus]

I would be interested if there are any actual studies on the reliability of beer tasting competitions. There's a fair amount of published research on wine testing and the bottom line is it's not reliable.

Like, at all.

One study analyzed judge performance at a major wine competition from 2005 to 2008 using replicate samples and found only about 10% of the judges were able to replicate their score within a single medal group. (Hodgson Journal of Wine Economics 2008)

In a study, of over 4000 wines entered in 13 competitions, the probability of winning a Gold medal at one competition was stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone. (Hodgson Journal of Wine Economics 2009)

I would not be surprised if beer tasting was just as unreliable.

this is a fair point. there are a few things in play here though. one is wine appreciation, for lack of a better term... this is the skill acquired necessary to be able to taste a wine and talk about it's qualities and characteristics specifically and accurately.... what grapes were used, where were they grown, what vintage, what producer in some cases, is it juicy or flinty? did it spend time in French oak or American oak? new or used barrels? and so forth. with training these are things that are factual that an experienced taster can be accurate and articulate about.

the other side is "rating" - is this a 100 or a 98? to me this is informed by some guidelines but is also very very influenced by individual experience and as such can be subjective. when you're eating a wine you're , to some degree, rating it against all similar wines you've ever had. if that library of similar wines is not massive, then the experience upon which you draw your comparison can influence the score. this, combined with the personal theory and belief system you will naturally impose on wines you taste based upon your experience, will also create differences between your scores and someone else's scores. you may believe in more oaky Cabernets and someone else may prefer intense fruit.... so your scores of the same wine may be affected by the differing beliefs.

this can certainly play a role in beer as well. a style guideline is merely a guideline, it is impossible to account for standardization of personal preference and it would be impossible to rate so,etching without considering personal preference to some degree.