Are beer judges this fickel too?

[Renoun]

There's an article in The London Telegraph describing how bad some wine judges are. I can't imagine that beer judges are much better but I don't enter competitions so I can't claim to care that much. I like what I like, if I enjoy it I'll acquire more.

Is everything we know about wine wrong?

By Will Storr1:00PM BST 29 Apr 2014Comments247 Comments
If you’re in the business of fermenting grapes for money, you’re going to want a medal from the California State Wine Fair. A bronze, silver or gold from that event means recognition, esteem and a solid boost to your sales figures. This is not only the oldest judging competition in the US – it started in 1854 – but also one of the most prestigious. Every year, its 70-strong array of noses and tongues, which belong to many of the best critics, sommeliers, academics and winemakers in the nation, pass judgment on about 2,500 bottles.
But how meaningful are their results? A few years ago, a frustrated vintner named Robert Hodgson, who had a background in statistics, thought of a way of testing the testers. He wondered what would happen if he supplied 100 wines for consideration but, without the judges knowing, slipped each wine to them three different times. Would they notice? Surely, with their trained and articulate palates, they’d at least be consistent in how they rated the identical drinks?

 

[JBOGAN]

I share Hodgsons sentiments across the board on this one and for me its really about getting some feedback and deducing what is valid or not. In a perfect world all certified beer judges would enjoy every style they taste but I know that not to be the truth. Then again what I am really looking for is off flavors and slight flaws and not so much someones ability to pick up thirty different floral scents and or tastes. Yes I have seen a review on Beer Advocate where some one actually described 30 different notes...In the end I will still make beer at home and as long as I deem it drinkable than FTW.

 

[organ]

There's a documentary on Netflix called Somm. Shows how ridiculous that whole world is... and I'd say the same goes for the world of beer judging.

 

[strambo]

I'm not sure about wine judging, but I like that beer is judged against style guidelines so it is at least rooted in something more objective to start.

"The best" is always very subjective, but hopefully flaws or out of style traits would be more consistently noted. I only entered 1, got a 30, thought they were being nice. I did get good feedback which is all that mattered.


Sent from my iPhone using Home Brew

 

[Phunhog]

Probably worse. The wine industry has several extensive tasting programs. Beer has the Cicerone and the BJCP but they are both rather new. Also most BJCP judges are not in the industry but rather homebrewers.

 

[Beernik]

I've heard this about wine tasters for a long time. Some being so bad that they didn't notice a white wine had a red wine label on the bottle.

I suck at wine tasting. I don't have the taste references for it. I'm much better with beer. But I'm also more passionate about beer.

If you read ratebeer you will see people that are just as bad in their tasting notes. I've read some and thought "no way in hell are you tasting that in this beer" becauae none of the ingredients had that flavor.

Personally, I've never put any stock in "this beer won this medal in this festival." I figure any beer entered in enough festivals will eventually win some award.

 

[kaconga]

The differences between wine and beer is the style guidelines and the ingredients. Wine is very boring in its recipe. When your biggest decision is whether you want the skins on or off then it isn't really about what you do in the process of making wine but rather how you grow the grapes. Beer has tons of unique ingredients and a variety of yeast which can produce a vast array of distinct flavors. This is what sets beer apart from wine. It is not that wine is simple, just that it is very limited. Beer is limitless and even today we are pushing the bjcp to recognize more styles such as cascadian dark or black IPA. So forty different Bordeaux all taste similar, is that surprising. That would be the equivalent of tasting 40 different mo/ekg smash beers. I am not surprised in the least.

 

[mysteryshrimp]

I've entered one spiced cider into five competitions. 2 Best Of Shows (in the 40s), 2 scored in the teens, one in the mid twenties.

The same Smoked Lambic got a 16 and a 38 on back-to-back weekends in the Belgian Specialty category.

My take: in the pretty standard categories, we are as solid as a group of humans with far differing experience levels can be. In the outlying styles like "other smoked", "spice, herb, and vegetable", fruit beers, any specialty, any sour, any cider, it's a toss-up. Most of the people best qualified to judge those have entered something and can't judge the category.

 

[opiate82]

I can't remember if I read or heard it on a podcast, but I heard of a similar scenario being used on BCJP judges with similar results. Beers scored "widely" differently 3 different times by the same judges.

Ultimately there is so much that goes into tasting, both subjectively as well as palate-wise (palate fatigue and what not) you really need to understand the human element when it comes to any sort of judged contest. You can get useful information, especially in regards to off-flavors. But if you're in-it-to-win-it, that can be a little bit of a crap-shoot.

 

[Phunhog]

Even if you look at commercial beer comps (GABF, World Beer Cup) there is a lot of variability in judging. Commercial brewers really try to make their beer the same every time. So it begs the question that if won first last year it "should" win first the following year.....or at least place. Unless of course there is some unknown variable ( new beer placing, change in recipe, etc) I think at the commercial level the judging/awards is strictly for marketing purposes. Otherwise the same half dozen (fantastic) beers in each category would win every year. That doesn't happen though in most cases. It seems like it is a rotating cast of beers that win one year and then they never win again.

 

[SixDemonBag]

There's a documentary on Netflix called Somm. Shows how ridiculous that whole world is... and I'd say the same goes for the world of beer judging.

I really enjoyed this. Thanks for the tip!

 

[biereblanche]

Beer judges aren't fickle

We might be perceived as arrogant, ignorant, or incongruent
We might appear to have our heads in the clouds or other less sunny places
We might be a touch mercurial
or have certain dubious notes
Sometimes we are presumptuous, pretentious, presumptive and impudent
Some judges have hints of know-it-all-idness with supporting self-righteousness
and not much distinguishable wrong-ness

But if you describe beer judges as fickle, you're probably just mis-perceiving us.

 

[mmurray]

Going from my own experience judging... I judged a Herb,Spice, Vegetable comp a few weekends ago and our first round top beer was a Belgian Sour and I can't remember all that was in it, but it was really good. In retrospect I remember another judge mentioning a slight lemon but it was transient. We went back and tasted again to see if lemon was there and we just couldn't get back there so we disregarded that flavor. So that beer went on to BOS (top 2 beers from each judging station) and was the first beer kicked... we were surprised so we snitched a sip from the 2nd bottle they were using since they didn't need it anymore and LEMON was all over that beer... aroma and flavor it was like I got sprayed with lemon pledge. 2 bottles of the same beer and they were completely different!

So for me I was saddened for a few reasons. 1. such a good beer loses out because the homebrewer messed up likely during packaging. 2. Had we had the BOS bottle, that beer would not have made out of the first round and the tables #3 beer that lost by 1/2 point would have went on to BOS.

So one persons mistake, kinda screwed over a few people.

 

[Denny]

I wrote a chapter about evaluation in the book I just finished and I detail MANY tests like this in both the beer and wine world. It's proof of how hard it is to objectively evaluate your own beer.

 

[JBOGAN]

I wrote a chapter about evaluation in the book I just finished and I detail MANY tests like this in both the beer and wine world. It's proof of how hard it is to objectively evaluate your own beer.

I agree with this for most people and me at times. Being an artist myself i tend to be my worst critic and i treat my beer the same way even if it is a well liked beer amongst my peers.

 

[pdietert]

I wrote a chapter about evaluation in the book I just finished and I detail MANY tests like this in both the beer and wine world. It's proof of how hard it is to objectively evaluate your own beer.

Hey Denny, When will this book be available? I would very much like a copy (an autographed one if one is available)

 

[krackin]

Fickle, maybe. Pickled, good chance.

 

[pdietert]

Taste is so subjective. In my own experience it can differ from day to day and can also be influenced by so many factors. Try drinking a dopplebock like Salvator and then tasting any other kind of beer. Its next to impossible for me even with any kind of "palate cleanser". Your sense of smell has so much to do with it also. Next time you are in the mall, go through a department store perfume department and then sit down at a restaurant and have a beer that you have had before, it will taste different. I know these are sensory shockers but it does happen on the small scale also and most times we are not aware of these factors.

 

[helibrewer]

I've started looking at comps from a different perspective. I submit my beers that I questions about...some flaw that I can't identify or a beer I just don't like...that way I get judges opinions on what's wrong and maybe how to correct it. I already know my good beer is good, I don't need a judge to affirm it...I want to know why my bad beer is bad.

 

[Denny]

Hey Denny, When will this book be available? I would very much like a copy (an autographed one if one is available)

Nov. 2014. Got the galleys to proof today and it's GORGEOUS! I'd be happy to autograph one for ya if you come to Eugene, or I happen to be in Houston!

 

[pdietert]

Nov. 2014. Got the galleys to proof today and it's GORGEOUS! I'd be happy to autograph one for ya if you come to Eugene, or I happen to be in Houston!

Eagerly anticipating the release...

 

[organ]

Beer judges aren't fickle

We might be perceived as arrogant, ignorant, or incongruent
We might appear to have our heads in the clouds or other less sunny places
We might be a touch mercurial
or have certain dubious notes
Sometimes we are presumptuous, pretentious, presumptive and impudent
Some judges have hints of know-it-all-idness with supporting self-righteousness
and not much distinguishable wrong-ness

But if you describe beer judges as fickle, you're probably just mis-perceiving us.

Well, you sure know a lot of adjectives... checks out, guys! How can you tell a guy's a beer judge? He'll tell ya!