competiton scoring error by judge

HomeBrewTalk.com - Beer, Wine, Mead, & Cider Brewing Discussion Community.

Help Support Homebrew Talk:

Joined
Jun 2, 2008
Messages
64,957
Reaction score
16,503
First off, let me preface this by the fact that I do not care at all that a judge made an error - I am always pleased to get feedback on my beers.

But,

just curious - my score sheet listed the following:

Aroma = 9
Appearance = 2
Flavor = 16
Mouthfeel = 4
Overall = 8
Total = 37 (9+2+16+4+8 should = 39)

Again, I don't care - my beer was good and the judges agreed, but just wondering how often this happens, and why.

Oh, and it did mean the difference between a silver and broze - but again, I seriously don't care about that :D
 

Yankeehillbrewer

Well-Known Member
Joined
Dec 7, 2008
Messages
1,561
Reaction score
38
Location
Meridian, ID
If you really didn't care, you would not know that it made a difference in your placing. And, you should care about those details. IMO, there is no room for error if your judging something, no matter what it is. If you're judging something, you should be an expert in that field. At the very least you should know basic addition.

I know that's not the answer you're looking for, but I feel that kind of mistake is unacceptable.
 
OP
AZ_IPA
Joined
Jun 2, 2008
Messages
64,957
Reaction score
16,503
ok, I do care dammit :) but not enough to do anything about it. I was pleased with the feed back I received but when I saw the scoresheet, the math just didn't add up
 

Yankeehillbrewer

Well-Known Member
Joined
Dec 7, 2008
Messages
1,561
Reaction score
38
Location
Meridian, ID
I can imagine that the feedback is invaluable. Someday I'll enter my brews in a competition. FWIW, if you can, you should consider letting the judge know about the mistake, in a nice way. He or she would probably really appreciate that, and if they really care, they will pay more attention so they don't make that mistake again. In the long run, it will make them better at what they do.

You can't correct a mistake unless you know you're making it.:mug:
 

z987k

Well-Known Member
Joined
Feb 13, 2007
Messages
3,512
Reaction score
35
Location
Anchorage
If you really didn't care, you would not know that it made a difference in your placing. And, you should care about those details. IMO, there is no room for error if your judging something, no matter what it is. If you're judging something, you should be an expert in that field. At the very least you should know basic addition.

I know that's not the answer you're looking for, but I feel that kind of mistake is unacceptable.
Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.
 
Joined
Jul 16, 2006
Messages
7,357
Reaction score
1,175
Location
Redding Ca
If you placed, and that is the difference between silver and bronze. You bet your A$$ I would let them know. Some would ask why... But I always ask, Why not?
 

Yankeehillbrewer

Well-Known Member
Joined
Dec 7, 2008
Messages
1,561
Reaction score
38
Location
Meridian, ID
Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.
no, but what does it matter?? Math is Math, the judge was wrong.
 

carnevoodoo

Well-Known Member
Joined
May 17, 2007
Messages
4,258
Reaction score
24
Location
San Diego, CA
If you placed, and that is the difference between silver and bronze. You bet your A$$ I would let them know. Some would ask why... But I always ask, Why not?
I've never been in a comp where they go strictly by score to award places. They'll get the top 5 or so and they'll do a mini BOS and they'll pick that way.
 

DanVader

Well-Known Member
Joined
Apr 19, 2008
Messages
98
Reaction score
0
Location
tucson
that is pretty goofy, I'd agree that the judges picked 1,2,3 and forgot to adjust the score (maybe?) I've gotten a bronze for a beer that scored 1 point under the minimum for gold, a silver would have been cool, but the 2 guys ahead of me did better.

out of the 3 judges, only one was BJCP, and he gave me a 42, that made me feel pretty good.
 

Beerrific

Well-Known Member
Joined
Mar 4, 2007
Messages
5,537
Reaction score
61
Location
Georgia
At a minimum you should let the competition director know. Every comp. I have been to (stewarded or judged) the director mentions several times: "double check your math" and "stewards, please double check the judge's math." I have miss-added on score sheets before, and the steward has caught it (hopefully every time). Sometimes you get to the end of a 13 beer flight simple math can easily escape, not that it was easy to begin with.
 

big supper

Well-Known Member
Joined
Jul 29, 2007
Messages
424
Reaction score
2
Location
Montreal
I would let the competition know, and not the judge.

I was a steward this year, and I double checked the scores for all of the flights that I was working with.
 

FlyGuy

Well-Known Member
Joined
Jan 13, 2007
Messages
3,604
Reaction score
224
Location
Calgary, Alberta
That sort of thing is unfortunate, but it happens. Judges are humans, and if you have ever judged three sessions in a row (and 30 or so beer later), you can start to understand how it happens. That doesn't make it acceptable, but I hope people can try to be understanding that a judge might make the occasional math error.

Regardless, a small error like two points by one judge will rarely affect the order of how a beer places. Whenever the average scores by all judges in a flight are within a point or two, those beers typically get re-evaluated head-to-head. The scores themselves don't determine everything.
 

Bokonon

Well-Known Member
Joined
Mar 12, 2007
Messages
1,279
Reaction score
16
Location
Tri-Cities, WA
There definitely is an error somewhere but it might not be exactly where you think it is. One way to judge is to give points for each category and add them up another way is to decide the total score and then come up with category points to make that work.

So its possible that your score was what the judge intended but they didn't put the right numbers above to make it work.
 

mmb

"I just got a new pet toaster!"
Lifetime Supporter
Joined
Feb 20, 2008
Messages
40,702
Reaction score
9,906
Location
Mid Mittigan
That should have been caught by the stewards or contest director. I do contest judging and contest administration for an educational compilation (not beer) and the judge double checks, the proctors/administrators double check, and then there is a "grading room" where everything is check again.

We're talking scholarships and not ribbons as rewards, but the idea is the same.
 

Sigafoos

Well-Known Member
Joined
Aug 1, 2007
Messages
847
Reaction score
4
Location
Buffalo, NY
Yeah, that's not the judge's fault, it's the steward's. One of my sheets was off by a point I think, but ti was low enough where I doubt I'd have gotten a bronze or anything. Definitely let the competition coordinator know; even if you don't care, it's good for them to know to say 'hey stewards, count more carefully next year.' I'm glad you're taking it in stride (I can see people emailing them and saying WTF I DESERVE MOAR PRIZES!!!), but an FYI is always helpful.
 
OP
AZ_IPA
Joined
Jun 2, 2008
Messages
64,957
Reaction score
16,503
Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.
Yes, that's clearly what happened - the judge was 3 points higher than the other judge and brought it down to 37 rather than 39 (other judge had it at 36); but the numbers didn't add up.

Thinking about it more - it's a clerical error - the judges agreed that the beer deserved a 36 or 37 -- the score sheets just didn't reflect that.

I'm sure this stuff happens all the time - I haven't been to a contest or judged, but I'm sure it's pretty hectic trying to accurately rate beers and be as consistent among judges as possible.

I'll take my bronze and be happy :mug:

Thanks for everyone's comments
 

taylornate

Well-Known Member
Joined
Jan 10, 2009
Messages
224
Reaction score
2
Location
Springfield, IL
another way is to decide the total score and then come up with category points to make that work.

So its possible that your score was what the judge intended but they didn't put the right numbers above to make it work.
Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.
Are contests really run this way? It really does not seem ethical to me.
 

cactusgarrett

Well-Known Member
Joined
Apr 8, 2008
Messages
2,003
Reaction score
654
Location
Madison, WI
Seems like it should be, as evaluating a beer should be more of a qualitative process, as opposed to a quantitiative, in my opinion. I think it would be more wrong if you had two beers side-by-side and crowned one a winner when the judges think that overall the other tastes better, despite numerical values.
 

Sigafoos

Well-Known Member
Joined
Aug 1, 2007
Messages
847
Reaction score
4
Location
Buffalo, NY
Are contests really run this way? It really does not seem ethical to me.
It seemed weird to me too, but it actually makes a lot of sense.

Generally, when judges disagree a lot, there's something they haven't considered. One will say 'this is amazing' while the other will say 'this is horrible,' but then they each say why they like/dislike it and see the other person's side. They never agree exactly, but you can still be 7 points apart (which is a lot). If you think a beer is amazing but someone else thinks it's cow piss, chances are you're overlooking some flaw and they're putting too much weight on it and are missing some of its good points.
 

FlyGuy

Well-Known Member
Joined
Jan 13, 2007
Messages
3,604
Reaction score
224
Location
Calgary, Alberta
Generally, when judges disagree a lot, there's something they haven't considered. One will say 'this is amazing' while the other will say 'this is horrible,' but then they each say why they like/dislike it and see the other person's side. They never agree exactly, but you can still be 7 points apart (which is a lot). If you think a beer is amazing but someone else thinks it's cow piss, chances are you're overlooking some flaw and they're putting too much weight on it and are missing some of its good points.
That's more or less the way it works. Before each judge in the flight finalizes their score for a particular beer, everyone checks their scores to see if they are all close to one another. Often they are not, it is a red flag that someone may have missed something. A quick discussion ensues among the judges about why the scored the way they did, and a quick re-evaluation takes place. In most cases, the judges will be honest and recognize if they missed something important and adjust their overall score, and then compensate their comments and component scores to match. The point of this is to be as fair as possible so that nothing is missed (good or bad).

Occasionally, however, judges will not agree because everyone has slightly different perception (that's expected). In those circumstances, judges try to get their scores only as close as they are comfortable (generally within 7 points, if they can), but they are not required to necessarily agree. Those can be quite interesting evaluations to get back because the judges obviously had strong opinions about their perceptions, enough that they weren't willing to back down on their scores. I pay close attention to these scoresheets because it is likely that they judges spent a lot of time evaluating that beer, relative to the others in the flight.
 

z987k

Well-Known Member
Joined
Feb 13, 2007
Messages
3,512
Reaction score
35
Location
Anchorage
Seems like it should be, as evaluating a beer should be more of a qualitative process, as opposed to a quantitiative, in my opinion. I think it would be more wrong if you had two beers side-by-side and crowned one a winner when the judges think that overall the other tastes better, despite numerical values.
It really is both, and that can be one of the reasons a final score is changed and the numbers don't add up.
 
Top