competiton scoring error by judge

[bob1852]

First off, let me preface this by the fact that I do not care at all that a judge made an error - I am always pleased to get feedback on my beers.

But,

just curious - my score sheet listed the following:

Aroma = 9
Appearance = 2
Flavor = 16
Mouthfeel = 4
Overall = 8
Total = 37 (9+2+16+4+8 should = 39)

Again, I don't care - my beer was good and the judges agreed, but just wondering how often this happens, and why.

Oh, and it did mean the difference between a silver and broze - but again, I seriously don't care about that

 

[Yankeehillbrewer]

If you really didn't care, you would not know that it made a difference in your placing. And, you should care about those details. IMO, there is no room for error if your judging something, no matter what it is. If you're judging something, you should be an expert in that field. At the very least you should know basic addition.

I know that's not the answer you're looking for, but I feel that kind of mistake is unacceptable.

 

[bob1852]

ok, I do care dammit but not enough to do anything about it. I was pleased with the feed back I received but when I saw the scoresheet, the math just didn't add up

 

[Yankeehillbrewer]

I can imagine that the feedback is invaluable. Someday I'll enter my brews in a competition. FWIW, if you can, you should consider letting the judge know about the mistake, in a nice way. He or she would probably really appreciate that, and if they really care, they will pay more attention so they don't make that mistake again. In the long run, it will make them better at what they do.

You can't correct a mistake unless you know you're making it.

 

[z987k]

If you really didn't care, you would not know that it made a difference in your placing. And, you should care about those details. IMO, there is no room for error if your judging something, no matter what it is. If you're judging something, you should be an expert in that field. At the very least you should know basic addition.

I know that's not the answer you're looking for, but I feel that kind of mistake is unacceptable.

Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.

 

[Jaybird]

If you placed, and that is the difference between silver and bronze. You bet your A$$ I would let them know. Some would ask why... But I always ask, Why not?

 

[Yankeehillbrewer]

Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.

no, but what does it matter?? Math is Math, the judge was wrong.

 

[carnevoodoo]

If you placed, and that is the difference between silver and bronze. You bet your A$$ I would let them know. Some would ask why... But I always ask, Why not?

I've never been in a comp where they go strictly by score to award places. They'll get the top 5 or so and they'll do a mini BOS and they'll pick that way.

 

[DanVader]

that is pretty goofy, I'd agree that the judges picked 1,2,3 and forgot to adjust the score (maybe?) I've gotten a bronze for a beer that scored 1 point under the minimum for gold, a silver would have been cool, but the 2 guys ahead of me did better.

out of the 3 judges, only one was BJCP, and he gave me a 42, that made me feel pretty good.

 

[Beerrific]

At a minimum you should let the competition director know. Every comp. I have been to (stewarded or judged) the director mentions several times: "double check your math" and "stewards, please double check the judge's math." I have miss-added on score sheets before, and the steward has caught it (hopefully every time). Sometimes you get to the end of a 13 beer flight simple math can easily escape, not that it was easy to begin with.

 

[big supper]

I would let the competition know, and not the judge.

I was a steward this year, and I double checked the scores for all of the flights that I was working with.

 

[FlyGuy]

That sort of thing is unfortunate, but it happens. Judges are humans, and if you have ever judged three sessions in a row (and 30 or so beer later), you can start to understand how it happens. That doesn't make it acceptable, but I hope people can try to be understanding that a judge might make the occasional math error.

Regardless, a small error like two points by one judge will rarely affect the order of how a beer places. Whenever the average scores by all judges in a flight are within a point or two, those beers typically get re-evaluated head-to-head. The scores themselves don't determine everything.

 

[Bokonon]

There definitely is an error somewhere but it might not be exactly where you think it is. One way to judge is to give points for each category and add them up another way is to decide the total score and then come up with category points to make that work.

So its possible that your score was what the judge intended but they didn't put the right numbers above to make it work.

 

[mmb]

That should have been caught by the stewards or contest director. I do contest judging and contest administration for an educational compilation (not beer) and the judge double checks, the proctors/administrators double check, and then there is a "grading room" where everything is check again.

We're talking scholarships and not ribbons as rewards, but the idea is the same.

 

[Sigafoos]

Yeah, that's not the judge's fault, it's the steward's. One of my sheets was off by a point I think, but ti was low enough where I doubt I'd have gotten a bronze or anything. Definitely let the competition coordinator know; even if you don't care, it's good for them to know to say 'hey stewards, count more carefully next year.' I'm glad you're taking it in stride (I can see people emailing them and saying WTF I DESERVE MOAR PRIZES!!!), but an FYI is always helpful.

 

[bob1852]

Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.

Yes, that's clearly what happened - the judge was 3 points higher than the other judge and brought it down to 37 rather than 39 (other judge had it at 36); but the numbers didn't add up.

Thinking about it more - it's a clerical error - the judges agreed that the beer deserved a 36 or 37 -- the score sheets just didn't reflect that.

I'm sure this stuff happens all the time - I haven't been to a contest or judged, but I'm sure it's pretty hectic trying to accurately rate beers and be as consistent among judges as possible.

I'll take my bronze and be happy

Thanks for everyone's comments

 

[taylornate]

another way is to decide the total score and then come up with category points to make that work.

So its possible that your score was what the judge intended but they didn't put the right numbers above to make it work.

Have you ever been to a competition?

AZ, have you seen how the judging usually happens? What were the other scores? Without knowing, it's possible that he was a touch higher than everyone else and agreed to bring it down a touch and forgot to adjust up top.

Are contests really run this way? It really does not seem ethical to me.

 

[cactusgarrett]

Seems like it should be, as evaluating a beer should be more of a qualitative process, as opposed to a quantitiative, in my opinion. I think it would be more wrong if you had two beers side-by-side and crowned one a winner when the judges think that overall the other tastes better, despite numerical values.

 

[Bokonon]

Are contests really run this way? It really does not seem ethical to me.

Even if you have no ambition to be a judge I suggest reading through:
http://www.bjcp.org/docs/How_to_Judge_Beer.pdf
and
http://www.bjcp.org/docs/Judge_Procedures_Manual.pdf

 

[Sigafoos]

Are contests really run this way? It really does not seem ethical to me.

It seemed weird to me too, but it actually makes a lot of sense.

Generally, when judges disagree a lot, there's something they haven't considered. One will say 'this is amazing' while the other will say 'this is horrible,' but then they each say why they like/dislike it and see the other person's side. They never agree exactly, but you can still be 7 points apart (which is a lot). If you think a beer is amazing but someone else thinks it's cow piss, chances are you're overlooking some flaw and they're putting too much weight on it and are missing some of its good points.

 

[bob1852]

maybe one of the judges was Russian

 

[FlyGuy]

Generally, when judges disagree a lot, there's something they haven't considered. One will say 'this is amazing' while the other will say 'this is horrible,' but then they each say why they like/dislike it and see the other person's side. They never agree exactly, but you can still be 7 points apart (which is a lot). If you think a beer is amazing but someone else thinks it's cow piss, chances are you're overlooking some flaw and they're putting too much weight on it and are missing some of its good points.

That's more or less the way it works. Before each judge in the flight finalizes their score for a particular beer, everyone checks their scores to see if they are all close to one another. Often they are not, it is a red flag that someone may have missed something. A quick discussion ensues among the judges about why the scored the way they did, and a quick re-evaluation takes place. In most cases, the judges will be honest and recognize if they missed something important and adjust their overall score, and then compensate their comments and component scores to match. The point of this is to be as fair as possible so that nothing is missed (good or bad).

Occasionally, however, judges will not agree because everyone has slightly different perception (that's expected). In those circumstances, judges try to get their scores only as close as they are comfortable (generally within 7 points, if they can), but they are not required to necessarily agree. Those can be quite interesting evaluations to get back because the judges obviously had strong opinions about their perceptions, enough that they weren't willing to back down on their scores. I pay close attention to these scoresheets because it is likely that they judges spent a lot of time evaluating that beer, relative to the others in the flight.

 

[z987k]

Seems like it should be, as evaluating a beer should be more of a qualitative process, as opposed to a quantitiative, in my opinion. I think it would be more wrong if you had two beers side-by-side and crowned one a winner when the judges think that overall the other tastes better, despite numerical values.

It really is both, and that can be one of the reasons a final score is changed and the numbers don't add up.