the BJCP so called certification drives me crazy

[chewse]

It drives me crazy when I submit the same beer to two local competitions; each competition within 2 weeks of each other. And, in one competition my average score is 22 and comments say the beer is out of style. And in the other competition, I receive a score of 37 and get a red ribbon.

Those doing the judging all have BJCP so called certifications.

The same story repeats itself year-over-year for me; at least over the last 15 years.

Having a BJCP certification is sooooo meaningless.

 

[WBB]

And if you submitted those entries again 6 mos later and the same judges just so happen to judge again, you’d get different evaluations.

 

[twd000]

This is why I have never entered a competition. I can get random feedback from my buddies for free. Sometimes they even bring over a bag of pretzels.

 

[Jokester]

Yea me too, and I want to get more forum buddies who can give me more feedback LOL. I must be a sucker for punishment.
Cool.
Srinath.

 

[jekeane]

In general you wont get a true evaluation of a beer until it has been critiqued multiple times. And to be certified in the BJCP you don't need to do a ton. National and beyond is where you start to see the most informed opinions in my opinion. That being said:

Do you know how your entry was stored for each competition? Are you certain that your bottling practice was consistent for each bottle? Are you sure some contaminant didnt make it into a bottle? Where were you beers placed in the flight?

Judging is very subjective even with guidelines and you will see that the scale for what is "fair" ,"good" or "excellent" vary along the way. For instance the BJCP says that a "Good" beer is in the range of 21-29. If I give a beer a score in that range it has an obvious flaw or is mis entered. It would be a beer I likely couldn't have a second pint of. To me a 35 is a good beer that may not medal but is solid and 40+ are you medal worthy beers.

I am a certified judge that has the points to be national but can't be bothered to take the test. I competed heavily before going pro and still compete now when I can. I love it and it has made me a better brewer. I learned that you have to take everything with a grain of salt. You need to realize that when you got to second round it becomes completely subjective. Just this past week I got my highest score ever and one of my lowest. An Alt I entered was judged as having a sour and vegetal note. The beer was excellent toll it kicked in the keg, but something obviously went tits up in the packaging process. Maybe a fruit fly, or a dirty bottle, or maybe they mis sorted bottles I'll never know. It's part of the game and you either love it or hate it but I don't think the BJCP itself is flawed.

The competition circuit made me a significantly better brewer without a doubt.

 

[dmtaylor]

It drives me crazy when I submit the same beer to two local competitions; each competition within 2 weeks of each other. And, in one competition my average score is 22 and comments say the beer is out of style. And in the other competition, I receive a score of 37 and get a red ribbon.

Those doing the judging all have BJCP so called certifications.

The same story repeats itself year-over-year for me; at least over the last 15 years.

Having a BJCP certification is sooooo meaningless.

Yup.

There are some BJCP judges who are actually pretty damn good at what they do. But taking a random distribution... odds are poor. I've always estimated, and will continue to estimate, that about 40% of judges are pretty good, and 60% are pretty damn bad.

I don't think Nationals and Masters are necessarily any better than anyone else either. It's totally random. And even a good judge can have a bad day. It definitely happens.

 

[Qhrumphf]

In general you wont get a true evaluation of a beer until it has been critiqued multiple times. And to be certified in the BJCP you don't need to do a ton. National and beyond is where you start to see the most informed opinions in my opinion. That being said:

Do you know how your entry was stored for each competition? Are you certain that your bottling practice was consistent for each bottle? Are you sure some contaminant didnt make it into a bottle? Where were you beers placed in the flight?

Judging is very subjective even with guidelines and you will see that the scale for what is "fair" ,"good" or "excellent" vary along the way. For instance the BJCP says that a "Good" beer is in the range of 21-29. If I give a beer a score in that range it has an obvious flaw or is mis entered. It would be a beer I likely couldn't have a second pint of. To me a 35 is a good beer that may not medal but is solid and 40+ are you medal worthy beers.

I am a certified judge that has the points to be national but can't be bothered to take the test. I competed heavily before going pro and still compete now when I can. I love it and it has made me a better brewer. I learned that you have to take everything with a grain of salt. You need to realize that when you got to second round it becomes completely subjective. Just this past week I got my highest score ever and one of my lowest. An Alt I entered was judged as having a sour and vegetal note. The beer was excellent toll it kicked in the keg, but something obviously went tits up in the packaging process. Maybe a fruit fly, or a dirty bottle, or maybe they mis sorted bottles I'll never know. It's part of the game and you either love it or hate it but I don't think the BJCP itself is flawed.

The competition circuit made me a significantly better brewer without a doubt.

This. +100. All of it.

There are a lot of factors and definitely a lot of crummy judges. The bar to pass the exam for recognized is quite low (it's actually very hard to fail if any effort at all is put in), and with moderate preparation certified rank is easily obtainable (and I say that as an exam grader).

The bar to make National rank is *far* higher, and the bar to make Master is astronomical.

Of course I've seen recognized and certified judges write better, and more accurate sheets than some national judges (many of whom after doing it a long time just get lazy).

However, as said it's entirely possible that the judges were objectively accurate and there was something else going on. While a little variation is normal (which is why mini-BOS, and BOS itself, are things) a swing that wide could be bottle variation on your part, or contamination, or mishandling (on your or the comps part), or flight order, or flight length, or something about the judging setting.

However, the sheets from any level of BJCP judge to tend, on average, to be better than non-BJCP judges (except maybe Cicerones). I actually find non-BJCP pros to make the worst judges.

It's part of the game. And it applies to call comps, BJCP or otherwise. If you don't like it, don't enter.

 

[murphyslaw]

I'd encourage you to consider that the judges gave up a day of their weekend to try your beer and give their honest feedback and advice.

I've only judged a handful of competitions, and it's been a while, but in my limited experience I've always witnessed a group of people doing their best to give fair, consistent, and helpful feedback. If one person thinks something is off, the judges are supposed to talk about it and come to a consensus.


If your sending your beer into multiple subjective tasting competitions, run by amateurs and volunteers tasting 2 oz of 12 or more beers, with unknown storage procedures, and you expect consistent grades across competitions, I'd suggest that the error is in your expectations of what you get for that $5 entry fee.

I'd suggest that you look for common themes and comments. Or if there is no consistency, Lookat the feedback, drink the beer, and see if you can taste what the judge described. If so, perhaps you learn something. If not, oh well, its just beer.

 

[bracconiere]

who the hell cares about anyone's opinion besides their own? you want to sell beer? call it (insert catchy brand name) blue ribbon...lol

 

[Deadalus]

Yup.

There are some BJCP judges who are actually pretty damn good at what they do. But taking a random distribution... odds are poor. I've always estimated, and will continue to estimate, that about 40% of judges are pretty good, and 60% are pretty damn bad.

And the other 10% don't know how to add!

 

[Cameron Gray]

I'm also a certified BJCP judge and echo what has been said by everyone above.

At the end of the day, the honest feedback, of experienced beer drinkers and often brewers themselves can only help your brewing.

I have often called for second bottles of a beer that isn't quite right and you'd be amazed the difference you get between two bottles from the same brewer from the same batch. I've even experienced this with some commercial beers. Not just the explosively off gushing bottles. An infection that has just started won't immediately turn into a gusher.

Our judging always consists of multiple judges on a panel to try and remove the inconsistency of an individual palate but at the end of the day everyone perceives tastes differently and perception threshold vary from person to person for different chemicals. Have a read of some of the sensory training information to get an idea.

We also all provide our contact information and I am very open to being contacted with questions about my feedback.

 

[Unicorn_Platypus]

I entered a competition recently. Submitted the same beer in two categories.

I was actually pretty impressed by the consistency of the feedback from four different judges. Scored a 38 as a blond ale & 34 as Kolsch. No ribbon, but good scores so am happy.

 

[ba-brewer]

If you bottle from a keg, fill an extra one or two to sample when you get the results back. If you bottle more than one you can sample it close to the same time as the judging too. Some beers change in the bottle compared to when sampled from the keg.

I feel you need to dumb down your beers and brew to the main style points and not to your own preference. The more open styles are toughest as the judges get to choice what they think is the ideal beer, you will quite often see one judge pick one interpretation and another judge a different one.

 

[Jako]

brewing to a style is part of the fun for me. I agree sometimes you get a score you don't agree with. But lest be honest you are not going to make a sub 30 beer and expect to win. If you make a dam good beer it will score that way.

 

[madscientist451]

Iin one competition my average score is 22 and comments say the beer is out of style. And in the other competition, I receive a score of 37 and get a red ribbon.

There are just too many variables that you can't control and don't know about to blame the judges or the BJCP certification.
Did the judges go to a beer blast the night before and were hung over?
Did the judges sample 50 beers before yours?
Where was your beer in the flight?
What was the serving temperature?
Was a really exceptional version of the style served right before yours?
Having said all that, I agree that there are problems with beer competitions but I disagree that BJCP certifications are completely meaningless. At least the BJCP judges have met SOME minimal standard for qualification.
I don't think you'll ever be able to get personal likes/dislikes out of beer judging.
Your beer might not have any flaws, but the "overall impression" is something that is highly subjective and thus consistent scoring really can't be expected.
I believe the best you can get out of competitions is to take each judge's comments and scores individually and learn what you can from that.

 

[kingmatt]

If you enter enough competitions you are going to get results that disappoint/confuse you, it is the nature of the beast. I have had many beers judged where I felt my overall score or the feedback I received was wrong or not helpful. It is definitely frustrating but since becoming a judge I have learned to not take it personally.

Others in this thread have pointed out the many variables that go into how your beer may be perceived and judged that day so I won't belabor those points. To say 40% of certified judges are "bad" or that being certified is worthless is unfair, and frankly insulting, in my opinion...we are humans, not robots, doing our best to quantify something that is ultimately extremely subjective (how something tastes, smells, feels etc).

As a judge I do my best to provide an honest, accurate review of the beer I am judging based on the style guidelines the BJCP provides. I judge a lot of competitions to give back to the homebrew community and I don't get paid for doing it. I've given a lot of my free time (and money) to become certified so that I can do a better job at judging so that, hopefully, the advice or critique I provide helps whoever brewed it make a better beer.

If you are unhappy with the scores/feedback you receive at a particular completion, reach out to the competition coordinator to voice your concerns so that they can relay that feedback to the judges. Or stop entering that particular competition/all competitions in general and avoid the frustration. I'll keep judging beer for those that appreciate the feedback.

 

[Northern_Brewer]

As a judge I do my best to provide an honest, accurate review of the beer I am judging based on the style guidelines the BJCP provides. I judge a lot of competitions to give back to the homebrew community and I don't get paid for doing it. I've given a lot of my free time (and money) to become certified so that I can do a better job at judging so that, hopefully, the advice or critique I provide helps whoever brewed it make a better beer.

I'm sure most judges would say something similar. But the thing about judging to the guidelines is a key point - part of the frustration with the BJCP process lies with the guidelines rather than the people. For instance there's plenty of British commercial beers that don't meet the BJCP guidelines for their styles, so what's a judge to do when these beers have been brewed that way since long before the BJCP existed? Some will take the view that the guidelines are all that matters and fail the beer for not being to style, other judges will cut a bit of slack and try to judge the beer fairly even if it doesn't conform to the letter of the law.

There's good arguments for both approaches - I tend to the "cutting-slack" approach myself - but they can lead to dramatically different outcomes for beers that don't quite tick the style boxes.

 

[kingmatt]

I'm sure most judges would say something similar. But the thing about judging to the guidelines is a key point - part of the frustration with the BJCP process lies with the guidelines rather than the people. For instance there's plenty of British commercial beers that don't meet the BJCP guidelines for their styles, so what's a judge to do when these beers have been brewed that way since long before the BJCP existed? Some will take the view that the guidelines are all that matters and fail the beer for not being to style, other judges will cut a bit of slack and try to judge the beer fairly even if it doesn't conform to the letter of the law.

There's good arguments for both approaches - I tend to the "cutting-slack" approach myself - but they can lead to dramatically different outcomes for beers that don't quite tick the style boxes.

Judging to the BJCP style guidelines is the only way to ensure a fair competition. Otherwise, you are judging to personal taste which is WIDELY subjective. I personally don't like a ton of roasty notes in my stouts but I know that is acceptable for the style so I don't take points off when I judge one. If I were to just go by my personal taste I would not score a roasty stout as well which would not be fair to the entrant.

If you enter a BJCP competition you have to accept that fact that the beer will be judged based on the style guidelines. If you want a competition based on how tasty your beer is, regardless of whether it fits a classic style, you should enter a "people's choice" type of competition.

 

[applescrap]

I respect the work and importance of the bjcp judge. Without them, there is no competition so like them or not be grateful they are there and iiac volunteer a lot of their time and energy to support competition. They are human and can only go with what they have. Many issues have been stated already, i like the idea that maybe before yours an exceptional one came up for example. And as they go higher they learn and have to do more iirc. They become a professional so to speak at their craft. This doesnt make them right or even relevant but certainly gives them the tools to do the job that you need them to do, that they love doing.

All that said, dont forget that when brulosophy crunches all the data, across dozens of tests, bjcp get no more right than your average bud drinker. For a simple good or bad I trust my wife a non beer drinker. But for what was good or bad and how to fix, a bjcp judge would be better imo. Bottom line they are great people who do a good job. The competition thing just is what it is and for seasoned competitors I think they take it for what it is and understand that.

 

[Miraculix]

Judging to the BJCP style guidelines is the only way to ensure a fair competition. Otherwise, you are judging to personal taste which is WIDELY subjective. I personally don't like a ton of roasty notes in my stouts but I know that is acceptable for the style so I don't take points off when I judge one. If I were to just go by my personal taste I would not score a roasty stout as well which would not be fair to the entrant.

If you enter a BJCP competition you have to accept that fact that the beer will be judged based on the style guidelines. If you want a competition based on how tasty your beer is, regardless of whether it fits a classic style, you should enter a "people's choice" type of competition.

I think a competition based on personal taste would be the only fair competition there is, as everybody would know that it's based on personal bias/taste. Not like the bjcp events which claim to be not to be based on personal bias but as evidence shows, they are clearly far away from being non-biased.

 

[kingmatt]

I think a competition based on personal taste would be the only fair competition there is, as everybody would know that it's based on personal bias/taste. Not like the bjcp events which claim to be not to be based on personal bias but as evidence shows, they are clearly far away from being non-biased.

How would it be fair if you entered an excellent smoked lager into the competition and I gave you 15 out of 50 because I don't really like smoked lagers?

 

[Miraculix]

How would it be fair if you entered an excellent smoked lager into the competition and I gave you 15 out of 50 because I don't really like smoked lagers?

It would be fair because in this case everybody would know that there is no straight way of judgement, but it is only based on personal preferences and I would know that when entering. In other words, nobody would claim to follow a certain set of rules, and I would know this from the start when entering.

However, it is unfair to claim that the judgement is strictly based on given rules and then deliver such a vast range of judgements for the same beer that can only be explained with strong personal bias.

Therefore, a personal taste based competition would be fairer than bjcp competitions.

 

[kingmatt]

[

It would be fair because in this case everybody would know that there is no straight way of judgement, but it is only based on personal preferences and I would know that when entering. In other words, nobody would claim to follow a certain set of rules, and I would know this from the start when entering.

However, it is unfair to claim that the judgement is strictly based on given rules and then deliver such a vast range of judgements for the same beer that can only be explained with strong personal bias.

Therefore, a personal taste based competition would be fairer than bjcp competitions.

Agree to disagree I guess. Everyone judging a BCJP competition, especially certified judges and higher, understand that they are expected to judge the beer based on how well it fits the style guidelines, not by how much they like the beer. Some judges are admittedly better than others at ignoring personal bias, and some people are by nature better than others at picking up off flavors or subtle nuances in a beer (I know many people that flat out can't detect diacetyl).

I have given very good scores to beers I wouldn't like to drink outside of judging because they were good representations of the style. I have also had to give poor scores to beers that I actually enjoyed but didn't fit the style guidelines. Maybe people don't really understand how the competition works before they enter their beer but that is not the competition or the judges fault.

I fail to see how a competition where beers are randomly judged based on what the judge assigned likes and doesn't like would be beneficial to anyone.

 

[Hoppy2bmerry]

It drives me crazy when I submit the same beer to two local competitions; each competition within 2 weeks of each other. And, in one competition my average score is 22 and comments say the beer is out of style. And in the other competition, I receive a score of 37 and get a red ribbon.
The same story repeats itself year-over-year for me; at least over the last 15 years.

What is your motivation to enter? Winning, getting feedback, is it validation?

I’ve had a first place beer score in the 20s it another competition and in reading the feedback it seemed apparent that one judge in particular wasn’t at all familiar with the style. The same beer scored in the 30s in another comp and took first place. It it possible I sent a “bad” bottle? Perhaps but no where in the comments was the dreaded “watch your sanitation” remark.

If you are going to compete, then it is best to realize that the judges are human, have their own preferences, and make mistakes. Also that perhaps even a great beer presented in the wrong order won’t shine. Mostly don’t get butt hurt over a low score, and don’t take it personally.

I’m not a judge and I know I really don’t have the nose and palette for it, but I do steward and many times I’ve tasted with the judges (keeping my thoughts to myself of course) I know, and they know when a great beer or the best beer in a flight is evaluated. Judges give the same attention tasting and sniffing the not-so-good samples as they do the good ones, and for that alone they should be commended. If several are excellent and score high the mini BOS weeds out the lesser beer, in that case a generously scored beer may not place.

 

[Miraculix]

[


Agree to disagree I guess. Everyone judging a BCJP competition, especially certified judges and higher, understand that they are expected to judge the beer based on how well it fits the style guidelines, not by how much they like the beer. Some judges are admittedly better than others at ignoring personal bias, and some people are by nature better than others at picking up off flavors or subtle nuances in a beer (I know many people that flat out can't detect diacetyl).

I have given very good scores to beers I wouldn't like to drink outside of judging because they were good representations of the style. I have also had to give poor scores to beers that I actually enjoyed but didn't fit the style guidelines. Maybe people don't really understand how the competition works before they enter their beer but that is not the competition or the judges fault.

I fail to see how a competition where beers are randomly judged based on what the judge assigned likes and doesn't like would be beneficial to anyone.

I get that, but based on the feedback I see here and elsewhere, people get such a wide range of feedback for the very same beer, that the reality seems to be that bjcp competitions just are as biased as a personal preference based competition.

And in addition, bjcp tries to define styles from other countries and... Come on... Why should an American, living in the USA define a style for a Belgian beer? That's just wrong. And it often is literally wrong as the style is described more based on personal imagination than on reality in the respective country.

So overall, nope.

 

[goodolarchie]

I don't have enough knowledgeable friends around me who I can really trust to give good feedback, they all like my beer "mmm, this is good, man!" So I rely on BJCPs in competitions to pick up off flavors and such.

I'd be pretty miffed if I got a 22 and a 38 (never happened yet, most beers score within 2-4 points of one another in my experience)... but I would be REALLY interested in what they both had to say. Also taking and parsing feedback is one of the hardest skills to build, professionally or in your hobbies, but the quicker you learn how to listen, empathize and then decide what's signal and what's noise... the better you'll do in situations where you're "putting yourself out there" - whether that's beer, art, a performance, anything.

 

[kingmatt]

I get that, but based on the feedback I see here and elsewhere, people get such a wide range of feedback for the very same beer, that the reality seems to be that bjcp competitions just are as biased as a personal preference based competition.

And in addition, bjcp tries to define styles from other countries and... Come on... Why should an American, living in the USA define a style for a Belgian beer? That's just wrong. And it often is literally wrong as the style is described more based on personal imagination than on reality in the respective country.

So overall, nope.

We try to do the best job we can at evaluating the beer in an impartial way and I think (or at least hope) the feedback we provide as judges can be helpful, but these type of competitions aren't for everyone. You seem to understand and accept that at least.

 

[Hoppy2bmerry]

And in addition, bjcp tries to define styles from other countries and... Come on... Why should an American, living in the USA define a style for a Belgian beer? That's just wrong. And it often is literally wrong as the style is described more based on personal imagination than on reality in the respective country.

So overall, nope.

I don’t think they are defining the style at all they describe guidelines based on actual Belgian beer. On German beers etc.

 

[Miraculix]

I don’t think they are defining the style at all they describe guidelines based on actual Belgian beer. On German beers etc.

That's what they claim to do, but as they live far away, the imagination seems sometimes to play a bigger role than the actual brewing habits of the particular country.

 

[tmendick]

So you think that one guy wrote all of the style guidelines and didn't consult anyone else?

Was the feedback significantly different between the two results? I've judged with people that say "good beer" is 30-39 and others that say "good beer" is 20-29. There's a national judge I know that said he can count the number of 40+ beers he's scored on on hand.

Some regions tend to have different palates, I'd have to look it up, but I know someone said something like "midwest judges tend to expect more malt and south judges tend to expect more dry finishes".

As asked above, what do you want from the competition?