"Style bias" on ratebeer.com and elsewhere

Homebrew Talk - Beer, Wine, Mead, & Cider Brewing Discussion Forum

Help Support Homebrew Talk - Beer, Wine, Mead, & Cider Brewing Discussion Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.
Casey27

Casey27

Well-Known Member
Joined
Feb 20, 2008
Messages
83
Reaction score
0
Location
WA
OK, this has been bugging me for a while.
First let me say that ratebeer.com and other sites like BA are a great resource, and I do value them.

But it bothers me that some beer styles are always rated higher than others - this implies that the people who supply the ratings can't judge a beer against it's style, rather, they are playing favorites, giving their preferred styles boosted ratings. This is a problem because it implies that the average reviewer (certainly not every reviewer though) cannot judge a beer objectively. If this is the case, what good are the ratings?

For example, IPAs are on average rated higher than milds. This is even more exaggerated in some styles, such as double IPAs, that seem to get extra points simply for being in that style. To see what I mean just compare the overall percentage ranking to the style percentage ranking. (Of course the style ranking is a much more meaningful number.)

I mean, everyone is using the same 50-point scale for each beer, right? Does everyone think that higher gravity beers automatically get more points?
 
That annoys the hell out of me too. I love how half of the top rated beers in the world are Imperial stouts.
 
I've noticed this too. 9 of the 10 "Best of BeerAdvocate" are either Imperial Stouts or Imperial IPAs. I think it may just be that beer geeks tend to gravitate towards big beers, and if you're writing reviews for BA, you're probably at the top of the list of beer geeks.
 
Oh Rly.

You want some excitement?

Rate a BMC according to style. Be honest and accurate TO STYLE. Put all your preferences to taste aside.

And see how quickly your rating gets pulled from RateBeer because it isn't inline with "Moral" majority.
 
There are more hophead fanatics willing to do a review, that's all. If you look at the top 50 it's mostly Imperials. It's kind of like going to a stereo or camera specialty site. There is a bias and the "common" brands/models get rated down. It's elitism - so long as you know that going in it shouldn't bother you. Read for what's useful to you - ignore the rest.

-OCD
 
I think I understand.
A great oatmeal stout on ratebeer = "I love this beer!!!"
It does not = "an outstanding oatmeal stout"

However that statistics used on ratebeer.com are excellent...but that is a separate issue from the reviews themselves (i.e., the data).
 
I think pliny the elder and pliny the younger are great examples of this! I think pliny the elder is such a better beer then pliny the younger! Pliny the younger just happens to be rare and a quadruple IPA sooo...you know...ITS BETTER! Its retarded.
 
Casey27 said:
But it bothers me that some beer styles are always rated higher than others - this implies that the people who supply the ratings can't judge a beer against it's style, rather, they are playing favorites, giving their preferred styles boosted ratings. This is a problem because it implies that the average reviewer (certainly not every reviewer though) cannot judge a beer objectively. If this is the case, what good are the ratings?
Click to expand...

These sites absolutely should not be considered to be sites for rating beers against a style guideline. As far as I know, neither site even offers style guidelines to form the basis for such a rating system.

Imagine a home brewing competition where there is a "people's choice" award. Rate Beer and Beer Advocate are analogous to the people's choice portion, not the formal judging. If they are trying to be like the formal judging portion they are failing spectacularly. As far as I know there is no good source of qualified judging against formal style guidelines outside of amateur and professional competitions (and the pages of Zymurgy). Sucks, but thats the way it is.

Another peeve I have with these sites is that they are bad at assigning beers to the best style category.

Take Pelican Pub's Kiwanda Cream Ale as an example. It is listed by BA as a cream ale, but it is a poor example of the cream ale style and materially different from other cream ales. It does happen to be a good example of a blond ale.

I use beer advocate to find out about new beers and events in my area and to sometimes read qualitative feedback on a beer I have not had and am interested in. IMO, the quality of the style assignments and numerical ratings is so poor as to be useless or even misleading.
 
Style and taste are all subjective to the person sampling anyway...In all reality unless someone tells you a beer tastes horrid you would likely only trust your own opinion, no? I assume most people on this site would taste almost any beer anyway just to see for themselves.
 
sdswingr said:
OK my first post..But I must say I can't stand the taste of most IPAs...and really hoppy beer makes me want to cry
Click to expand...

My GF was the same way until I brewed a moderately hoppy, clean citrusy IPA this summer. She loved the tangerine and orange(Cascade and Amarillo) flavors and aromas. It changed the way she felt about hops and she understood the contribution to flavor and aroma better.

Now she's ordering Great Lakes Nosferatu at bars and stuff.

I used to not get overly hoppy beers either. Now I freakin love them. You will get there too I bet.
 
The other issue with 'community rating systems' is that people who rate those beers don't necessarily have much experience with beers in other categories which further enhances their bias.
 
There'd still need to be some sort of method for moderating the judgements to eliminate bias.

Of course, that might self regulate to some extent. A practicing judge who publicly exhibits a bias or incompetence might find himself unwelcome as a judge at events very rapidly.
 
There are a lot of hopheads out there and I cringe when someone who thinks they are into craft beer say they only drink IPAs. They tend to be very vocal and focused on one aspect of the beer. If it strips the plaque off their teeth, it's a 5.0. Hence the ratings.

Don't expect online rating to reflect the style guide. And if you do use them as a reference, only compare reviews within a style.
 
From the RateBeer FAQ:
Q. Should I rate a beer to style as a beer judge would?
A. In short, no. While RateBeer encourages its members to learn as much as they can about beer and beer styles, RateBeer uses a Hedonic Scale to judge a beer according to how much it pleases the nose, eyes and tongue.

Rating to style is what is done at the Great American Beer Festival, many homebrew competitions and other beer competitions around the world. We're glad these contests exist and that they do rate to style, however we feel the same way about dog shows.

Dog shows are nice for those people interested in defining the best possible pinscher, poodle or Afghan. This is a great use of rating to style. If the dog doesn't look exactly like a certain breed should then it's score is dropped and the dog isn't given an award.

Now that's all fine and dandy, but we think our hedonic scale is more interesting for both everyday consumers and beer lovers in particular. We're not interested in poodles that look exactly like poodles. We're interested in the smartest dog, the highest jumping dog, the meanest dog, the biggest dog, the fastest dog, the strangest dog, the most amusing dog. We don't care what the hair looks like or what color it is -- we want dogs that thrill us.

This is the way we like our beer. We don't want it to conform to a mold, we want it to challenge us, intrigue us, surprise us, thrill us, dominate us, introduce us to new ideas... Simply put, we recognize great beer regardless of definitions. So please, learn about classic styles, learn about beer history, learn about the great brewers but when you rate put style aside and tell us how GOOD the beer is.
​
 
Keep in mind too that in anything, whatever's in fashion becomes "right." Stouts and IIPAs are what's cool right now, so anybody who's following the logic that appreciating beer means liking those beers are going to rate them highly. I wouldn't let it get to you, there are a lot of morons in the world.
 
I think that the reason you see so many Imperials rated is because people tend to spend more money on a bomber of an 11% Imperial, so it is a bigger deal to them, so they want to review it.

I have reviewed several beers and I usually review the beers that make a lasting impression on me. It's also much easier for me to review a beer when I can pick out strong flavors and aromas.

Also my tastes have changed since I started drinking real beer. First, I had a taste for belgians, then I went for malt monsters (i couldn't stand hops at that point) and stouts/porters. Then after Arrogant Bastard, I found myself at odd times craving the taste of hops. It literally crept up on me. Now I have a taste for extremely hoppy AND high ABV beers like DIPAs, Triple IPAs and Barleywines.

I have a feeling that at some point I will settle back into more subtle styles like milds, browns and bitters. Or maybe they'll just have to pry my Imperial IPA from my cold, dead hands.
 
DavidP said:
From the RateBeer FAQ:
Q. Should I rate a beer to style as a beer judge would?
A. In short, no. While RateBeer encourages its members to learn as much as they can about beer and beer styles, RateBeer uses a Hedonic Scale to judge a beer according to how much it pleases the nose, eyes and tongue.

Rating to style is what is done at the Great American Beer Festival, many homebrew competitions and other beer competitions around the world. We're glad these contests exist and that they do rate to style, however we feel the same way about dog shows.

Dog shows are nice for those people interested in defining the best possible pinscher, poodle or Afghan. This is a great use of rating to style. If the dog doesn't look exactly like a certain breed should then it's score is dropped and the dog isn't given an award.

Now that's all fine and dandy, but we think our hedonic scale is more interesting for both everyday consumers and beer lovers in particular. We're not interested in poodles that look exactly like poodles. We're interested in the smartest dog, the highest jumping dog, the meanest dog, the biggest dog, the fastest dog, the strangest dog, the most amusing dog. We don't care what the hair looks like or what color it is -- we want dogs that thrill us.

This is the way we like our beer. We don't want it to conform to a mold, we want it to challenge us, intrigue us, surprise us, thrill us, dominate us, introduce us to new ideas... Simply put, we recognize great beer regardless of definitions. So please, learn about classic styles, learn about beer history, learn about the great brewers but when you rate put style aside and tell us how GOOD the beer is.
​
Click to expand...

Ha! Now I actually do remember reading this a while back!

Still, most people who casually use the site (like me; and I am not an expert taster or a judge by the way!) probably just look at the big percentage number that accompanies each beer.

I guess my point in starting this thread was that I just wanted to stick up for the little guy. (The little guy being all those neglected styles like cream ales and milds that always have low ratings!)
 
Basically Rate Beer's FAQ is saying that they don't care that it is a popularity contest but please learn something about beer.


*shrug* If all we are discussing is Rate Beer's thing then they are doing exactly what they said they'd do.

However, if we were to turn this discussion into a discussion as to how it could be done better, perhaps something good could come of that.
 
I think a site where there were reasonably well defined styles and the beers were properly placed in those styles (and in a "specialty" type style if they don't fit elsewhere) would be a start. Don't need the inertia against new styles that BJCP has, as soon as there are like 10 beers that are similar make a new style.

Then I would adjust the rankings so that the distribution of rankings in each style were forced to be similar within a tolerance.

Absurdly simple example:

Raw Rankings
American Light Lager
Bud Light - 2
Coors Light - 1
Sam Adams Light - 3

Russian Imperial Stout
Dark Lord - 5
Darkness - 4.9
KBS - 4.8

Adjusted Rankings
American Light Lager
Sam Adams Light - 4
Bud Light - 3
Coors Light - 2

Russian Imperial Stout
Dark Lord - 4
Darkness - 3
KBS - 2
 
It just goes to show that most beer snobs don't appreciate beers that don't slap them in the face with hops or alcohol. Brew snobs tend to have a higher appreciation for the less in your face styles.
 
I think that you are all looking way too deep into this. I could care less how "in style" a beer is. I could really also care less if the crowd likes the beer. I would rather see tasting notes on a beer instead of a rating. This way I could easier identify beers that I may enjoy.
 
Casey27 said:
OK, this has been bugging me for a while.
First let me say that ratebeer.com and other sites like BA are a great resource, and I do value them.

But it bothers me that some beer styles are always rated higher than others - this implies that the people who supply the ratings can't judge a beer against it's style, rather, they are playing favorites, giving their preferred styles boosted ratings. This is a problem because it implies that the average reviewer (certainly not every reviewer though) cannot judge a beer objectively. If this is the case, what good are the ratings?

For example, IPAs are on average rated higher than milds. This is even more exaggerated in some styles, such as double IPAs, that seem to get extra points simply for being in that style. To see what I mean just compare the overall percentage ranking to the style percentage ranking. (Of course the style ranking is a much more meaningful number.)

I mean, everyone is using the same 50-point scale for each beer, right? Does everyone think that higher gravity beers automatically get more points?
Click to expand...


I noticed the exact same thing.... kudos for the post.

I don't like IPAs at all... I did find that Belgians are highly ranked as well though, and I like Belgians :D
 
passedpawn said:
? .
Click to expand...

I think the guy is trying to pin beer snob vs. brewer and I don't really see it. What I see on RB and BA is the same thing I see on Yelp and any other social site. The hype machine will take over and the people will flock to the most popular and/or rare items. It seems to be the nature of the type of people who write reviews on the internet. They do it for recognition as much as anything else, and a lot of them want acceptance by looking cool to their peers.

I'm a total beer snob, but I'll take a good pint of Nautical Nut Brown over just about anything crazy and extreme. I know the difference between good and hype.
 
Casey27 said:
OK, this has been bugging me for a while.
First let me say that ratebeer.com and other sites like BA are a great resource, and I do value them.

But it bothers me that some beer styles are always rated higher than others - this implies that the people who supply the ratings can't judge a beer against it's style, rather, they are playing favorites, giving their preferred styles boosted ratings. This is a problem because it implies that the average reviewer (certainly not every reviewer though) cannot judge a beer objectively. If this is the case, what good are the ratings?

For example, IPAs are on average rated higher than milds. This is even more exaggerated in some styles, such as double IPAs, that seem to get extra points simply for being in that style. To see what I mean just compare the overall percentage ranking to the style percentage ranking. (Of course the style ranking is a much more meaningful number.)

I mean, everyone is using the same 50-point scale for each beer, right? Does everyone think that higher gravity beers automatically get more points?
Click to expand...


One day, when I have no dumb projects left to do, I'm going to pull all the beers and ratings from them, and find out what the correlations are for style, gravity, country of origin, etc, to score.
 
JKoravos said:
It just goes to show that most beer snobs don't appreciate beers that don't slap them in the face with hops or alcohol. Brew snobs tend to have a higher appreciation for the less in your face styles.
Click to expand...


People on rate beer are exactly that: beer snobs. They're not beer enthusiasts, which I think most people here are.
 
Jdecarol said:
Don't forget people voting higher based on the rarity of said beer.
Click to expand...

To be fair, there is a euphoria factor at play when you're drinking something that's very hard to get that makes it tough to be unbiased. Very often you've traveled farther, worked harder to obtain, or spent more money to taste that beer. I'm sure if everyone did blind tastings their opinions on the top-ranked beers may change, but that's not going to happen anytime soon outside of competitions.
 
i use BA to get a feel for new beers I randomly buy. I'm a noob brewer but i've always loved beer. Those reviews are mostly uselees. They make me laugh. so often there will be 25 flavored mentioned, some coming late, lingering blah blah blah, yeah right dude, you have a magic palate that can detect 35 flavors at once. mostly that place is for beer snobs who like to feel important by reviewing, the reviews are even more pointless because they already have a biased before the even take a sip(they know what they are drinking, brand, hype, rare, price all that sshhh). sorry rant over;)
 

Similar threads

VikeMan
ABV Bias in Beer Ratings
Replies
19
Views
2K
bike2brew
bike2brew
julia_1
  • Poll
1/24-1/31 - Time to Enter - National Homebrew Competition Entry Window
Replies
7
Views
1K
julia_1
julia_1
S
Greetings from mid-IL
Replies
4
Views
485
Stormcrow
Stormcrow
B
Bottling CO² Vol limits theory
Replies
9
Views
3K
MaxStout
MaxStout
F
Entering a Homebrew Completion? Tips?
Replies
30
Views
2K
fendersrule
F
Yakima 300

Latest posts

Back
Top