I would be interested if there are any actual studies on the reliability of beer tasting competitions. There's a fair amount of published research on wine testing and the bottom line is it's not reliable.
Like, at all.
One study analyzed judge performance at a major wine competition from 2005 to 2008 using replicate samples and found only about 10% of the judges were able to replicate their score within a single medal group. (Hodgson Journal of Wine Economics 2008)
In a study, of over 4000 wines entered in 13 competitions, the probability of winning a Gold medal at one competition was stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone. (Hodgson Journal of Wine Economics 2009)
I would not be surprised if beer tasting was just as unreliable.
this is a fair point. there are a few things in play here though. one is wine appreciation, for lack of a better term... this is the skill acquired necessary to be able to taste a wine and talk about it's qualities and characteristics specifically and accurately.... what grapes were used, where were they grown, what vintage, what producer in some cases, is it juicy or flinty? did it spend time in French oak or American oak? new or used barrels? and so forth. with training these are things that are factual that an experienced taster can be accurate and articulate about.
the other side is "rating" - is this a 100 or a 98? to me this is informed by some guidelines but is also very very influenced by individual experience and as such can be subjective. when you're eating a wine you're , to some degree, rating it against all similar wines you've ever had. if that library of similar wines is not massive, then the experience upon which you draw your comparison can influence the score. this, combined with the personal theory and belief system you will naturally impose on wines you taste based upon your experience, will also create differences between your scores and someone else's scores. you may believe in more oaky Cabernets and someone else may prefer intense fruit.... so your scores of the same wine may be affected by the differing beliefs.
this can certainly play a role in beer as well. a style guideline is merely a guideline, it is impossible to account for standardization of personal preference and it would be impossible to rate so,etching without considering personal preference to some degree.