Taking the BJCP Tasting Exam

It’s been over a month since my last post, perhaps my longest hiatus since starting this blog nearly four years ago.  In the meantime, I’ve been to Colorado for a workshop, Idaho for vacation; South Korea for a neutron scattering conference; and upon returning home spent several evenings in my yard pulling the weeds that grow prodigiously in the hot, wet Ohio summer when left to their own devices.

When I checked in last month I had just signed up to sit the BJCP tasting exam (officially the BJCP judging exam) in late July.  The exam took place this past Saturday, and I thought some of you might be interested in an update.  What follows is a detailed account of the exam and my preparations for it, probably too detailed if truth be told.  Hopefully some of my ramblings will be of use to others who are preparing for this challenging exam.

Two-A-Days

Some folks take eight week courses to prepare for this exam, some read books like Randy Mosher’s Tasting Beer or Jeff Alworth’s Beer Bible, others seek out undocumented Czech immigrants to school them on the finer points of Bohemian floor malted pilsner and Saaz hops.  I eschewed all of these approaches in favor of my tried and true method, drink beer and write down what I’m experiencing. My travels tended to interrupt the training, trust me there is a limited amount you can learn about beer while visiting Korea, but I did my best to review one, two, occasionally three beers per day.  It’s tough work to be sure, but there’s no substitute for practice.

The list of styles I reviewed, in order of descending score, are given below (remember the BJCP uses a 50 point scale):

• Category 26A – Trappist Single (Spencer Trappist Ale): 43 points
• Category 23A – Berliner Weiss (Jackie O’s Berliner Weiss): 43 points
• Category 4A – Munich Helles (Paulaner Original Munich Lager): 41 points
• Category 3B – Czech Premium Pale Lager (Pilsner Urquell): 41 points
• Category 7A – Vienna Lager (Devil’s Backbone Vienna Lager): 41 points
• Category 24A – Witbier (La Trappe Witte Trappist): 41 points
• Category 9A – Doppelbock (Ayinger Celebrator Doppelbock): 40 points
• Category 21B – Specialty IPA (Commonhouse Summer Sesh): 40 points
• Category 20C – Imperial Stout (Founder’s Imperial Stout): 40 points
• Category 5D – German Pils (Sierra Nevada German Pilsner): 38 points
• Category 25B – Saison (Yellow Raven, my homebrew): 38 points
• Category 5C – German Helles Exportbier (Great Lakes Dortmunder Gold): 37 points
• Category 19C – American Brown Ale (Telluride Face Down Brown): 37 points
• Category 17B – Old Ale (Bell’s Third Coast Old Ale): 36 points
• Category 6B – Rauchbier (Aecht Schlenkerla Rauchbier): 35 points
• Category 20A – American Porter (Great Lakes Edmund Fitzgerald): 35 points
• Category 5B – Kölsch (Gaffel Kölsch): 34 points
• Category 4C – Helles Bock (Rogue Dead Guy Ale): 34 points
• Category 27 – Gose (Fate Brewing Gose): 33 points
• Category 7B – Altbier (Fort Collins Brewing Red Banshe): 33 points
• Category 19B – California Common (Anchor Steam): 33 points
• Category 1A – American Light Lager (Actual Photon): 30 points
• Category 23B – Flander’s Red (Destihl Flander’s Red): 29 points
• Category 5B – Kölsch (Drifting Sun, my homebrew): 28 points
• Category 21A – American IPA (Fly to the Rainbow, my homebrew): 27 points

This approach gives you a pretty decent notion of the characteristics that each style should embody, which is an essential knowledge for the exam.  It’s less helpful in calibrating you to the accepted judging scale.  For that you really need to rate beers in the company of seasoned judges.  After the exam, I found an old online BJCP study course where among other things Grand Master judge Gordon Strong rates various commercial beers.  He scores a few of the same beers that I did, giving 46 and 43 points to Pilsner Urquell and Great Lakes Dortmunder Gold, respectively.  A glance at the above list shows that I scored the same beers 5-6 points lower.  In retrospect I can see that my scores on those beers are low, after all if a fresh sample of Pilsner Urquell isn’t a world class example (45-50 points) of a Czech Premium Lager then what would be?  You have to remember that a score in the mid 40s doesn’t mean that you personally think this is one of the best beers you’ve ever tasted, it simply means that the beer exemplifies what the style guidelines say it should be. I’m pretty sure that Gordon would have sighed in resignation if he could see my scores of 33 for Anchor Steam or 35 for Aecht Schlenkerla Rauchbier, both prototypes of their respective styles.

That brings us to a philosophical point, can you have a world class example of a beer style that is boring, even personally off putting?  After all the examples given in the BJCP guidelines for American Lager (Category 1B) are the usual suspects (Budweiser, Miller High Life, Coors, etc.) and I just can’t imagine giving any of those beers a score in the 40s.  Well if there is any doubt about the error of my thinking just listen to this audio review where Gordon gushes (in his restrained deadpan way) about a PBR, awarding 45 points.

To augment my drinking I poured over the style guidelines multiple times.  There is a very useful BJCP app you can get for your smart phone, very handy for transoceanic flights as well as quiet time in the bathroom.  I also wrote out the basic facts (SRM, ABV, IBU) and short 2-3 sentence descriptions of most styles to try and imprint the key components of each style in my head.

One very useful tip I received for reviewing beers.  Make sure you use the BJCP exam scoresheets and set a timer for yourself when you practice.  During the exam, you have 90 minutes to evaluate 6 beers, that’s only 15 minutes per beer.  If that sounds like an ample amount of time, it’s not.  I was advised to practice for the exam by setting a time limit of 12 minutes per beer.  More often than not I couldn’t finish in 12 minutes, but I usually was able to finish by the 15 minute mark.  That was crucial, because during the exam I needed every one of those 90 minutes to finish.

Drinking one exemplary commercial beer after another is not so helpful when it comes to recognizing off flavors or providing constructive feedback to the brewer.  I mean how realistic is it for me to give advice to the brewers at Paulaner on how to improve their Munich Helles.  I did spend one evening tasting beers that were dosed by my friend Hans with drops of imitation butter (diacetyl), banana extract (esters), and white vinegar (acetic sour).  That was pretty useful, ideally I would have done more of that.  You can get some tips on how to simulate off flavor tasting using ingredients solid in the grocery store by clicking here.  For giving constructive feedback there’s really no substitute for drinking a variety of homebrewed examples, ideally with a knowledgeable judge.

Exam Structure

The exam is organized by an exam administrator.  He chooses the date, the venue, and the beers that will be evaluated.  With a little help he distributes the beers and scoresheets during the exam, and collects everything afterward.  The exam I attended was organized by Tom Morgan, a member of the Dayton Homebrew Club DRAFT as well as an English professor at the University of Dayton.  It was held at the Yellow Springs Brewery at 10 am on a hot, humid Saturday morning.  If I counted correctly there were 6 of us taking the exam (despite what I said last time about the challenges of getting a spot on an exam there were empty seats at this one).  There were also 3 proctors, experienced judges who evaluate the same beers as we do.  Their scoresheets basically serve as the key for grading the exams.  After the exam is over someone who was not at the exam, probably not even from the same state, will grade the exams by comparing the examinees descriptions and scores of the beers with those of the proctors. I’m told it will take approximately 14 weeks before the exams are graded and the results released.

The names of the proctors were not announced, but one of the proctors was none other than Gordon Strong, a grand master judge and three-time winner of the national homebrew competition.  I knew he was from Dayton, so it wasn’t a total surprise when I found out he was proctoring.  Maybe I should have read one of his brewing books prior to the exam, or at least listened to and read more of his reviews on the internet.

Grading

For anyone who didn’t commit to memory my last post on this topic let me remind you that the exam is graded on five equally weighted aspects: scoring accuracy, perception, descriptive ability, feedback, and completeness.  The BJCP exam scoring guide (worth a read for anyone who is preparing for the exam) spells out the outcomes of the exam based on your performance.

In scoring a test, the scorer should be comfortable that the examinee has demonstrated skills that relate to the judge level for which the score qualifies, using the following scale:

< 60: The examinee displays weak tasting skills, and the scoresheets will generally have unacceptably low levels of completeness, descriptive information and/or feedback. This examinee will be an Apprentice judge.

60-69: The judging exam demonstrates the minimum acceptable communication and judging skills expected of a Recognized judge.

70-79: At least three of the six exam beers are accurately evaluated. The scoresheets should have reasonably good completeness, descriptive information, and feedback, appropriate to the Certified judging level.

80-89: At least four of the six exam beers are accurately evaluated with the high quality scoresheets expected of a National judge.

≥ 90: it should be obvious that the examinee is an experienced beer taster. At least five of the six exam beers are accurately evaluated and the scoresheets have Master levels of completeness, descriptive information, and feedback.

Scoring accuracy is the most straightforward to grade, your overall score is compared to the consensus score of the proctors.  You get full marks if the two scores are the same. In the highly probable scenario that the two are different, you are deducted 5% for every point that the two scores deviate from each other.  For example, if you were to give a score of 31 and the proctor’s consensus score was 27, a difference of 4 points, you would receive 80% of the scoring accuracy points on that beer.  As mentioned above, to attain certain ranks on the exam you must accurately evaluate a specific number of the beers. A necessary but not sufficient criterion for accurately judging a beer is to be within 7 points of the proctor’s consensus score.

The instructions given to graders on the other four areas of the exam, as taken from the BJCP exam scoring guide, are as follows:

Perception: Points should be deducted for missed flaws and errors in aroma, appearance, flavor, and mouthfeel perception. The rubric formed by the proctors’ scoresheets enables the graders to make a correlation between the characteristics identified by the examinees and those noted by the proctors.

Descriptive Ability: A beer judge should be able to describe the intensity and characteristics of the aroma, appearance, flavor, and mouthfeel using the proper terminology. The BJCP Style Guidelines serve as somewhat of an answer key for this aspect of the scoresheet.

Feedback: The brewer should receive useful and constructive feedback explaining how to adjust the recipe or brewing procedure in order to produce a beer that is closer to style. The comments should be constructive and consistent with the characteristics perceived by the examinee as well as with the score assigned to the beer.

Completeness/Communication: A complete scoresheet should have well-organized, legible and have informative comments that fill all of the available space. The checkboxes for stylistic accuracy, technical merit and intangibles should also be marked. This aspect of the scoresheet is generally consistent with the level of descriptive information and feedback conveyed by the examinee.

The Exam

As soon as the exam started the room became so quiet that a monk would have felt at home.  The beers were poured out of a pitcher into tasting glasses that were probably 4-6 oz in size, roughly the kind you get with a flight of beer, and then distributed to the examinees and proctors.  This was followed by a room full of mostly middle aged folks evaluating those beers as if they were auditioning for a role in CSI, feverishly writing as fast as they could on their scoresheets. I find it a little tricky to evaluate the head when the beer is served from a pitcher, not to mention the fact that some of the aroma dissipates before it gets to you, but everyone was working with the same constraints. You could ask for more beer if you wanted, but to be honest there was so much writing to do that there wasn’t really time to deeply ponder each beer the way I normally would.  Every 15 minutes a new beer was brought out.  Without fail I was still finishing up my review of the last beer when the new one arrived.

Here’s a blow by blow account of the exam.  For anyone not contemplating the exam the entertainment value of this account is questionable, probably right up there with reading in flight magazines on an airplane, or sitting through 90 minutes of C-span.

Beer 1 – American Lager (Category 1B)

While I was preparing for the exam there were many times when I had a chance to sit down and judge a macro lager.  Hell, just about every beer I encountered while in Korea would have fallen under this category, but time after time I convinced myself that the chances were slim that a macro lager would be one of the beers selected for the exam.  Imagine my surprise when the first beer turned out to be nothing other than Category 1B: American Lager.  I later found out we were drinking a local craft brewed take on the American Lager style, but still a beer in the same category with Budweiser and Miller High Life.  I let my style bias creep into the judging and was pretty hard on this beer.  I could swear there was a corny adjunct flavor to this beer, which would not be surprising and perfectly allowable for the style, but after the identity of the beer was revealed I researched it and learned that this beer is not made with adjuncts.  Apparently, I’ve come to incorrectly associate the low levels of flavor in these beers with the use of adjunct grains, because even in other all grain examples of this style I normally taste something of a corny flavor (maybe it’s DMS).

There also seemed to be a hint of an off flavor in the beer, but it was hard for me to pin down what it was.  Budweiser is known for having a little acetaldehyde, so maybe it was a subtle green apple note I was picking up.  I ended up convincing myself it was mildly skunked from being light struck.  Given these issues I gave the beer a score of 19 points.  Unfortunately for me the beer we were drinking is only sold in cans, so it’s nearly inconceivable that it would be light struck.  The proctors were not in love with this beer, noting that it was a little too bitter for the style, and mentioning some subtle but hard to identify off flavors, but they were considerably more generous than me arriving at a consensus score of 31 points.  I wish I had listened to that review of Gordon giving PBR a 45 before I took the exam.

What a rocky start, I identified two flavors that obviously were not present in the beer and undershot the proctor’s score by 12 points.  It’s just as well I didn’t realize the error of my ways until after the exam was over.

Beer 2 – Saison (Category 25B)

Saison is one of my favorite styles, so I was happy to find myself on more familiar ground for beer #2. In fact the last beer I judged as part of my exam prep was one of my homebrewed saisons. I thought beer #2 was a solid example of the style with expressive yeast derived esters and black peppery phenolics. I awarded a score of 32 points, which was not far from the proctor’s consensus score of 35 points.  I didn’t quite catch the details of the saison that we were drinking, other than it was made with the same yeast that Firestone Walker used on their now discontinued saison, Opal.  Probably a homebrewed example.

Beer 3 – British Brown Ale (Category 13B)

This is not a style I drink very often, it’s probably been a few years since I last tried a British Brown Ale, but I’m old enough to remember when it made you seem vaguely worldly to order a Newcastle Brown Ale.  I liked the malt complexity in the exam beer and picked out flavors like toasted bread, caramel, and if I remember correctly even a nutty accent.  I was also getting a buttery diacetyl note in the background, which at low levels is allowable in many British beer styles and did not seem to detract too much in this beer.  However, now that I’m looking over the style guidelines I see that diacetyl is not specifically mentioned for British Brown Ales.  I later learned that the beer we were tasting was an old bottle of Sam Smith’s Nut Brown Ale.  I was pleased to hear that one of the proctors also felt there was a buttery diacetyl dimension to the beer, but less enthused to learn that the proctors also dinged the beer for an oxidized, papery off-flavor that I had missed.  That might help to explain why my score of 35 points was considerably more generous than the proctors, who settled on a score of 28 points.  On the bright side the discrepancy was just barely within the seven point window.

Beer 4 – Lambic (Category 23D)

The next beer was a sour, funky, nearly uncarbonated Lambic.  I was a little surprised to see this beer in the lineup because unlike Gueuzes, straight Lambics are a rare find outside of Brussels.  Though our exam administrator, Tom Morgan, was wearing a Cantillon t-shirt so maybe I shouldn’t have been surprised.  I was enamored with the funky, complex barnyard type flavors in this beer, and while the acidity was unmistakable, it was not over the top.  If drinking for pleasure I would have gladly taken another glass of this beer.  I liked it so much I gave it a score of 40 points.  The judges were quite far apart in their opinions of the beer.  Gordon seemed to enjoy it quite a bit and also awarded 40 points if I remember correctly, while one of the other proctors picked up a baby diaper aroma (butyric acid?) that was understandably hard to get past. They settled on a consensus score of 36 points, which was thankfully not too far from my score.  After the exam Tom told us we were drinking a mixture of 4 different bottled lambics, one of which had an unmistakable bacterial infection.  He meant for this to be the “bad beer” of the group, but the mixture ended up tasting surprisingly good.  I guess that speaks to the wonders of blending.

Beer 5 – American IPA (Category 21A)

This was my second stumble of the exam. We were given an Imperial IPA (8.0% abv, 99 IBU) from one of the local craft breweries, but told it was an American IPA.  I’d read that this tactic was not uncommon at tasting exams, but that didn’t make it any easier to identify.  It was definitely hop forward and to me the hops tasted and smelled herbaceous, dank, almost cannabis-like, so that’s how I described it on my scoresheet.  Interestingly the proctors felt the hop flavor/aroma tended more toward garlic and onion.  I have to admit that while I’ve often seen people use these terms for hops, I’ve rarely tasted a beer where I would use those descriptors to describe hop aroma and flavor.  Who knows maybe it’s just a blind spot in my sense of taste, or maybe I need to train my palate better.  The proctors also felt it was too bitter for the style and under attenuated, both of which make sense for a DIPA masquerading as an IPA.  They also described a sulfury note that I didn’t pick up on.  Basically, they didn’t care much for this beer (at least as an American IPA) and gave it their lowest score, 25 points.  Unfortunately, I overshot that mark by a sizable margin, giving it 36 points.  The take home point is that even if the flavors and aromas are appropriate for the style (in this case hop-forward and bitter) the intensities might not be.  The same thing could apply to an American Porter pretending to be an English Porter, or a Doppelbock masquerading as a Märzen.

Beer 6 – Belgian Dark Strong (Category 26D)

The final beer was a 10.5% abv homebrewed Belgian dark strong.  Thinking back it had all of the elements of a Belgian Dark Strong, but the alcohol was a little too readily apparent for my taste and I thought there could be a little more complexity.  After all world class examples of this style are some of the most amazing, elegant beers in the world.  I gave it a score of 28 points, but given the fact that the beer in front of me was within the style parameters with only minor flaws I should have scored at least in the low 30s.  I think after giving relatively high scores to the previous three beers I subconsciously felt the urge to score a little lower.  I was also running short of time at this point, which didn’t help.  The proctors scores ranged from 28 to 40 points, and I didn’t catch what they agreed upon as the consensus score, presumably 34-35 points.  The proctor who scored the lowest felt the alcohol was too apparent, echoing the comments I left on my scoresheet, so I’ll take that as an encouraging sign, even though my score was likely 6-7 points away from the consensus score.

Final Thoughts

I can’t complain about the styles on offer at the exam.  Four of the six styles are ones that I enjoy and have drank many times.  I feel pretty good about my evaluation of the Saison and the Lambic, and while I missed some elements I wasn’t wildly off the mark for the British Brown Ale and the Belgian Dark Strong.  The two styles that I whiffed on, American Lager and American IPA, are possibly the two styles that I’ve consumed the most of over my lifetime, but obviously there is a difference between throwing back a few Buds at a tailgate party and critically evaluating the same beer.  I guess the lesson is to not take familiar styles for granted, your notion of these beers might not be exactly the same thing as the style guidelines.  It’s also critical not to let your personal distaste for a style negatively influence your rating.

Looking back I can see that my overall scores were a little out of line on some of the beers.  I’ve read the following guidelines for overall score in a few places and I think it would have helped me to follow these more closely:

• A clearly out-of-style beer is capped in score at 30, with slightly higher scores possible for very technically well-made beers.
• A clearly out-of-style beer that does not have technical faults bottoms out at a score of 20
• A beer with a single bad fault is capped in score at 25
• A beer with a bad infection is capped in score at 20
• A beer that is obviously in the correct style and only has minor balance flaws should score at least a 35
• A beer with no flaws and that exemplifies the style well should score at least 40
• A beer with no flaws that exemplifies the style well and is perfectly brewery-fresh should score at least 45

I’m glad the exam is over, but on the whole I enjoyed the experience.  Unlike most things in life, the training was probably the best part. I think I have a decent shot of scoring in the 70s, which would qualify me as a certified judge.  If that is the case then I’ll be happy.  Whatever the outcome I’ll be sure to let everyone know the results once they come back.