Whiskymetrics: Scoring whisky – the results

Writing an article on how I score whisky would be as useful as an article about how I ride a bike – it may help you but more likely it may not. So rather than explain HOW I score, here is a little on WHY I score and what I can do with this information afterwards. There have been many heated debates around the subject of scoring whisky. Those debates I have had with many of my whisky blogging friends & associates, much of the problem (& negativity) I hear stems from a few simple factors:

  1. Jim Murray and his infamous whisky bible
  2. The subjective effect
  3. An inability [or reluctance] to put a specific value to a given whisky

I personally have never had an issue with any of these – hence I score or ‘Inscribo ergo sum’

The problem as I see it isn’t that Jim Murray (& his team of around a dozen pre-taster individuals) score whisky and produces his yearly bible – but rather how much weight the readers of this book place on those scores. Equally Serge Valentin, the taster of 10,000 whiskies over at WhiskyFun.com also has a strong market influence on those in the know enough to follow his musings. In fact by their construct Murray’s method is more akin to the proliferative and far too numerous Whisky Awards type competitions that adorn our bottles with gold, silver  and bronze medals each year. These are panel-led tastings and represent an overall opinion of all of the tasters whereas at Whisky Fun you are buying into Serge’s preference & flavour profile as well as his vast experience, which to a lesser degree I am sure Murray still gets to pick his favourites too. Personally I tend to use both Murray and Serge’s recommendations for things I should seek out and try for comparison – the problem is everyone else is stockpiling these self-same high-scoring bottles for personal consumption or flipping at whisky auctions to make a quick buck. Those I have managed to sample have given mixed results – some were great, and some not so – but all were equally worth trying. Like any advice these scores should be taken with a pinch of salt.

Yes scoring is subjective – my scores reflect how much I like the whisky – you may not agree (that is actually a good thing as it opens up room for a debate and what better to talk about than whisky?), but with experience you are able to tell quality in whisky even though the flavour-profile is not your favourite. So inability is no excuse. In fact I find it impossible to believe that inadvertently all whisky drinkers don’t unconsciously score their whisky. The number of whisky drinkers I have heard proclaim a whisky is ‘better than the other one they tried from that distillery’ or ‘not as good as my favourite’ etc, only increases the more you subject them to the variety that whisky has to offer. For example every whisky festival I have attended has had more than one of my fellow tasters recommending a particular whisky or asking if I have tried a whisky on offer as it was the best they had tasted that day, or was exceptional, etc.

I started scoring whisky right from the beginning of my whisky journey, before I was even aware of or had ever picked up a copy of JM’s bible. When I started scoring I took advice from other fields such as wine scoring, true 100-point scales vs (Robert) Parker Points (50-100) etc. and very quickly decided to form my own which you can find over on our scores page but I will repeat here for clarity:

Tasting scale:

90-100 Exceptional, a truly great whisky
80-89 Outstanding whisky with great character and style
70-79 Superior whisky with special qualities
60-69 Good solid whisky
50-59 Average whisky, drinkable with minor flaws
40-49 Below average, drinkable but not recommended
30-39 Poor whisky with obvious flaws
0-29 Potentially hazardous to health / fire water

Professional (actually, amateur!) reluctance from bloggers about scoring however, could be born out of the fact that every free whisky sample must be met with a glowing review declaring it top stuff, so that the free samples keep coming. Reporting an exact score in a review automatically gives a direct comparator between your reviewed samples – allowing the provider to see if you rate theirs better or worse than their competitors. There have been rumours that this leads to less free samples – I wouldn’t be so sure that distribution of PR samples by some companies operates at much higher intelligence than ‘which blogger has the most followers – job done’ as any PR is good PR – in fact I have witnessed some producers publicising reviews which were rather unfavourable? I don’t have much time for companies that would rather not have honest reviews of their products, or who prefer their products reviewed by people who bought 5k fake twitter account followers yet can’t put together a decent tasting note. Blogging is about having opinions, with the -s being the most important i.e. more than the 1 opinion that is “yeah it’s great – just as wonderful as every other free sample [i.e. please keep sending me more]”. That said I still have companies more than willing to send out samples for honest reviews – which is nice.

Having scored well  over 1,000 whiskies using the same criteria (as near as possible) I hope to start using some of this data to see what it tells me about my own preferences – after all I am a regular whisky drinker! Below is the initial analysis using 100 randomly selected anonymised whiskies from our published archive at The Whiskyphiles. So what does the data tell us?

score vs cost

Expensive whisky is good whisky – in general yes. The more expensive it is the more likely it scores highly, however this analysis suffers from the fact that the more expensive it is the less likely I am to try it and so my dataset is stacked towards the bottom end of this graph. The observant may spot a cluster that equates to ~ £60-£80 a bottle where odds are you will find something that scores very well. To account for variation in bottle size (e.g. 70 vs 50cl etc.) I have normalised cost to £/cl. An important point to note is that the majority of whiskies are scored with no prior knowledge of the cost though this is not a truly blinded experiment as for some even only a few details eludes to the rarity and potential cost. I do try where possible to taste first and formulate my initial reactions and score before reading the whisky sample details e.g. distillery/age/maturation type etc.

score vs abv

Cask strength is good whisky – Careful with this one as maturation age goes up then cask strength goes down and so there is a peak at around 50-55% ABV, anything over that and scores decrease with strength likely reflecting the likelihood of younger whisky and also the effect of high spirit strength on the palate. Also Old cask strength whisky mixes it at the bottom end of the scale with those whiskies diluted to standard concentrations (the familiar 40, 43 and 46% ABV of staple expressions) – speaking of these in general average score increases as ABV does across these points. In general ABV is not a good indication of quality but does show some interesting trends.

score vs age

Old whisky is good whisky – Again yes, either biased or un-biased by the inclusion or exemption of no-age declaration whisky (plotted as 0 by default as I cannot ascribe an exact age and there will be massive variation). There appears in general to be a subtle increase in enjoyment score with increasing age with some notable outliers – again ABV may have influenced some of these. NAS whisky show the largest range of scores, and apart from a couple of outliers generally score similar to age-stated expressions. Something we have explored previously in The Great Age-Statement Debate

Of course I will continue to happily score and report that score for each and every whisky we sample and also continue with our Whiskymetrics analysis with a bigger dataset!