THE EXTRAORDINARY NUMBERS BEHIND LEICESTER’S UNLIKELY TRIUMPH
[Published: Thursday 12th May 2016]
Leicester City received the championship trophy for the English Premier League on Saturday, concluding one of the great sporting upsets of modern times.
But just how unlikely was it? How big a role did ‘luck’ play in their triumph? And why did bookmakers lay them at 5,000/1 at the start of the season?
WERE LEICESTER LUCKY?
The short answer is ‘yes’.
But the long answer (and ‘better’ answer, if you can stick with reading this whole damn thing) is a bit more complicated and nuanced. We will dig into it in the article below.
Leicester were undoubtedly ‘lucky’, as in they had many variable factors go their way. But if you look at almost any great sporting achievement closely enough there will be moments when some random chance fell favourably for the individual/team who ultimately triumphed. Even truly great football teams often need a bit of luck along the road to some great achievement1, such as winning a penalty shoot-out during a cup competition that helps them win a treble.
Leicester aren’t a ‘great’ team, or even anywhere near to earning that kind of description. But saying Leicester were ‘lucky’ isn’t the same as saying they didn’t deserve to win. They did. Winning a 38 game league can’t ever be ‘flukey’. But various stars certainly aligned during the season that helped them on the road to their enormous upset.
Readers of our articles will know we don’t really like the word ‘luck’ anyway. We prefer ‘randomness’. So if you will allow us the pedantic distinction, we think Leicester were on the favourable end of quite a few breaks of random chance.
While we think it’s problematic to say Leicester were lucky, we can say with no shadow of a doubt that their league victory was unlikely. Very, very, unlikely. Although not quite as unlikely as the chances bookmakers ascribed to them at the start of the season. More of that later.
THE WORST WINNERS EVER?
Up to this season, in the history of the English Premier League (first played in season 1992/3) the lowest finishing league position any champion had in the preceding season was 3rd. Leicester finished 14th last season. That alone qualifies their victory as somewhere between ‘remarkable’ and ‘astonishing’.
The single biggest factor in Leicester’s victory was the under-performance of the established ‘big’ clubs in the Premier League. This season’s league was an ‘easy’ one to win. Leicester’s low-80’s2 points haul would normally be enough only for a gallant runner-up or third place spot. But their annus mirrabilis has happened to coincide with the collective failure of the the league’s established ‘Big 4 Beasts'; Man City, Arsenal, Chelsea and Man Utd.
With Leicester, plus the emergence of Tottenham as a genuine title challenger (after years of mostly flirting with Champions League positions) perhaps the days of the Big 4 are over (the new Big 4 formed in the season after Sheik Mansour’s purchase of Man City in 2008. City supplanted Liverpool as a Big 4 team in 2009/10) and a new order is emerging.
Since the 09/10 season this Big 4 have taken up 21 of the Premier League’s 24 Champions League spots for top 4 finishes3 and won all of the league championships.
BREAKING THE LINK BETWEEN SALARY BUDGET AND PERFORMANCE
There is a well-established causal link between the salary budget that top professional teams spend, and the level of performance they attain on the field. This is because the talent of players is the most important factor in how well a team plays, and the market in professional football players’ salaries is pretty efficient.
So the best players command the highest wages, which only the richest clubs can pay – so they become the best teams. There is no way to break this relationship over a long-term because players (essentially) have freedom to move to whichever club is willing to pay them the biggest salary.
This is the pretty solid general rule, but it’s obviously possible to be rich and break it by being awful through incompetent management of your club (see Newcastle). And it is also possible to have short-term success with a smaller budget with a bit of clever/lucky scouting, and/or development of young players – plus if you are lucky enough to get the timing right. Smaller clubs can unearth and get the benefit of star players for a short period of time before they get inevitably get lured away (Matt Le Tissier is the only real exception in living memory that comes to mind who stayed with a ‘smaller’ club).
Clubs like Southampton have had a terrific record at finding/producing quality and buying under-valued players, getting the benefit of them on the pitch for a season or two, and then selling them on to bigger clubs (usually Liverpool) for a large profit. Leicester this season had the good fortune that three star players all ‘popped’ at the same time. Mahrez, Varcy and Kante are Champions League level players who individually and collectively have elevated a an essentially mid-table standard side, mostly made up of journeyman pros, into a very effective football team which is competitive at the upper reaches of the league. If they had emerged at different times then their collective impact would have been less as one, two or perhaps all three would have been sold to bigger clubs and would never had a season playing together. But Leicester kept them together (fresh and fit mostly) this season.
The recruitment and development functions at Leicester deserve big credit for finding and delivering these talents. But you shouldn’t bet on them (or any other similar sized club) being able to repeat the feat any time soon. There will always be a big element of randomness in the recruitment/development of players, so the most that a smaller club can do is to keep finding and exploiting inefficiencies in the market – and then hoping for the best. There’s no magic formula for finding Champions League level players for bargain basement prices. So Leicester were undoubtedly fortunate to have their ‘Big 3′ emerge and have great seasons all at the same time. But boy did they make the most of that fortune.
It’s hard to put a number on the impact, but Leicester have almost certainly benefitted from the freshness that comes with playing fewer games throughout the season, and enduring far fewer injuries than the norm.
Leicester have played 42 games in their season, and made the Premier League their absolute priority. They got knocked out of the FA Cup (in a replay vs Spurs) in the 3rd Round in January, and went out of the League Cup in October having played three ties. In all their cup ties they rested first-teamers, clearly prioritising the league. In contrast Arsenal have played 54 games, counting the Community Shield. Man City have had 59, including Champions League games where resting players is not really an option. Spurs have played 53 competitive games.
In league matches Leicester were able to make just under 30 changes to their starting line-up up to the point they secured the title, when the average for EPL champions is just over 90.
As we saw above, Goal Difference (or Supremacy, which is average GD per game) correlates very strongly long-term with points won. So if you want to look at whether a team has been ‘lucky’ to finish so high up in a final league table (or ‘unlucky’ to finish so low) a good simple way to look for an indication of their ‘true’ underlying performance4 is to look at the GD column, rather than at their Points.
Leicester are (depending on their result at Chelsea on Sunday) going to finish with a GD of around 30. As you can see in the Big 4 GD graph above, that is normally less than the average of the Big 4 in a season, and miles off the standard usually set by the champions.
WINNING BY THE ODD GOAL
Generally speaking, a team who wins games comfortably by multiple goal margins will go on to perform better in future games/seasons than teams who win their games by a single goal. So GD ‘matters’ in that long-term sense, although it doesn’t strictly matter short-term if a team never wins by more than a single goal. George Graham’s Arsenal and Diego Simeone’s Atletico made winning by the odd goal into something of an art form, and very much a repeatable ‘skill’.
But usually, sneaking a succession of single goal victories turns out to be a high-wire act that will eventually catch teams out. In other words, winning lots of games by a single goal is a decent indicator of ‘luck’ in results, which makes logical sense as just one break of the ball at either end of the field going for/against turns 3 points into 1. Leicester this season have had a lot of single goal victories – 14 of their 23 wins in total. The last time an EPL winner had a ratio that high was in Alex Ferguson’s last year at Man Utd.
It is commonly held that Utd started to deteriorate after Ferguson’s departure, but another reading of the data could suggest that they had actually started to decline during his last season, and that they held on for the title there as much by luck as excellence. Whatever, a 1 goal margin ratio (how many of their wins were by a single goal?) as high as 60% is a warning sign that a team may be in for a large ‘correction’ in its results – aka a regression in the mean. In layman’s terms, they are unlikely to get so lucky again next time.
HOW MUCH CREDIT SHOULD CLAUDIO RANIERI GET?
The likeable Italian has brought so much to this season with his affable and humorous nature that it seems a little churlish to downplay his role. But to return briefly to a theme discussed in some of our other articles, the importance of the ‘coach’ of a professional football team is generally massively exaggerated. Ranieri took over the reins, and replaced Nigel Pearson as a cog in a well-designed machine at Leicester5, and carried on the work and improvement that was already well established. Consider these two possible explanations;
- Claudio Ranieri, after years as a ‘journeyman’ coach without any league titles, and a record of achieving results pretty much in line with the salary budgets of the clubs he was managing, and after a recent disastrous tenure as the manager of the Greece national team (they lost to the Faroe Islands) finally, in his 65th year, unlocks the ‘secret’ of football and single-handedly and gloriously transforms little Leicester City from relegation contenders into dominant title winners. Or..
- Claudio Ranieri comes into the head coach position at Leicester, where he is asked to take over responsibility for training the first team. He is supported by a backroom staff that remains the same as it was with the previous person in that position. Three star players emerge, and the Big 4 clubs have a collective shocker. The likeable Italian fosters a positive dressing room atmosphere. Little Leicester ride their luck in plenty of games, and also exploit superior freshness and player availability to record enough points to win the league, despite their goal difference being well below the level usually required to be champions.
Which of these two scenarios sounds likely to be closer to the truth?
Apparently it was Ranieri’s agreement to come into Leicester as a ‘head coach’ without insisting on bringing in his own staff (which is the ‘traditional’ thing with managerial appointments at UK football clubs, but is much harder to insist upon if you have just lost to the Faroe Islands) that helped him land the job ahead of others with a higher profile/stock at the time.
Another interesting technical observation of Leicester’s play this season is that they have pretty much played 4-4-2 the whole way (they lined up in this ‘old-fashioned’ system for 33 of their first 37 league games, according to whoscored.com). As we’ve talked about before in our articles, there is a general tendency to over-estimate the importance of tactics and formations in assessing the merits of a football team. The level a team attains is almost all down to the aptitude of the players, plus a bit for the chemistry between them that can create a whole which is greater than the sum of its parts.
So finding a decent formation that suits the players is pretty important, at least in the sense that it is possible to get this bit of team-building badly wrong. But perhaps Leicester’s triumph will help to dispel the notion that there is something inherently superior in currently fashionable tactical formations such as 4-2-3-1.
HOW DO LEICESTER COMPARE TO THE CHAMPIONS OF THE OTHER BIG EUROPEAN LEAGUES?
Leicester are comfortably the weakest of the winners of the 5 major European leagues (England, Spain, Italy, Germany, France) and would be made big outsiders in a one-off game or a league season against any of the others.
If Leciester played Barcelona (both at full stength) in a meaningful one-off game like a cup final on neutral territory right now, the market price for Barcelona to win that game in 90 minutes would be roughly 1.25 (1/4 as a fractional odd, 80% as a percentage)6.
Most very successful football teams have a high average percentage of possession in the games they play. So we can say there is a ‘correlation’ between high possession% and success. But there is no direct ‘causal’ relationship between possession and success. Having high possession doesn’t cause a team to be successful, and having a minority of possession doesn’t automatically mean a team will struggle. “Correlation does not imply causation”.
Possession% stats are more a symptom of playing style than they are a cause of success. Leicester are an excellent example of this, as are Atletico Madrid who are in another Champions League final and went close in La Liga with a minority of possession during their season. You could say; it matters much less what your plan is, than that you have a plan. A plan that everybody buys into it.
However Leicester are conspicuously different in style to the champions7 of the other major European leagues this season, and this is reflected in their possession stats.
Another way to look at a team’s performance is to see how many of their goals they score from ‘Open Play’ (as defined by whoscored.com). As you can see below, Leicester have scored a much bigger percentage of their goals from non open play situations. This is another indicator suggesting that good fortune has played a fair part in Leicester’s success, as creating goals from open play is a much more repeatable skill/trait/style than relying on them coming from other means.
Atletico Madrid, as in many things relating to football and stats, are a riddle wrapped in a mystery inside an enigma. Last season they mounted another decent challenge in La Liga (finishing 3rd) with a Non OP Goal ratio of 56.7% (they scored 30 goals from set-pieces). Had Simeone discovered a new ‘winning ugly’ way of playing football that would allow teams on smaller salary budgets to compete with the world’s richest clubs? Well, this season they almost won La Liga with a Non OP Gls% of just 24.6%. Go figure.
The reason for highlighting Atletico is to say that while all/many of the performance indicators foretell of a big comedown next season for Leicester, and a probable return to a mid-table league position (you will probably be picking up that we reckon this is the most likely scenario next season) Atletico have shown over recent seasons that maybe it IS possible to keep defying odds/stats/logic by staying at the top of a major league amongst clubs with a much higher salary budget – at least for a few seasons.
HOW HIGHLY SHOULD WE ALL HAVE RATED LEICESTER AT THE START OF THE SEASON?
Not everybody likes looking at ‘fancy stats’ that relate to football (in fact some people are openly hostile to such ‘nerd nonsense’) but looking at underlying performance indicators is essential in getting an edge in disciplines such as betting. Football is a low scoring sport; goals are scarce, league tables lie, results lie. Short term success or failure can be down to little more than random variance.
So looking at performance that underlies how a team is actually playing can be very useful. At OddsModel we have various methods of quantifying the performance of football teams that goes deeper than their bare results, and one pretty simple one is to look at the ratio of Shots on Target (SOTR) in its league games (of all the shots on target in their games, what % did the team take?).
Looking at these ratios for last season gives a suggestion that Leicester were an improving unit, showing a decent increase from 1st to 2nd half of the 14_15 season.
By this measure, Leicester played the second half of last season like a solid mid-table side, rather than the borderline relegation contenders they presented as in their first 19 games. We will see a bit later that this improvement (and Chelsea’s decline) was picked up by the Asian betting market, which is not surpising as many of the more influential, large-staking professional gamblers and syndicates who help to shape the Asian market use metrics such as SOTR% to dig into underlying performance and shape their ratings of teams.
Looking at the same measure of underlying performance to include this season, Leicester continue to improve. This is a common ‘shape’ to a good team’s development. Revolutions are rare. Good teams tend to arrive at their potential after a period of sustained improvement, rather than after an overnight transformation. To us it supports the theory that Ranieri was more an enabler of a continued upward trend of improvement, rather than a magician who conjured a title-challenging team from thin air.
But as you can see in the table below, in the second half of this season Leicester come out as a genuine top-notch side by this measure. The momentum of their run of results seemed to sweep up even their journeymen players. When Vardy and Mahrez didn’t get the winner, somebody else would step up.
The importance of a settled spine is probably an under-rated factor in the level that football teams attain. And the degree to which the efficiency of a central defensive pairing is down to their ability to work as a pair, as opposed to raw individual merit, is also a recurring factor in the performances of successful teams. Nobody before this season would have rated Wes Morgan or Robert Huth as ‘title winning class’ centre-backs, but through playing together consistently in front of the same excellent goalkeeper, and (perhaps most importantly) behind an excellent shielding central midfield pair they were able to look top class.
That Leicester managed to find themselves top of the league at Christmas having had a minority of all shots on target in their games is a big-time statistical anomaly. There is an element to which playing style can buck the normal trends of SOTR analysis. If teams prioritise and become extremely adept at counter-attacking then they can sustain a period where they create a better quality of chance than is the norm for a ‘shot on target’ stat. A shot from around the penalty spot with only the keeper to beat is ‘worth’ much more than a shot taken from outside the D with a phalanx of defenders in the way, which barely wriggles it’s way through to the keeper.
Currently trendy Expected Goals models are over-rated for this reason, and can under-rate sides like Leicester and players like Mahrez and Vardy. While they recognise that a shot from around the penalty spot is worth more than a shot from the D, they don’t recognise how much room the shooter is in, how much pressure there is on the ball, if it’s on his stronger/weaker foot or head, if the ball is bobbling, where the defenders are, what position the keepers is in………
So the high league position versus a sub-average SOTR performance at halfway in the league is probably a better indicator of Leicester’s style of play, than of its quality to that point. It’s easy to see in the mind’s eye repeated scenarios throughout the season where Kante won the ball back with a tackle or interception around halfway, quickly fed Mahrez wide right, who made ground quickly, skinned a defender and then slipped in Vardy to race clear of the defence and bear down 1-on-1 with the goalkeeper. Similar-ish scenarios seemed to be happening every time you watched a Leicester game all season.
Leicester’s opponents converted their shots into goals at an unsustatinably low level. And Leicester converted their own chances at a rate it is very unlikely they can repeat. There is some skill, but also a whole load of randomness involved in conversion rates.
It is probably also a factor that Leicester were under-estimated by opposition teams, and so the opposition scouting and preparation by their opponents wasn’t anywhere near as good as it should have been. The Kante-Mahrez-Vardy triumvirate is a formidable force to try to stop, but it felt like teams kept playing right into their hands week after week.
SO WHAT PRICE SHOULD LEICESTER HAVE BEEN?
About 416/1 is the short answer.
Again, the long answer is necessarily a bit long and involved – read on if you like a bit of bit of ratings and modelling chat. Also, read here for a good account of a bookmaker insider’s perspective on the same questions posed and answered below.
Something we can say for certain at the outset though is that they should NEVER have been a 5,000/1 shot.
Working out the ‘true price’ of an outcome in a league competition involving 380 games is tough. The level of compexity and inter-relatedness of events is huge. By far the best way to do model/price something this complex (that we know of) is to create a Monte Carlo simulation8 of the league, using a random number generator to create results of every game. You run the simulation a large number of times, and the output of the model is a set of probabilities (aka prices/odds/percentages) derived from the number of times each eventuality occurred during the many iterations of the model.
The random number is used to generate outcomes within the model that over a large enough sample size reflect the true odds of each outcome occurring. In other words, if the model gives Manchester United a 50% chance of winning a given match, then the model will have them winning the game on half of all the iterations it simulates.
The hard part of saying what price Leicester ‘should’ have been at the start of the season is to run a simulation now that uses no benefit of hindsight. ‘After-timing’ (using the benefit of hindsight, being wise after the event, claiming you knew what was going to happen when you really had no idea at all….) is a mortal sin in our world, so we need a set of ratings for each team to put into the model which have zero influence from anything that happened after the season started on 8th August 2015.
So we used ratings derived directly from the Asian betting market for each team – i.e. ratings that are worked out from the Asian Handicap prices for league games. The ratings we put into our MC Sim model were generated using prices from the previous season, and up to the opening round of games in this season’s fixture list.
They are therefore free from any bias from our own subjective opinions, or from any post-fitting based on knowledge of results, performances, or changes to the ‘opinion’ of the market after kick-off in those first games. We calcuated the ratings on the same basis for ‘now’ – i.e. the end/close of this season. The ratings are in the table below.
A tricky element in such a method is how to model for improvements/declines by teams throughout the season. It wouldn’t represent what happens in reality for us to apply each team’s season opening rating for every game throughout the season – that would be poor modelling. Teams will always be improving/declining for a huge number of possible reasons. So our model allows for a team’s rating to vary by a small amount after each game according to the result the model randomly generates. We can’t go into detail about how exactly we model this (for reasons of time and confidentiality) but we can say with confidence that our model generates very reliable estimates of probabilities.
Projection of probabilities on future events that inherently contain randomness are always guesses. In our case we believe the guesses of our model are very good and ‘well educated’ guesses, but they are guesses all the same. An important factor to understand is that the output of a simulation model is never a prediction. Instead such models generate a range of possibilities. It says how likely various things are to happen, not what ‘will’ happen.
In common with all bookmakers/odds compilers/professional gamblers and investors, at OddsModel we belive in thinking ‘probabilistically’ – in other words we assign a likelihood/price/odd/chance to any possible outcome. We believe that (virtually) everything that happens in the universe is subject to randomness, so that the best you can do with any complex future event is to estimate the probabilities of all the different possible outcomes. Anything that can happen, will happen. Eventually.
So it is a very important aspect of modelling to take care to estimate well the chance of the unlikely events as well as the likely ones. In other words, we work hard to be work out the best estimation of prices for outsiders, as well as the favourites in any market. We never want to make the potentially ruinous mistake of denying the existence of Black Swans.
THE OUTPUT OF OUR SIMULATION MODEL
Using the Season Opening market rating we ran 100,000 simulations of the 2015_16 English Premier League season in our model and these are the outcomes…
If the ‘true’ odds (at least our best guess of the true odds, based on the objective ratings derived from the Asian market) were just over 400/1, how did some people get to bet on Leicester at 5,000/1?
Short answer; the bookmakers made a huge mistake – an almighty rick. 5,000/1 was just a terrible price for which they got soundly punished. Any price for Leicester in 4 figures represented a big ‘value investment’ opportunity for gamblers. More on that below.
HOW DID THE 8TH BEST TEAM IN A LEAGUE MANAGE TO WIN IT?
The market started the season thinking Leicester were the 10th best team in the league. They ended the season rating them 8th. And the market is smart. In fact its really smart. So how is it possible that Leicester won?
Well, it was unlikely. But shit happens. Everything that can happpen, will happen eventually. One of the advantages of running a MC Sim model is that you get to see just exactly how unlikely the more outlandish possible outcomes really are. Without that aid to thinking probabilistically it is all too easy to mislabel something that is actually ‘unlikely’, as ‘impossible’. How often do we say “ah, it’ll never actually happen”? When actually it WILL happen eventually, if we just hang around long enough to see it.
The next table shows something pretty remarkable. It shows the output of our simulation model where the chances of each team winning each game are generated not by ratings, but by the actual odds taken from the Asian market each week. So we took the actual odds of each match finishing 1 X or 2 and generated results using a random number based on those possibilities. So this table shows how likely every outcome was according to how the market saw each round of games.
Leicester come out as LESS likely to win the league using these parameters!? How is that possible? The answer is that the market, although it was relatively bullish about Leicester at the start of the season (especially compared to the bookmakers’ outright prices), remained resolutely bearish on Leicester throughout the season. In other words, the market just couldn’t bring itself to rate Leicester anywhere near as highly as their league position suggested they deserved. The market even made Everton a better team than Leicester until just a couple of weeks ago.
So whereas in our simulation model using Opening ratings Leicester improved often enough from their initial rating to win the title once in every 417 instances, using the odds from each game they would only get the results needed to win the league once in every 493 runnings. In this case, the elasticity that we buit into the rating of the teams in our ratings based Sim was a little greater than the elasticity that the market allowed to Leicester’s improvement as the season went on.
The market gets pretty much most things right, pretty much most of the time. And while its always dangerous/wrong/unfair to ever criticise with the benefit of after-timing, you would say if the market was a human that it has seemed to spend all season adopting a pig-headed reluctance to accept that Leicester were actually more than a tiny bit better than their rating at the start of the season. Any better than the 9th best team in the league that means.
HOW LIKELY ARE LEICESTER TO REPEAT NEXT SEASON?
Unlikely. But by now hopefully you expect a bit more of a scientific and specific answer than that.
So let’s say 47/1. Or 2.1% (assuming they keep their Big 3 star players).
We are going to use the Asian market Closing ratings here. If after reading the ‘mistake’ it made with Leicester above you are inclined to dismiss the market as a judge of football teams, just bear in mind that the Asian market is undoubtedly the single best football analyst in the world.
It may occasionally seem like it doesn’t know what it’s doing, as with Leicester, Everton and Man City this season. But remember that millions of people ‘take on the market’ by betting on football, and over a long term the market beats 99.9% of them. It’s not infallible, but you have to be very good to earn your place among the 0.1%.
Something the 0.1% all have in common is a respect for the intelligence of the market. It has the benefit of using the wisdom of crowds, it ignores hype and it never reads journalist’s opinions. The market was ahead of the curve in recognising that Leicester were a solid mid-table outfit at the end of last season/start of this season, while many observers (pundits, bookmakers and league table absolutists) rated them down with the relegation candidates as the 15_16 season got underway.
The market thinks Spurs and West Ham have actually improved more throughout this season. And it has been dogged in its reluctance to up its rating of Leicester – even now as they parade the league champions’ trophy the market says they are the 9th best team in the land.
Is the market wrong? The answer to that question (when ‘the market’ referred to is the Asian Handicap market on top level European league football matches) invariably, is ‘no’. It has Leicester improving a significant amount throughout the season (0.4 of a goal in supremacy is a good amount, although it was only enough to move them up two spots in the rankings). With the benefit of hindsight, of course the market would have upgraded Leicester to that mark more quickly than it did – but then everything is easy for everybody using the benefit of hindsight.
All of which is to say that we are going to run our simulation model one more time. This time we are going to back to using ratings, only this time we are simulating using the ratings of the Asian market at the end/close of the season. Based on how good the market thinks each team is now, how likely are all the possible outcomes – assuming we started the league again from this moment?
SO WHY DID BOOKMAKERS OFFER PRICES AS BIG AS 5,000/1 AT THE START OF THE SEASON?
It is fair to point out that not all bookmakers were as big as 5,000/1, although all the major UK firms were at least a four figure price about Leicester up to the kick-off of this season, and then kept them at huge prices well into the season.
But the short answer to the question is pretty simple; they made a big rick. A whopping mistake. It was a calamitous collective failure in odds-compiling to rank with the shocker that the Premier League’s Big 4 teams suffered this season. This was the UK bookmakers’ Big Short moment.
The long answer is, again, a bit more involved but (hopefully) fairly interesting to look into.
How and why did bookmaker make such an error? Aren’t bookmakers supposed to be smart at estimating probabilities?
The true price we would have made Leicester was 416/1. Why the discrepancy? The answer to that one (and there is no way to say this without sounding rather arrogant) is that at OddsModel we are probably a good bit better at generating true prices for complex events like a 380 game league season than bookmakers are.
Our Monte Carlo simulation is probably better than theirs, and we are probably better at devising ratings for teams than they are – either by using the Asian market, or using statistical analysis. Also, a lot of the time they just guess, or copy each other on outright markets.
There are a few explanations for what actually went wrong with the bookmakers, and as is usually the case the truth probably lies in some sort of combination of these various factors, rather than in one all-explaining reason.
It seems like they rated Leicester lower than the Asian market ahead of the season starting. Maybe they were unimpressed with Ranieri’s appointment, perhaps buying into the media narrative that managers matter a lot, and that this new manager had just lost a game to the Faroe Islands. And who the hell is this guy N’Golo Kante?
If they have a simulation model, it probably isn’t a very good one. Getting the calibration of a model of a complex event is difficult, so their model may have had too little elasticity in how it adjusted the ratings of the teams. Or they might not have run enough iterations of the simulation. Or the way the simulation generates random outcomes may be flawed. Hard to know, but even with a negative view of Leicester at the start of the season it is tough to get it to say they were anything like 5,000/1.
Bookmakers ‘herd’. The ‘pure’ idea of bookmaking is where every layer independently comes up their own estimation of probabilities, from which they deduct a profit margin to arrive at their prices which they then offer to punters. But the modern reality of bookmaking is that generating prices to offer to its customers is much more to do with bookmakers ‘reading the market’ (copying prices from elsewhere, if you want to be un-generous) than it is to do with starting with a blank sheet of paper and working out what they think the ‘true’ prices are.
In football the two markets that bookmakers look to are on Betfair (the exchange, not the sportsbook) and in Asia (for example on sbobet.com). On individual games the prices of all bookmakers track the main Asian lines. On outright markets (on which the Asian bookmakers don’t have such a strong market) it is more likely to be Betfair. But in this instance odds-compilers had a problem because the maximum price on Betfair is 999/1.
Calculating small % probabilities is hard. Even if you have a brain like Garry Kasparov, telling the difference between something being a true 400/1 chance and a 4,000/1 possibility is actually really difficult (write these odds as percentages and the difference seems much less distinct; 400/1 is 0.24% while 4,000/1 is 0.024%).
So the reality is probably that bookmakers essentially guessed what Leicester’s true price should be, and guessed wrong. Because they herd together they were all left offering a rick price about Leicester, and paid the penalty when they incurred significant losses (probably £12m+ across the various UK bookmaking firms).
To get a bit deeper into the maths of the market, you can see from the table below that actually the bookmakers were offering wrong prices about the majority of teams in the league (according to our simulated prices anyway), although the value for punters was greatest on Leicester at 5,000/1.
Normally if your model is saying that there are 16 ‘value’ selections out of 20 in a market, then it would be safe to say your model was wrong. But not in this case.
This is because the boomakers got the ‘shape’ of the market wrong here. Their over-estimation of the chance of one of the Big 4 winning the league (crunching the numbers, their prices imply a 94.7% chance that one of the Big 4 would win the league, when our model shows that the true chance of that probability was 87.8%). The bookmaker prices probably reflect lazy ‘hedgehog style‘ thinking – ‘because one of the Big 4 always wins the league, one of them will win the league this year’. But the Big 4 aren’t as good as they were a few years ago when they regularly dominated the latter stages of the Champions League.
The single biggest discrepancy was with Chelsea, who the bookmakers all made favourites to win the league. But the Asian market had started to significantly downgrade them at the end of the previous season, even as they were apparently cruising to the title. Maybe the UK bookmakers decided to ignore this evidence, and the evidence of stats like their steep decline in SOTR%. They wouldn’t be the first, and are unlikely to be the last, to suffer for over-estimating Jose Mourinho, the most over-rated man in football.
Using our model and market rating Chelsea were a true 7.7 shot (13%) whereas the bookmakers made them no bigger than 2.875 (28%). That Chelsea price alone took a significant chunk out of the market, which should have been distributed among the other teams in the league. Over-rating Chelsea knocked the whole market out of kilter and created the conditions for the huge collective failure in pricing.
The Big 4 clubs suffered a very unusual collective failure this season. So could you argue the bookmakers were unlucky? In our opinion, no, not at all.
Boomaking is not about predicting what will happen. It is about estimating probabilities, and offering prices that reflect those probabilities (minus a little bit that you keep as your profit margin). So no, they didn’t get unlucky. They got the market wrong, and suffered the consequences of what awaits any bookmaker/casino/poker player/mortgage backed securities trader/gambler/investor who bets or trades or invests at negative value prices. Which is that the remorseless, irresistible forces of the laws of probabilities and big numbers will catch up with you in the end.
If you keep playing with fire by betting at negative value prices eventually you will get burned. Like in the Big Short, the bookmakers just misunderstood and mis-priced possible outcomes in the market. Of course they can claim to have been unlucky. Even by our estimation of Leicester’s true chances of winning the league, the bookmakers would only have had to pay up on them once in roughly every four centuries, whatever price they layed.
ANY SYMPATHY AT ALL FOR THE BOOKMAKERS?
No, not from us.
The ratings we used, derived from the Asian Handicap market, exploit data that is freely available in the public domain. In theory anyone can look at Asian lines and generate ratings from them (admittedly the process of extracting ratings from match prices is a little tricky), but certainly any odds compiler working for a bookmaker should be able to do it.
So by applying ratings in their simulation models at the start of the season (if that is how they priced the market) that had Leicester rated lower than the Asian market they were effectively taking a chance on the market being wrong. Great riches can await any football bettor or layer who can consistently beat the Asian market, but many more perish in the attempt than ever emerge victorious.
So this was a case of the biter bitten – week by week the bookmakers use the Asian market to help them generate the prices they offer to their customers on football coupons and on their websites. Using the strength of this market helps them generate a profit that they would otherwise struggle to make if they had to price games up without any reference to Asia or betting exchanges.
For the EPL 15_16 outright the bookmakers either ignored the direction that the Asian market was offering them (by pricing Leicester as though they were fringe relegation candidates at four figure prices, rather than the solid mid-table side the market had them, for example) or they didn’t have the skills to work out how the Asian market rated them. Or they didn’t even consider ratings, models and probabilities – and just guessed at the right price off the top of their heads. Or they just copied their competitors’ prices. Any way you look at it, although it was a rick that bookmakers would normally get away with, they deserved to lose.
While the prices shown above are ‘ante-post’ (i.e. before the season started) and the 5,000/1 price is the one that has been cited most often in the media, the reality is probably that the bulk of bookmaker liabilities actually piled up from bets on Leicester during the early months of the season. So it was a case of some good/lucky punters starting to belive in the Leicester miracle story before the bookmakers did. Bookmakers will ‘update’ prices on outright markets rather than start afresh after each round of games, and having started at way too high a base on Leicester they didn’t correct the mistake ‘in-play’. The seeds of the wrong prices they layed late last year were sown in their mis-pricing of the market in pre-season. Leicester won’t have been their only (or perhaps even biggest) liability on the market.
The essential fundamental of bookmaking is to accurately work out probabilities, and to express those probabilities in your prices. The bigger story behind their Leicester losses is one of bookmakers no longer possessing that ability. They reaped their long-term punishment for pursuing a policy of reliance on copying the prices of others, at the expense of investing in people and systems smart enough to estimate probabilities independently, starting from a blank sheet of paper9.
The job of a bookmaker odds-compiler is now mostly about following the market (which works very well the vast majority of the time, week to week on individual matches) rather than actually ‘compiling’. European bookmakers restrict and close the accounts of the minority of punters who are shrewd enough to win money off them, and compete through marketing spend for the recreational/mug/losing punters who make up the vast majority of the European sports betting market.
The long-term effect of these factors is that odds compiling skills in bookmaker trading rooms are much weaker than you would probably suspect. It isn’t a highly prized or valued skill any more, because it just isn’t needed by the bookmakers in order to win 99% of the time. The outright market on this season’s EPL was one of the 1% occasions though.
PREDICTIONS FOR NEXT SEASON?
As committed probabilistic ‘fox’ thinkers, we don’t normally do predictions. But we’ll make an exception here as a reward for you seeing this epic length article through to the finish;
We predict that NO teams will be priced as big as 5,000/1 by the UK bookmakers ante-post for the 16_17 Premier League season.
We predict that Leicester will NOT win the Premier League in 16_17.
We predict that Leicester’s championship win will NOT herald a new era where small clubs regularly win the league, or even qualify for the Champions League.
We predict that the league will largely revert to the normal way of things, and a small number of big clubs with the biggest salary budgets will dominate the top 4 places in the next few seasons.
We predict that wherever Jose Murinho turns up next, the results he gets at his new club will be in line with, or a little worse than what you’d expect given the salary budget at his disposal.
We predict that Kante, Vardy and Mahrez will NOT line up together for Leicester next season.
And we predict that Leicester will finish 8th in the Premier League table next season. Probably.
1 Easy to forget that Germany needed two late extra-time goals to get past Algeria in the Last 16 of the 2014 World Cup. And then another extra-time winner to prevail in the final vs Argentina. It’s the semi-final annihilation of Brazil that sticks in the mind most though.
2 We are writing/publishing this article a few days before Leicester play their final league game of the season on Sunday 15th May 2016.
3 Spurs in 9/10 and 11/12, and Liverpool in 13/14 have been the only blots on the dominance of the Big 4 in the previous 6 seasons.
4 A really good example of a team who over-achieved with a high league position was Newcastle in 2011/12. They finished 5th in that final table, just missing out on the Champions League. But their GD was only +5, which is usually only good enough for about 8/9th in the league. Chelsea finished below them in 6th on +19. So here GD was a far better indicator of future performance than league position. Chelsea finished 3rd in the following season, Newcastle 16th (with their manager, Alan Pardew, now tied to an 8 year contract).
5 Leicester, like Southampton, are a club which has eschewed the tradional British football club management model where a ‘manager’ is appointed and is given all-powerful overlord status. At Leicester and Saints the ‘head coach’ trains the first team, and is a replaceable cog in the machine, not the controller of all football operations at the club.
6 Barca would be roughly -2 goal favourites on the Asian Handicap line.
7 We are assuming Barcelona clinch the title on Saturday. Real Madrid have had 55.9% possession this season in the league.
8 Imagine that you inherited a roulette wheel (from an uncle who owned a casino, and left it to you in his will), and that you decided to start using it to take bets. But the numbered slots on the wheel didn’t appear to all be the same size, so you weren’t sure what prices you should be offering for each number. Instead of guessing at the correct prices by looking at all the slots and estimating the probability of the ball falling into each one, you start spinning the wheel and the ball and you record the result of every spin. You stand at the wheel until you have spun it 100,000 times, and you have a note of all 100,000 results. You add up the number of times the ball landed in each slot, and divide that number into 100k. The numbers you come up with are your estimations of the ‘true’ probabilities/odds/prices for each slot. That is a Monte Carlo simulation. You then take a little bit off the true prices as your profit margin and start offering bets at these adjusted prices, and trust that (no matter what happens in the short term) over a long-term the roulette wheel will generate you a profit. That is being a bookmaker. Or thinking/operating like a casino. Or a professional gambler/investor who believes in ‘value investing’.
9 To be fair to bookmakers, as we’ve explained above, the Asian market on football matches is now so strong that it would be crazy NOT to use it to help price up games. The market is a much smarter analyst and judge than any single individual working for a bookmaker, a betting syndicate or a professional gambler. Syndicates and pro punters only make a profit because they are selective in the matches they bet on, sticking to exploiting the small inefficiencies they can find in the market on a select few games. If they were to bet on every match, they would lose too. Also, in UK bookmaker firms the dominant betting sport historically has been horse racing. This sport is different/unique in the sense that so much of the information relevant to the chance of each horse performing at a given level in a race is not in the public domain. In football, golf, tennis, cricket etc. (virtually) all the evidence about how good teams and players are is laid out in high definition TV pictures for everyone to see, and a plethora of stats on every match are generated and can be looked at and used. But with racing, so much of what goes on with a horse happens behind ‘closed doors’ on the gallops each morning, and only a select few are privileged to the information that comes from there. So UK bookmakers have traditionally had to adopt a very defensive approach to laying bets – there were/are always people out there who know more about the horses than they do. But with other mainstream sports this isn’t the case – pretty much everything that is relevant to the performance of a football/cricket/rugby team, or a golfer/tennis player is out there in the public domain. So restricting/closing the accounts of people who bet on these sports is mostly just lazy bookmaking. With all the resources at their disposal there is no real excuse for bookmakers being worse at pricing up these sports, with the considerable built-in advantage of their over-round (theoretical profit margin) in their favour which should be more than a match for the punters’ advantage of being able to be selective in what/when they bet.
Note We only built our Monte Carlo Simulation model midway through the season. But the ratings we used to come up with the 416/1 price were the Asian market ratings we had at the start of the season, so it’s not an ‘after-timed’ price – it is genuinely the price we would have come up with had we had the model back then, and we used the market ratings to simulate the season (you’ll have to take our word about that admittedly, which we hope you do!). In fact our own OddsModel ratings have consistently had Leicester rated a bit better than the market so they would have been shorter if we used those. But we have been careful not to let any subjectivity creep in. By using the market ratings in our sim we can give a genuinely objective price for each outcome free from any after-timing, back-fitting or subjective opinions. And no, we didn’t back Leicester to win the league. We backed Spurs. So what the hell do we know?