We all know the 2018 Crusaders were good – but just how good?
If you lined them up in an imaginary match against the 2002 Crusaders, the 1997 Blues or the Brumbies of the turn of the century, where would the smart money go?
That's where we bring in Elo,a ratings system based on the work of Hungarian-American physics professor Arpad Elo, who came up with a system for calculating the relative skill levels of players in zero-sum games such as chess.
His system has since been tweaked and modified to calculate win probability in games and sports as divergent as basketball, baseball and Scrabble. Auckland University stats guru Wil Undy has used the Elo framework to assess the merits of every Super rugby side since inception in 1996.
When we first embarked on this exercise two years ago, the 2002 Crusaders emerged as the finest Super Rugby side of all time, with a seasonally adjusted Elo rating of 1703, a fraction higher than the all-conquering Graham Henry-mentored blues side of 1997. In fact, the Crusaders (2002, 2005 and 2006) and Blues (1997 and 2003) accounted for the top five sides, their collective dominance not interrupted until the year 2000 Brumbies gatecrashed the Elo party in sixth with a score of 1665.
In the intervening years, however, Undy has modified his formula.
"The major tweak is subtracting points from a team's rating if they underperform in a win.
Previously it would add points no matter what. This penalises the 2002 Crusaders team that had a lot of gritty wins but underperformed relative to expectation," Undy said.
"The way to explain it simply would be if the Crusaders beat the Sunwolves by one point they should lose rating points despite winning the game. Previously they would gain rating points, albeit it next to nothing."
Under the new formula, Undy has used a "scaled blend" that also takes into account that in early seasons, fewer matches were played. The scaled blend is a combination of the mean rating across the season, peak rating during the season, and the end of season rating - scaled for different numbers of games.
To that end, the highest "blended" end-of-season scores for the each of New Zealand's five franchises were:
The highest "blended" end-of-season scores for the each of New Zealand's five franchises were:
The sharp-eyed among you will recognise that the highest rating team there is not the 2002 Crusaders, or even the 2018 Crusaders, but the 2017 Hurricanes. Curiously, that wasn't the edition of the Hurricanes who won the competition; that was a year earlier.
This Hurricanes team finished the regular season winning 12 from 15, thrashing the Brumbies in Canberra in the first round of the playoffs before being tipped out by the Lions in Johannesburg.
"It's quite interesting to see that a lot of teams in the top 20 or so didn't actually win the title - most of them having been beaten by the Crusaders," Undy said. "What's evident is the Crusaders have something special when it comes to winning close games."
The following is the peak Elo for each team, including details of the match that immediately preceded it.
The following is the peak Elo for each team, including details of the match that immediately preceded it.
The Hurricanes that won that quarter-final in Canberra so impressively were peak Super Rugby, according to the Elo ratings. They were also humbled quite impressively a week later, falling out of contention for consecutive titles and a full seven Elo points in the process.
What the Elo ratings demonstrates, is the dominance of New Zealand's franchises. Below are the peak ratings of the remaining franchises, not including those no longer with us (Cheetahs, Force and Kings).
Here are the peak ratings of the remaining franchises, not including those no longer with us (Cheetahs, Force and Kings).
As it stands, the Chiefs have the lowest peak rating of any New Zealand franchise (1633), yet only five overseas franchises have eclipsed that.
