MMA Ranking Talk

Intro, accept it

One of the biggest issues with ranking MMA fighters is a mistaken belief that certain fighters are unbeatable. People think that the #1 guy never loses, and when he does they think the guy that beat him is now Captain Indestructible. History, statistics, and common sense would all beg to differ though.

For those that understand MMA is a sport, and even the best can be upset I’d like to move on and talk about expected win percentage (it is what I titled the rant after all). When I say expected win percentage I’m talking about the overall percent of the time Fighter A is expected to beat Fighter B.

For example, if I say Fighter A has a 60% expected win percentage against fighter B that means if they were to fight 10 times, Fighter A would be expected to win 6 of them. It’s certainly possible that Fighter A wins more than 6 times, or even loses every fight. However, the stat is telling us Fighter A should win 6(ish) out of 10.

This concept of expected win percentage is very important on the site because it’s the basis for a lot of comparisons. Under the covers the site looks at Fighter A and their rating, then looks at Fighter B and their rating. From those numbers it makes a determination about how often each fighter should win. Based upon that calculation and the actual fight outcome it then adjusts both fighters ratings accordingly.

History, learn from it

One of the things I try to do with the site is provide a level of transparency that rivals the partial tax returns offered by presidential candidates. One way I do this is via the Site Stats page. This page has a table that shows the results for fights across various rating gaps. For instance, if we look at the page quick (it’s ok to click it real fast, there aren’t any nasty roll over ads that are going to bite you) we’ll see that when fighters are within 70-80 rating of each other the underdog wins 40% of the time. That is to say that 6 out of 10 times the fighter rated 70 to 80 points higher than their opponent wins the fight.

Dance or no dance

Much like how some people have problems believing their favorite fighter can’t be beaten, many people have a problem understanding how the site works. They see the 60% number from the 70-80 rating difference and think the results are poor. What people need to realize is that the comparison that should be made is between how often the system thinks a fighter of a certain rating should win and how often that fighter actually does win.

There are a few ways I could illustrate this: a song, interpretive dance, or a bland HTML table. For now I’ll post the table, but keep your eyes out on youtube for “Expected Win Percentage by Evil Pooh and the Pigtails”.

What the above table shows is the difference between the win percentage the site system expects and the actual win percentage achieved by fighters. For example, looking at fights where the fighters had a rating gap of 195-205 points the site was off by 3.42%. As a simple example, assume the site expected the higher ranked fighter to win 66% of the time, in actuality the higher rated fighter won 69.42% of the time.

Looking at the above it does appear that the system is a bit conservative in that historically the higher rated fighter has won a bit more than expected. However, the gap is only a few percentage points in most cases.

What have you done for me lately

The above table showed the results for all fights ever, so lets take a look at a more recent sample. This table will only show the results of fights that occurred since 1/1/2006.

Again we see that for most rating ranges the results are within a few percentage points even for the most recent fights. The one exception is at 295-305, but due to the much smaller sample size it’s not as out of place as it first appears. Additionaly, it again errs on the side of the favorite winning.

And I read this why?

For a couple reason actually. Here’s what you should take away from this little rant:

- Fighters lose, even the best of the best of the best SIR~!!! Don’t think a guy is over the hill because of one loss and don’t think a fighter scoring one huge win makes them Captain Bignasty.

- Numbers need context. If the site shows Fighter A has a rating of 1802 and Fighter B has a rating of 1804 that does *NOT* mean that Fighter A has no chance in hell of winning so much as a round from Fighter B. It does not mean the site is claiming Fighter B will win every fight the two ever have. What it means is that the fighters are very close, but historically the higher rated fighter has won slightly more often.

- The site is running very close to where it should be from a statistical point of view. Don’t look at the 60% number and assume the site got 40% wrong. The truth is the site is actually getting a few too many fights right atm. *shrugs* What a terrible problem to have. 8P

To infinite and beyond

So where do we go from here? I’ve got a few tests to run on the system itself to lower the expected win percentage gap a bit. I’ve also thought about potentially adding the historical and maybe even the expected win percentage numbers to the Fighter Compare section of the site. That might help some people better understand just how closely matched some of these fighters are.

Should I add the win percentages to the fighter_compare? Are there any other site metrics you’d like to see? I had some from 2007 (year of the upset) which overall was about as expected. Just remember that lowering the sample size really skews the results. Don’t expect to see metrics for April of 2003 (although, yeah, I could provide them lol).

One term I see people throwing around more and more is “strength of schedule”. What this term refers to is the level of competition faced by a given fighter. Like all statistics though it’s important to understand exactly what the number represents. My hope is that this rant will help you understand the different ways strength of schedule can be calculated and help you better understand the pros/cons of each method.

One important note to make is that when talking about strength of schedule we always refer to the fighter’s competition at the time of the fight. Some people like to look at an opponent’s current standing when calculating strength of schedule for previous fights and that’s a big mistake.

Fighter A shouldn’t get the same amount of credit for beating Fighter B in Fighter B’s first professional fight as Fighter C does if they beat Fighter B 5 years down the road after Fighter B has dramatically improved and become champion. Additionally, beating a fighter in their prime shouldn’t be diminished at all because that fighter chose to keep fighting past their prime and tanked their record/rating. An entire rant could be dedicated to this point, but I’ll leave it here for now. If you have any questions though I can provide examples, charts, and graphs. 8)

Method One

One of the most common ways to calculate a fighters strength of schedule is to look at the average rating of all of their opponents.

Top 30(ish) with this method are:

1) Pawel Nastula – 1962 Fights – 4
2) Marcelo Pereira – 1960 Fights – 1
3) Tatsuya Iwasaki – 1959 Fights – 1
4) Istvan Majoros – 1932 Fights – 1
5) Aleksei Medvedev – 1919 Fights – 1
6) Hong Man Choi – 1919 Fights – 2
7) Vepcho Bardanashvili – 1917 Fights – 1
8) Mika Nagano – 1910 Fights – 1
9) Ryan Sturdy – 1898 Fights – 1
10) Bu Kyung Jung – 1893 Fights – 3
11) Jason Peck – 1891 Fights – 1
12) Albert Basconcelles – 1891 Fights – 1
13) Mitsuhiro Matsunaga – 1880 Fights – 1
14) Cesar Gracie – 1871 Fights – 1
15) Cami Hostetler – 1871 Fights – 1
16) Mark Hunt – 1867 Fights – 9
17) Nate Sauer – 1865 Fights – 1
18) Ray McDaniel – 1863 Fights – 1
19) Sergey Ignatov – 1862 Fights – 1
20) Jacob Zobnin – 1862 Fights – 1
21) Maynard Marcum – 1862 Fights – 2
22) Robert Stines – 1860 Fights – 1
23) Pang Ji Won – 1854 Fights – 1
24) Mike Cizek – 1851 Fights – 1
25) Aaron Williams – 1851 Fights – 1
26) Doug Edwards – 1850 Fights – 1

CONS:
As you can see from the list it’s very easy for a fighter to only have one or two fights against a highly rated opponent and secure themselves a very high strength of schedule rating.

Looking closer at the above list we also notice that most of the fighters listed above actually *LOST* their fights. This is because we are only looking at the strength of their opponents and not whether they actually managed to defeat them or not.

PROS:
The benefits of the above approach is that we get a career wide picture of the average opposition faced by a fighter. Having losses against highly ranked fighters can sometimes help explain why a fighter has a lower personal rating. Something along the lines of “Wow Frank has a pretty low rating, but take a look at the amazing fighters that have kicked his ass~!!!”

Method Two

Another method for looking at strength of schedule removes losses from the equation and only looks at the strength of opposition a fighter has actually defeated.

Top 30(ish) with this method are:

1) Georges St. Pierre – 1829
2) Mark Hunt – 1805
3) Chuck Liddell – 1803
4) B.J. Penn – 1802
5) Rameau Thierry Sokoudjou – 1801
6) Randy Couture – 1785
7) Andrei Arlovski – 1779
8) Josh Barnett – 1773
9) Mirko Filipovic – 1771
10) Anderson Silva – 1770
11) Antonio Rodrigo Nogueira – 1754
12) Ricardo Arona – 1752
13) Mauricio Rua – 1751
14) Fedor Emelianenko – 1745
15) Kenny Florian – 1741
16) Tito Ortiz – 1735
17) Paulo Filho – 1734
18) Dan Henderson – 1734
19) Fabricio Werdum – 1733
20) Demian Maia – 1733
21) Josh Koscheck – 1731
22) Shinya Aoki – 1728
23) Thiago Alves – 1726
24) Antonio Rogerio Nogueira – 1723
25) Ryoto Machida – 1720
26) Joachim Hansen – 1719
27) Rashad Evans – 1713
28) Patrick Cote – 1710
29) Jon Fitch – 1709
30) Takanori Gomi – 1708
31) Tyson Griffin – 1707
32) Gilbert Melendez – 1702
33) Aleksander Emelianenko – 1700

CONS:
One of the problems with this approach is that it can reward fighters with very few fights/wins. For example, Mark Hunt has only 9 career MMA fights. Of those 9 fights, only 5 are wins, and two of those were against very highly ranked opponents.

Additionally, fighters that have been around for awhile (ex. Matt Hughes, Wanderlei Silva) are penalized to a degree because not every single one of their opponents was as highly ranked as they are. As such their strength of schedule rating is relatively low despite the fact that they have faced some of the highest ranked fighters ever.

PROS:
One of the main benefits of this approach is that it prevents losses from factoring in. This means a fighter can’t improve their strength of schedule rating dramatically by being stomped out like a brush fire by a highly ranked opponent.

Additionally, this approach shows career long average which lets people see who has consistently faced very difficult competition their entire career.

Method Three

The final method I’ll mention also only looks at a fighter’s wins. However, unlike the last method it only looks at their biggest wins. Instead of calculating strength of schedule for all of the opponents the fighter has beaten it instead looks at only the rating of the ten highest rated fighters they have defeated.

Top 30(ish) with this method are:

1) Fedor Emelianenko – 1966
2) Anderson Silva – 1952
3) Georges St. Pierre – 1948
4) Chuck Liddell – 1929
5) Mirko Filipovic – 1913
6) Quinton Jackson – 1913
7) Antonio Rodrigo Nogueira – 1910
8) Josh Barnett – 1896
9) Wanderlei Silva – 1894
10) Randy Couture – 1891
11) B.J. Penn – 1871
12) Rich Franklin – 1865
13) Mauricio Rua – 1863
14) Shinya Aoki – 1861
15) Takanori Gomi – 1856
16) Matt Hughes – 1852
17) Dan Henderson – 1852
18) Andrei Arlovski – 1850
19) Renato Sobral – 1844
20) Thiago Alves – 1837
21) Tim Sylvia – 1836
22) Joachim Hansen – 1828
23) Alistair Overeem – 1820
24) Paulo Filho – 1819
25) Jon Fitch – 1817
26) Tito Ortiz – 1816
27) Nathan Marquardt – 1813
28) Yushin Okami – 1810
29) Ricardo Arona – 1809
30) Karo Parisyan – 1807

CONS:
The real drawback to this approach is that you greatly limit the sample size. Some fighters on this list have 30+ professional fights and this stat only looks at a handful of them. When talking about a “career strength of schedule” this number can be very misleading.

Another potential drawback is that the above stat might not change very much or very often for some fighters. For example, both #5(Cro Cop) and #26(Tito Ortiz) from the above list haven’t had a win against a highly ranked opponent in almost two years. Both of them likely have numerous fights left before they retire, but if neither of them was to get another big win then the above rating wouldn’t change at all.

PROS:
As with method two the above numbers only count wins. Another benefit to this method is that fighters aren’t penalized for having a few weaker fights mixed in with numerous difficult ones.

Almost out of soda

Since I’m almost out of soda I’m going to wrap this up now. I know some of you are wondering which method is “best” and the answer (as usual) is that it depends.

Some people when they talk strength of schedule want to know the person who fight after fight for their whole career has faced the strongest opponents. For those people Method 1 is likely best. However, it’s important to keep in mind fighters with very few fights and fighters with losses against top competition padding their rating.

Other people will want to know who has beaten the best fighters time after time. (if you fall I will catch you I’LL BE WAITING~!!!) For those people Method 2 will likely yield results similar to what they are expecting. It’s important to keep in mind though that this approach favors fighters with fewer fights and can penalize fighters who have been competing for awhile.

Finally, if you are looking for that “big game hunter” type of number. That fighter that has stepped into the ring with some of the toughest fighters ever and walked out victorious, then the results from the third method will be more to your liking. Just realize that your results might be a bit dated in the sense that John “Dragonslayer” Doe might not have actually done much slaying of late.

As usual it’s not about “right” and “wrong” it’s about “accurate” and “inaccurate”. When speaking with other people about strength of schedule be sure to qualify your criteria. Additionally, if someone tries to tell you that Fighter A has had a very difficult strength of schedule ask them their criteria. It’s ok if their criteria and yours don’t jive. The only problem comes when their criteria doesn’t match what they are claiming. If that’s the case then send them here and I’ll have my charts and graphs ready. 8)