Some of your excellent comments/tweets about my two recent blog posts on Rex Grossman (QB, Washington Redskins) have inspired me to write a post on quarterback ratings.
The standard “QB rating” is a complicated formula that has a lot of drawbacks. It is a constructed scale that attempts to reduce a quarterback’s performances to a single number. According to Wayne Winston,
The NFL’s current system for rating QB’s is a complex. nonsensical formula that combines a QB’s completion percentage, Yard per Pass Attempt, TD Pass Percentage, and Interception Percentage into a single incomprehensible number.
I don’t claim to disagree. However, while the QB rating formula is a mess, it accurately reflects a quarterback’s performance for the most part.
Let’s compare three quarterback rating systems: the QB rating, Wayne Winston’s system, and the wages of win rating. The latter two rating systems are considered to be superior to the QB rating system. Below is the rank for each of the34 QBs in the NFL according to each of these three systems. The quarterbacks are sorted according to the QB rating, since that is the “gold standard” for the casual, non-nerdy fan. I refer you to Wayne Winston’s book Mathletics for the details.
First, note that all three rating systems rank Aaron Rodgers as #1. All three put Eli Manning in the top 6 and Tony Romo in the top 10, two quarterbacks that fans love to complain about.
There is quite a bit of agreement among the ranking systems. There are two notable differences. The first is with Alex Smith, who is raked #9 according to his QB rating. He appears to be overranked: Winston and Wages of win rank him as #17 and #22, respectively. On the other hand, rookie phenom Cam Newton is ranked #19 according to his QB rating (a so-so ranking), whereas Winston and the wages of win rank him #8 and #4, respectively. Many would argue that Cam Newton is underrated, and this would reinforce that claim.
| Quarterback | QB rating rank | Winston rank | Wanges of win rank |
| Aaron Rodgers, QB | 1 | 1 | 1 |
| Tom Brady, QB | 2 | 3 | 3 |
| Drew Brees, QB | 3 | 5 | 2 |
| Eli Manning, QB | 4 | 2 | 6 |
| Matt Schaub, QB | 5 | 4 | 5 |
| Matthew Stafford, QB | 6 | 10 | 8 |
| Ben Roethlisberger, QB | 7 | 6 | 7 |
| Ryan Fitzpatrick, QB | 8 | 13 | 14 |
| Alex Smith, QB | 9 | 17 | 22 |
| Tony Romo, QB | 10 | 7 | 9 |
| Matt Hasselbeck, QB | 11 | 15 | 13 |
| Curtis Painter, QB | 12 | 9 | 26 |
| Michael Vick, QB | 13 | 12 | 12 |
| Andy Dalton, QB | 14 | 21 | 20 |
| Jason Campbell, QB | 15 | 14 | 21 |
| Jay Cutler, QB | 16 | 18 | 10 |
| Mark Sanchez, QB | 17 | 23 | 17 |
| Donovan McNabb, QB | 18 | 22 | 25 |
| Cam Newton, QB | 19 | 8 | 4 |
| Philip Rivers, QB | 20 | 16 | 11 |
| Tarvaris Jackson, QB | 21 | 29 | 30 |
| Matt Cassel, QB | 22 | 27 | 28 |
| Joe Flacco, QB | 23 | 11 | 18 |
| Matt Ryan, QB | 24 | 24 | 16 |
| Chad Henne, QB | 25 | 19 | 27 |
| Kevin Kolb, QB | 26 | 20 | 15 |
| Kyle Orton, QB | 27 | 32 | 31 |
| Colt McCoy, QB | 28 | 28 | 24 |
| Josh Freeman, QB | 29 | 26 | 19 |
| Sam Bradford, QB | 30 | 25 | 23 |
| Blaine Gabbert, QB | 31 | 30 | 32 |
| Matt Moore, QB | 32 | 34 | 33 |
| Rex Grossman, QB | 33 | 31 | 29 |
| Kerry Collins, QB | 34 | 33 | 34 |
Punk Rock Operations Research 
October 24th, 2011 at 4:34 pm
To complicate the picture even further, look at how ESPN’s Total QBR rates QB’s. http://espn.go.com/nfl/story/_/id/7141925/nfl-week-7-total-qbr-season-leaders They put Cam at 14 and Alex at 28, positions that don’t really agree with any of the 3 rating systems you listed. I’m surprised that the rating systems are that different from each other. Now I’m intrigued: thanks for blogging about this.
October 25th, 2011 at 7:50 am
[…] Reading: on quarterback ratings No Comments Filed under: Uncategorized RSS Posted on: October 24th, 2011 Tags: […]
October 27th, 2011 at 12:50 pm
Wayne Winston is incorrect. There are rationale behind the formula used, it’s not that complex, and it does it make sense if one looks into it. The Wikipedia article http://en.wikipedia.org/wiki/Quarterback_rating is helpful.
Here are some useful references I found when looking into this previously:
Pro Football Hall of Fame page: http://www.profootballhof.com/history/release.aspx?release_id=1303
Online calculator: http://www.brucey.net/nflab/statistics/qb_rating.html
“The N.F.L.’s Passer Rating, Arcane and Misunderstood,” Richard Sandomir, New York Times, January 14, 2004
http://query.nytimes.com/gst/fullpage.html?res=940CEFD61330F937A25752C0A9629C8B63
January 2nd, 2012 at 8:44 pm
This is interesting. I think the traditional QB rating formula is perfect with the exception that it should come in variations. Like we should have an overall rating, a short pass rating, a deep pass rating, a non-YAC rating, a 2-min drill rating, etc. All these give us a different analysis of the QB’s game that can be overlooked in the traditional rating. I mean anytime you try to compress the multidimensional array of QB stats into a single number, you’re going to get errors, but the question is can we come up with complimentary sets of rating numbers that minimize this error?
This is something I’ve been playing around with for a few years (I remember defending Jason Campbell and Patrick Ramsey here in DC) and I think that its a very interesting problem, especially because its something that the world (ala people outside of just mathematicians) cares a lot about and hence it provides a useful means for people to stop hating mathematics.