Preseason PFF Ratings are Less Worthwhile than Preseason Games

Some people love Pro Football Focus’s rankings, some people vehemently hate them, though, most of us find some sort of middle ground in their viability. Some things they do are unquestionably more reliable than the average statistic. For example, they track every single play and look at more than standard statistical categories. Some things they do are unquestionably less reliable than the standard statistic, such as their subjective ratings that are given to human error. Yet, that is the basis of what happens during the regular season... Why then are they less reliable in the preseason?

It’s all in the secret sauce:

And by “secret sauce”, I mean, the formula they use to get these ratings on players. If you have never looked into it, they basically grade every play on a + or - 2 rating scale. Regardless of who they line up against. Meaning, a team with crappy CB’s vs. OBJ is probably going to have it’s CB’s roasted for a -2 on a lot of plays, leaving little or no room for individual talent levels. Then they add up all of the snaps the player had in a game together to get a rating. They can extrapolate those snap counts out across the full game in the preseason; however the exacts of what they do are unknown (I know regular season it’s all about snap count because they require a certain number of snaps to get a rating). For all of the subjective qualifiers that PFF takes into account: a well blocked run vs. an exceptional RB performance or a well placed pass vs. a difficult catch for the WR etc... They never take into account the level of talent being played against.

Now, in the context of a 53 man roster with 22-25 starters and about 10-15 rotational players, plus specialists like the immortal John Denney... it’s a relative talent gap from team to team. Such is the parity of the NFL, aka “Any given Sunday”. Now consider how that which is a minimal issue during the regular season being a huge issue when discussing a 90 man roster. About 40% of those guys will be in a new profession or completely unemployed in the next two weeks. Surely the small variance in talent from team to team on a 53 man roster is doubled, possibly tripled with a 90 man roster.

From a statistical and analytical perspective, this skews the process beyond what it is capable of withstanding. Essentially, they leave the mean (talent variance = 0) and then compound the standard deviation from that mean (actual talent level on field).

Too much variance for viability-

With this variance growing larger within each team, it stems to reason that a greater statistical variance would grow between NFL rosters as well. Meaning, while there is parity in the NFL at 53, it’s a literal crap shoot when the players get to 90 per team. Especially when you consider the good- great teams with an established 53 man roster will tend to sit a lot of starters for much of the preseason. Meaning their opponents are unlikely to face the top level talent on the roster. Consider the Patriots and the Packers here, much of Pittsburg’s offense. etc. etc. Often teams that go 1-3 or 0-4 in the preseason end up having great seasons. Why? They sit most of their top talent during preseason in order to evaluate the rest.

Then you have other teams like Miami whom are trying to replace a starting QB, that are now trying to replace a Middle LB. Miami has to put it’s starting units on the field more than they would like to do so, in oder to integrate new players. Some guys, like Jordan Phillips, they need to see a consistent motor from and are making him play more snaps than normal. So while he has starter level talent he is being played against guys who are going to be selling insurance, or bagging groceries in the next few months.

Similar circumstances have been surrounding Cordrea Tankersley. Here is a 3rd round pick in an absolutely loaded CB class, on a team with two penciled in starters and some really solid depth. He came in as CB 5, at best. However, there are a few teams I can name where he would probably be the starter... Buffalo, anyone? Now, Miami has him playing against most of these same guys who are scrapping for a practice squad position. So when PFF comes out and says Tank got the best score last week along with Phillips... I was like... “well no shit, the two guys with more talent compared to their counter parts on the field had the best scores...you don’t say”. If you would have told me when in the game and how many snaps those two were going to take I could have guessed they would have astronomical PFF scores.

Putting it together:

Many feel a dislike for PFF ranking in the regular season, for being too subjective. Others dislike some of the methods of subjectivity. And some hail them as the greatest thing to happen to sports analysis. However, no matter your feelings about their in season viability, it is laid bare by the facts of statistical analysis that their rating system is completely unhinged by the preseason games and all of the circumstantial occurrences that accompany them, if not alone on the foundational fact that the 90 man roster exists. To me, preseason PFF rankings mean as much as: team power rankings, or the final score to a preseason game, or what Joe Philbin is thinking now... which is, nothing at all.

Don’t be afraid to sound off in the comments section, whether you agree or disagree. Most of you know I love a good debate.