Rethinking the Conventional Wisdom of the NFL Draft, Part 2: Is Jimmy Johnson's Draft Day Chart Still Relevant?

Talk about being stiff, look at that hair. My goodness. Blustery conditions? Mwahahaha, pfffft, whatever. Those silver threads could endure winter in Antarctica. Jimmy Johnson's got more hair product hidden away in his panic room than most of us will ever use in our life. But back in the 1980's as Head Coach of the Dallas Cowboys, he did something slightly more useful than inspire Derek Zoolander's roommate Meekus to revere styling gel. Some described it as revolutionary at the time and it has been a foundation for all NFL Draft trade talks since its inception: he created a chart that ascribed point values to each of the picks in the NFL Draft, from #1 to Mr. Irrelevant. Jimmy Johnson's value chart (courtesy of The Harvard Sports Analysis Collective):

Let's first off talk of the irony of draft day swaps by taking a look at 2 specific examples. In 2014, the Miami Dolphins traded down twice and landed this guy named Jarvis Landry. Prima facie, it doesn't really matter if the points add up from the chart - we traded down TWICE and got the Juice. We traded with San Diego from the 50th spot (400 points) to the 57th (330 points) and got an extra 5th rounder, #125 (47 points). We got shafted by 23 points. We then dealt down again with San Francisco, going from #57 (330 points) down to #63 (276) and received an extra 5th rounder, #171 (24 points). We got shafted by 30 points, leaving with a grand total of 53 negative points relative to the deals we made. Again, despite "losing" both trades, we still landed Jarvis Landry. The fact that we picked up 2 picks while drafting an important member of our nucleus is a draft day win.

Let's look at a deal where we were actually the benefactors…kind of. In 2013, we traded up from #12 (1200 points) to #3 (2200 points) to select the most accomplished and highly skilled drug user we've had since Ricky Williams: Dion Jordan. In order to make that move, we threw in our 2nd round pick, #42 (480 points). We shafted the Raiders by 520 points and yet still totally blew it. Draft trades are highly volatile in how they work out because it deals with the inherent volatility of the draftees themselves.

So how valuable is the chart in its ability to accurately reflect draftees' worth? Well, it wasn't that valuable to begin with, even at its very core - I mean, look at the intervals between #3 and #4 (400 points) and #4 and #5 (100 points). The numbers seem arbitrary at best, and simply do not account for the weighted intervals one would expect going from pick to pick. And yet, research done by Cade Massey and Richard Thaler (found in the July 2013 edition of Management Science) studied the 1,078 draft day trades from 1983-2008 and found that the picks correspond eerily well with JJ's chart when looked at as a whole, with obvious variance in specific cases on either side (meaning, on a case-by-case basis, some teams "won" the deal and some "lost" the deal, but over time, the sum total of the trades correspond well with "the Chart."). How do we make sense of this? How do such smart, hard-working people follow something that appears iffy just looking at it? Jimmy Johnson's chart is more a measure of GM ego than GM skill in identifying draftees' worth. What do I mean by that?

Remember how in the 1st installment of the series we showed how a team's draft return is mostly luck? Well, GM's have it in their head that it's skill, something Massey and Thaler called "The Loser's Curse." The chart from the 1st installment demonstrated quite nicely that the NFL Draft position is an efficient market, meaning that draft order was the biggest predictor of draftee success. However, it's mistakenly seen as picking skill, which creates the narrative that earlier picks will turn in to franchise-changers and are thus worth exponentially more points than, say, the #20 pick. Overconfidence blinds them in seeing the volatility in determining the commodities' potential worth. Remember one of the mantras from Part 1 of this series: overvalue the commodity, and he will bust (or at the very least, not return the gains that you expected).

Think about how much time a GM and their respective scouting department spends dissecting players. It creates a false sense of security that they can differentiate among the similarly-rated "elite" players with accuracy by virtue of the investment of time. In other words, look at how much time I've spent on this! There's no way in Hell I'm drafting Ryan Leaf, or Jamarcus Russell, or Tim Couch, or Dion Jordan, or Vernon Gholston, or Akili Smith, or Courtney Brown, or Steve Emtman, or Rick Mirer, or Aaron Curry (supposedly the "safest player" to draft in 2009). I've worked too hard to make THAT mistake! Until you realize that it's EXACTLY what has happened to NFL front offices in the past. Quite a vicious circle.

Let's see a visual, demonstrating the regression we saw in Part 1 and compare it to "the Chart":

You'll notice two things: 1) JJ's chart ridiculously overvalues the top picks in the draft, then sloping significantly somewhere between picks 10 and 20, and 2) JJ's chart moderately undervalues the mid-round picks, starting at about pick #70. The implication of JJ's chart is that the difference in draftee caliber after the high picks is astronomical. If teams are following this chart, even "with a grain of salt", it suggests they are putting way too high a price on the earliest picks - and considering the Grand Canyon between the two regressions, one can only deduce that NFL front offices like looking in the mirror and telling themselves how pretty they are.

According to Jason Lisk of Pro-Football-Reference, teams with top 5 picks correctly pinpoint who goes on to have the best career 10.3% of the time. Thought of another way: the most dangerous thing in the NFL Draft is to trade away Day 2 picks in an effort to control their own fate with one of the top picks. It certainly doesn't help to have JJ's way-too-revered chart dangling in front of them like the proverbial carrot in front of the donkey as justification.

We've seen ego affect the Miami Dolphins as well, and I'm not thinking of the trade up for Dion Jordan - the particular example I'm thinking of doesn't involve the NFL Draft but rather free agency. In a "make or break year" for Jeff Ireland, he used our healthy cap space to sign the free agency darling: Mike Wallace. I was against signing any of the "Big 3" that year: Mike Wallace, Greg Jennings, and Dwayne Bowe. (Proof.) But we all knew he'd sign one of them, the writing was on the wall. The conditions were not ripe to remove Ireland's ego from the equation, as he was in self-preservation mode, and unfortunately, we spent lots of money on a player we'd trade away almost 2 years later to the day.

If we gather anything from this enterprise, it would be these 2 mantras: 1) ego is the enemy of long-term growth, and 2) risk management is paramount in the NFL Draft. Psychology is possibly the best modern way at trying to beat the market, namely being aware of our own cognitive biases. In short, understanding our weaknesses is a strength.

Next installment: In the information and social media age, and the amount of independent scouts and draft coverage, could we intelligently sift through others'  information and reduce our scouting budget?