Rethinking the Conventional Wisdom of the NFL Draft, Part 1: What Do Bill Polian and Wyatt Earp Have in Common?

Ezra Shaw/Getty Images

Hello, Dolphin brothers and sisters! We still have a head coach to hire, as well as the coordinator positions; we still have the re-signing and releasing of players and free agency. I'm sure I'm forgetting a bunch of things, too. The Dolphins will be busy. I still have laundry to do, I need to re-string my guitar, and I should probably empty the Diaper Genie. So I'll be super busy, too. But there's nothing quite like the NFL Draft, and when the regular season is over, my thoughts instantly drift towards the draft, however premature that may be.

We will hear a lot of "micro" thoughts about the draft (i.e. certain players we like, an orgy of mock drafts, prospects' draft values, etc..) and I look forward to drifting in that vast sea, but we rarely get to step back and look at the NFL Draft in the "macro" sense. This series of FanPosts (I'm thinking there will be 3-5, including this one…unless everyone thinks this FanPost idea sucks and no one REC's it, then I'll just go back to drinking beer and laughing at baby farts) will target exactly that: the big picture of the NFL Draft. We'll challenge assumptions, increase awareness of our own biases, rethink conventional wisdom, look at huge sample sizes that give us statistically relevant data (and use that to turn the NFL Draft upside-down), and use what we've learned to try and implement better strategies for approaching the draft. Without further ado...

What do Bill Polian and Wyatt Earp have in common? They are both incredibly legendary in their respective fields (the former perhaps the best NFL GM in the modern era, and the latter the most accomplished gunslinger of the Old West), but much like Neo in the Matrix, both are necessary statistical anomalies. Let me explain using Warren Buffett's coin-flip analogy: suppose 225 million Americans play in a single elimination head-to-head coin-flipping contest, with one coin flip per day. After 20 days, we would expect 215 people of those 225,000,000 to successfully call their flips 20 out of 20 times. But that doesn't mean those 215 people are any better at calling coins that you or I am. It's often meaningless in hindsight to ask "What are the odds of that?" In other words, in Bill Polian's case, it would have been meaningless in 2003 to ask: "What are the chances Bill Polian is just an average drafter who happened to have a streak of excellent drafts?"

I mean, c'mon, the guy drafted (not in chronological order) Marvin Harrison, Peyton Manning (over Ryan Leaf), Edgerrin James (over Ricky Williams), Tarik Glenn, Reggie Wayne, Rob Morris, Dallas Clark, and Dwight Freeney with 8 consecutive 1st round picks. Is there any wonder that the Colts were a Playoff staple? What's more impressive is that Polian essentially built the team from scratch, and he made at least 2 difficult decisions (Manning and James) that benefitted the organization exponentially, and choosing the alternative (Leaf and Williams) would've likely doomed both him and the franchise.

Here's the thing about we human beings - we recognize coin-flipping as lucky but see drafting successful NFL players as being more highly skilled in NFL Draft talent acquisition than similarly hard-working colleagues. In 2003, if someone were to say Bill Polian was just an average drafter who got lucky, that person surely would have been placed in an ultra-minority of padded-wall looney tunes by his or her peers. I mean, the track record spoke for itself. It would have been the equivalent of calling CT a relentless optimist, or Kevin and James lazy, or Darrell Owen a jerk, or Alpha appropriate.

Something mysterious happened. No, Alpha didn't get a date with a female human being. Yes, you guessed it, Bill Polian S'ed a D for 5 straight years and got fired. But, but, but, wh..wh..wh..what happened was...

From '07 to '11, Bill Polian drafted Anthony Gonzalez, Donald Brown, Jerry Hughes, and Anthony Castonzo - and packaged the 1st rounder in 2008 to select Tony Ugoh. Well…he could have made up for his 1st round whiffs with Day 2 and Day 3 selections - I mean, s**t - teams do it every year. During that same time span, in the 2nd and 3rd rounds, Polian drafted a bunch of beastly studs (heavy sarcasm): Mike Pollak, Pat Angerer, Fili Moala, Ben Ijalana, Dante Hughes, Philip Wheeler (LOL, we know how bad THAT guy sucked), Jerraud Powers, Drake Nevis, Kevin Thomas, and Quinn Pitcock. Almost overnight, he went from draft prodigy to draft dunce.

The reason it would be meaningless to ask in 2003 what the odds were that Bill Polian was truly skilled or JAG is because it's not the real question we want to ask. Polian was excellent in the draft right up until the moment he stopped being excellent and started sucking. We're not interested in seeing success in the past, the real question we want to answer is "How easy is it to identify going forward which teams will be good or bad in the draft?"

Let's use another analogy here - let's think of the draft prospects as commodities in a marketplace. GM's try to make a statement on what a prospect is worth, just like traders to a commodity. Overvalue the player and draft him too early, he will bust; undervalue the player, and another team might take him.

Let's also consider the information available to teams. Aside from interviewing a prospect's 4th-grade teacher about how they responded to peeing their pants during a recital, every team has a tremendous overlap of the same information: physical data from the Combine, game tape, and college stats. Teams differ in how they interpret the data, and teams have different schemes, so they evaluate players differently. But in terms of the information, this is what we call the "efficient-market hypothesis": when markets are informationally efficient (i.e. everyone has access to it, and the information is robust), no individual can consistently achieve returns in excess of the market-wide average over time, adjusted to risk. (This "adjusted to risk" caveat, in NFL Draft terms, means the difference between drafting in the 1st and 7th rounds - there is less risk in drafting a 7th rounder. Each decision should be hedged against the potential risk it involves.)

If teams were truly skilled and could "beat the market", we'd expect to see greater returns from some teams/GM's in comparison to the average team, adjusting for the "risk" (or in our case, the round where the player was drafted). If the EMH holds true, we wouldn't see much relationship between draft performance across multiple years of drafts. Neil Paine at tested this empirically using 20 years worth of drafts from 1994-2013. Here is the skinny on what he found in his regression analysis:


  • Fluctuations happen equally on the upside and downside of the regression, meaning that players routinely play better - and worse - than the long-term averages.
  • Teams can't regularly predict which individuals will outperform or underperform relative to their draft position. There are certainly isolated success stories [Tom Brady] (and on the flip side, horror stories [JaMarcus Russell]), but nothing with consistency.
Chase Stuart from studied 44 years of drafts and found 0.07 correlation coefficient between a team's draft pick success from one year to the next. That's about as weak as it gets, ladies and gentleman. But SUTTON, teams go through GM's like WZB goes through blunts, does the data say anything about individual GM's?He looked at GM's tenure in 3-year segments, and the drafting performance between one 3-year segment to the next is weak at best (correlation of .24).

What have we learned in this incredibly wordy 1st installment of the series? (Thank you to those who made it all the way through). The NFL Draft's means of sorting players is largely an efficient system (meaning, the draft order itself is the strongest indicator of a player's performance - the graph above shows this quite nicely), resulting in none of its individual decision makers having the ability to "beat the market" in the long run. Some see short-term deviations on either side, but they prove unsustainable over large samples. In other words, what a team gets from each pick is determined overwhelmingly by…luck.

No one can beat the draft consistently. NO ONE. Thought of another way - every GM or draft day decision maker, most sooner than later, will get fired for sucking at drafting.

Next installment: Jimmy Johnson's infamous draft day chart: accurate tool or a collective reflection of self-inflated GM ego?

This is a FanPost and does not necessarily reflect the views of The Phinsider's writers or editors. It does reflect the views of this particular fan though, which is as important as the views of The Phinsider writers or editors.