HAPPY DRAFT DAY
How NFL teams ignore basic economics and draft players irrationally
VOX.COM
The NFL draft begins on Thursday. In the football world, it is a huge deal.
For non-football fans: this is when NFL teams select college football players to add to their rosters, the new players awkwardly hold up jerseys with the number one on them, and football fans everywhere freak out. The worst teams are given the highest picks, and if they want to improve, they need to use them as shrewdly as possible.
Teams that succeed — like the Seattle Seahawks, winners of this past season's Super Bowl — tend to build their rosters heavily through the draft, and merely supplement their drafted talent through trades and free agency. NFL executives invest huge amounts of time and resources into scouting players and ranking them, hoping to draft stars in the first round and solid contributors later on.
But here's the thing: despite years of data, most NFL teams still have no idea how to work the draft most effectively.
It's not their imperfect player evaluation, but something more basic — their refusal to follow the principle of risk diversification. That's the conclusion economists Cade Massey and Richard Thaler came to after analyzing fifteen years of draft data in a series of papers — and it's still true, despite recent changes to the wages rookies are paid.
Draft picks can be traded, and the success of any one player picked is highly uncertain. Because of that, their data says that in the current trade market, teams are always better off trading down — that is, trading one high pick for multiple lower ones — but many teams become overconfident in their evaluation of one particular player and do the exact opposite: package several low picks for the right to take one player very early.
"There are one or two teams out there that philosophically follow this idea," says Massey, who serves as a draft consultant with several NFL teams that he can't disclose. "But in my experience, teams always say they're on board with it in January. Then when April rolls around, and they've been preparing for the draft for a long time, they fall in love with players, get more and more confident in their analysis, and fall back into the same patterns."
To calculate how much teams pay to trade up, they looked at 1,078 trades made between 1990 and 2008. The trades work something like this: a team will trade the 16th and 32nd overall picks, say, for the 5th overall pick. They might also include picks from next year's draft.
Massey and Thaler graphed all trades, which let them assign a relative value to each pick in the first five rounds of the draft:
The most important thing about this graph: the curve is very, very sharp in the first round (picks 1 through 32). That means teams think the very top picks are extremely valuable: the value of the 10th pick is only about half that of the first pick.
Now, it's worth pointing out that for years, most teams followed something called "The Chart," which assigned theoretically fair point values to each pick in the draft for trade purposes. Since 2008, many teams have smartly stopped treating "The Chart" as gospel, and the curve has become slightly less steep.
But Massey says it still hasn't flattened out to anything near where it should be, in terms of the actual value derived from the players picked.
He and Thaler figured this out by calculating the odds that the first player picked at any given position will perform better — in terms of the number of games he starts in his first five seasons — than the second player drafted at that position. This is relevant because a team will often trade up when they identify a player they prefer at a needed position: they need a wide receiver, and a few highly-rated ones are available, but they trade up because they're certain one is much better.
But the data says that teams just aren't very good at figuring out when this is true. On average, the chance that first player will start more games than the second one picked at his position: 52 percent. Compared to the third, it's still only 55 percent, and compared to the fourth, it's merely 56 percent.
These numbers suggest that moving up eight picks (the average distance between the first and second players at the same position) should cost a small amount, since you're only increasing the odds of a getting a more productive player by four percent or so. But as the steep curve shows, teams pay a ton to move up, especially at the top of the draft.
"It's basically a coin flip," Massey says, "but teams are paying a great deal for the right to call which side of the coin."
One recent trade epitomizes this point — and shows that even in the post-"Chart" era, teams still overvalue the highest picks. In 2012, the Washington football team traded the 6th pick, the 39th pick, and their first round picks in 2013 and 2014 to the St. Louis Rams so they could move up four spots and take Robert Griffin III — a player they were certain was a once-in-a-lifetime quarterback — with the second overall pick.
Just two years later, the trade already looks pretty shortsighted. That's not because Griffin is a bad player (he's almost one of the few that might make a crazy trade like this seem worthwhile), but because the odds of getting a great player with one high pick are so much lower than getting one with four high picks.
The team would've been better served sitting tight and just drafting the next-best quarterback (Ryan Tannehill) and using the other three picks on much-needed players. Or they could have waited and drafted a quarterback this year — say, Johnny Manziel — when they would've had the 2nd overall pick themselves if they hadn't made the trade.
For each pick in the first round, they calculated all of the different two-pick packages a team could've gotten by trading down, based on the historical data (a team with the first pick, for instance, could get the 2nd and 181st picks, or the 14th and 15th picks, or any combination of picks in between that provide the same sum value).
Then they calculated what teams get out of these picks on average, in terms of the number of starts a player provides in his first five years and the number of Pro Bowls he's voted to. (They included Pro Bowls to counter the criticism that their analysis ignores the unique impact of superstar players solely available in the first few picks.)
Again, the data was unequivocal. On average, trading down and getting two players gave a team five more starts per season and slightly more total Pro Bowls.
You could chalk this up to the simple fact that more players start more games, but it's more than that. Even if you imagined that the team trading down could only keep the better one of the two players it drafted, it'd still get slightly more total starts and the same number of Pro Bowls. The truth is that teams are imperfect talent evaluators, so having two later picks is better than a single early one. Risk diversification at work.
For every single spot in the draft, the researchers compared the amount of salary paid to a player with the value a team gets from him on the field. (For this analysis, they used a more sophisticated metric for value: they looked at the average statistical production yielded by a player taken at that spot of the draft between 1994 and 2008, then converted that into dollars, based on what teams paid a player with that level of production on the free agent market on average).
In the graph below, the red line is the actual salary paid to a player picked at each spot, the green line is the average performance provided, and the blue is the difference — the money you save drafting someone at that spot instead of having to pay more for a veteran free agent who'd play at a similar level.
Massey and Thaler 2012
The blue line peaks around pick 33, the start of the second round — making it the sweet spot in the draft. If you had to make one pick, this is where you'd get the most bang for your buck, because there's a pretty good chance of getting a productive player and he won't command a particularly high salary. But since you can get multiple picks by trading a single high one, maximizing value means moving down into round two (picks 33 through 64).
One caveat is that the salary data comes from picks made under the old NFL collective bargaining agreement, which paid players picked in the top 10 considerably higher salaries then they get currently. But the salaries for picks from 10 onward have barely changed — and because trading down is so much more fruitful, Massey says the argument still applies for the first round as a whole.
Massey and Thaler came to this conclusion by looking at the number of wins a team had in any given season between 1997 and 2008, and the total value of all picks they'd made in the previous four years (the amount of time, on average, a rookie is under contract for).
They found that one standard deviation in pick value translated to 1.5 more wins per season on the field. Sure, it's a small sample size, and there's a lot of chance and other factors built into the system — a coach's strategy, for instance — but trading down correlates with a significant amount of victories, given that there are only 16 games in a season.
Why hasn't this happened? One answer is a widely-known psychological bias called the overconfidence effect. As people are given more information, the accuracy of their analysis often hits a ceiling, but their confidence in it continues to increase.
This tendency has been demonstrated in all sorts of areas, from bettors picking horses to psychologists making diagnoses. It's not hard to imagine that NFL general managers — who are given scouting reports on players that cover everything from their body fat percentage to their home life — fall victim to the same sort of overconfidence and, as Massey said, "fall in love" with certain players.
There's also the fact that the sports world as a whole tends to glamorize superstars — leading many to disproportionately attribute a 53-player roster's success to one or two highly drafted players. For a struggling GM, it might seem much easier to trade up and land a guaranteed superstar than patiently fill a roster with competent players.
The problem, though, is that there are no guaranteed superstars — and Thaler and Massey have found that, given a long enough timeframe, no teams are any better at accurately evaluating prospects than others. Sure, a GM might hit a hot streak over the course of a few drafts, but long-term, they estimate that 95 to 100 percent of the difference in teams' odds of striking gold with any one pick is driven by chance.
So the key isn't drafting better — it's just drafting more.
As Cassey noted, there are a few teams out there following his philosophy. In a recent interview, Eric DeCosta — assistant GM of the perennially-successful Baltimore Ravens — dropped a hint about the identity of one of them:
For non-football fans: this is when NFL teams select college football players to add to their rosters, the new players awkwardly hold up jerseys with the number one on them, and football fans everywhere freak out. The worst teams are given the highest picks, and if they want to improve, they need to use them as shrewdly as possible.
Teams that succeed — like the Seattle Seahawks, winners of this past season's Super Bowl — tend to build their rosters heavily through the draft, and merely supplement their drafted talent through trades and free agency. NFL executives invest huge amounts of time and resources into scouting players and ranking them, hoping to draft stars in the first round and solid contributors later on.
But here's the thing: despite years of data, most NFL teams still have no idea how to work the draft most effectively.
It's not their imperfect player evaluation, but something more basic — their refusal to follow the principle of risk diversification. That's the conclusion economists Cade Massey and Richard Thaler came to after analyzing fifteen years of draft data in a series of papers — and it's still true, despite recent changes to the wages rookies are paid.
Draft picks can be traded, and the success of any one player picked is highly uncertain. Because of that, their data says that in the current trade market, teams are always better off trading down — that is, trading one high pick for multiple lower ones — but many teams become overconfident in their evaluation of one particular player and do the exact opposite: package several low picks for the right to take one player very early.
"There are one or two teams out there that philosophically follow this idea," says Massey, who serves as a draft consultant with several NFL teams that he can't disclose. "But in my experience, teams always say they're on board with it in January. Then when April rolls around, and they've been preparing for the draft for a long time, they fall in love with players, get more and more confident in their analysis, and fall back into the same patterns."
A babyfaced Eli Manning: not worth what the Giants gave up to get the first pick. Photo by Chris Trotman/Getty Images
1) It's just not worth it to trade up
The first analysis Massey and Thaler did was to compare what teams are willing to pay to pick higher (in terms of extra draft picks packaged as part of a trade) with the value they ultimately get out of doing so, in terms of player production.To calculate how much teams pay to trade up, they looked at 1,078 trades made between 1990 and 2008. The trades work something like this: a team will trade the 16th and 32nd overall picks, say, for the 5th overall pick. They might also include picks from next year's draft.
Massey and Thaler graphed all trades, which let them assign a relative value to each pick in the first five rounds of the draft:
The most important thing about this graph: the curve is very, very sharp in the first round (picks 1 through 32). That means teams think the very top picks are extremely valuable: the value of the 10th pick is only about half that of the first pick.
Now, it's worth pointing out that for years, most teams followed something called "The Chart," which assigned theoretically fair point values to each pick in the draft for trade purposes. Since 2008, many teams have smartly stopped treating "The Chart" as gospel, and the curve has become slightly less steep.
But Massey says it still hasn't flattened out to anything near where it should be, in terms of the actual value derived from the players picked.
He and Thaler figured this out by calculating the odds that the first player picked at any given position will perform better — in terms of the number of games he starts in his first five seasons — than the second player drafted at that position. This is relevant because a team will often trade up when they identify a player they prefer at a needed position: they need a wide receiver, and a few highly-rated ones are available, but they trade up because they're certain one is much better.
But the data says that teams just aren't very good at figuring out when this is true. On average, the chance that first player will start more games than the second one picked at his position: 52 percent. Compared to the third, it's still only 55 percent, and compared to the fourth, it's merely 56 percent.
These numbers suggest that moving up eight picks (the average distance between the first and second players at the same position) should cost a small amount, since you're only increasing the odds of a getting a more productive player by four percent or so. But as the steep curve shows, teams pay a ton to move up, especially at the top of the draft.
"It's basically a coin flip," Massey says, "but teams are paying a great deal for the right to call which side of the coin."
One recent trade epitomizes this point — and shows that even in the post-"Chart" era, teams still overvalue the highest picks. In 2012, the Washington football team traded the 6th pick, the 39th pick, and their first round picks in 2013 and 2014 to the St. Louis Rams so they could move up four spots and take Robert Griffin III — a player they were certain was a once-in-a-lifetime quarterback — with the second overall pick.
Just two years later, the trade already looks pretty shortsighted. That's not because Griffin is a bad player (he's almost one of the few that might make a crazy trade like this seem worthwhile), but because the odds of getting a great player with one high pick are so much lower than getting one with four high picks.
The team would've been better served sitting tight and just drafting the next-best quarterback (Ryan Tannehill) and using the other three picks on much-needed players. Or they could have waited and drafted a quarterback this year — say, Johnny Manziel — when they would've had the 2nd overall pick themselves if they hadn't made the trade.
2) It really pays off to trade down
Philip Rivers: just part of the valuable package the Giants gave up to take Eli Manning. Photo by Christian Petersen/Getty Images
Given that teams, on the whole, are irrationally willing to pay a lot to trade up, smart teams can reap huge benefits by trading down. Even staying put and drafting from your original spot, the researchers' analysis shows, is not a good strategy. For each pick in the first round, they calculated all of the different two-pick packages a team could've gotten by trading down, based on the historical data (a team with the first pick, for instance, could get the 2nd and 181st picks, or the 14th and 15th picks, or any combination of picks in between that provide the same sum value).
Then they calculated what teams get out of these picks on average, in terms of the number of starts a player provides in his first five years and the number of Pro Bowls he's voted to. (They included Pro Bowls to counter the criticism that their analysis ignores the unique impact of superstar players solely available in the first few picks.)
Again, the data was unequivocal. On average, trading down and getting two players gave a team five more starts per season and slightly more total Pro Bowls.
You could chalk this up to the simple fact that more players start more games, but it's more than that. Even if you imagined that the team trading down could only keep the better one of the two players it drafted, it'd still get slightly more total starts and the same number of Pro Bowls. The truth is that teams are imperfect talent evaluators, so having two later picks is better than a single early one. Risk diversification at work.
3) Players picked lower are cheaper
All of these reasons to trade down don't even include the fact that players picked later in the draft get paid lower salaries, because the NFL collective bargaining agreement sets wages for the first few years of players' careers based on their draft slot. Saving money matters because all teams have to abide by a hard salary cap — this year, it's $133 million — so paying less to draft picks means more is left over for veteran players.For every single spot in the draft, the researchers compared the amount of salary paid to a player with the value a team gets from him on the field. (For this analysis, they used a more sophisticated metric for value: they looked at the average statistical production yielded by a player taken at that spot of the draft between 1994 and 2008, then converted that into dollars, based on what teams paid a player with that level of production on the free agent market on average).
In the graph below, the red line is the actual salary paid to a player picked at each spot, the green line is the average performance provided, and the blue is the difference — the money you save drafting someone at that spot instead of having to pay more for a veteran free agent who'd play at a similar level.
Massey and Thaler 2012The blue line peaks around pick 33, the start of the second round — making it the sweet spot in the draft. If you had to make one pick, this is where you'd get the most bang for your buck, because there's a pretty good chance of getting a productive player and he won't command a particularly high salary. But since you can get multiple picks by trading a single high one, maximizing value means moving down into round two (picks 33 through 64).
One caveat is that the salary data comes from picks made under the old NFL collective bargaining agreement, which paid players picked in the top 10 considerably higher salaries then they get currently. But the salaries for picks from 10 onward have barely changed — and because trading down is so much more fruitful, Massey says the argument still applies for the first round as a whole.
4) Teams that trade down more often win more games
The most straightforward piece of proof for all this analysis is the fact that trading down and amassing more pick value — in terms of the blue line in the graph above — correlates with more wins on the field.Massey and Thaler came to this conclusion by looking at the number of wins a team had in any given season between 1997 and 2008, and the total value of all picks they'd made in the previous four years (the amount of time, on average, a rookie is under contract for).
They found that one standard deviation in pick value translated to 1.5 more wins per season on the field. Sure, it's a small sample size, and there's a lot of chance and other factors built into the system — a coach's strategy, for instance — but trading down correlates with a significant amount of victories, given that there are only 16 games in a season.
So why don't more NFL teams follow this advice?
Baltimore Ravens GM Ozzie Newsome knows how to draft. Photo by Rob Tringali/SportsChrome/Getty Images
If all teams took note of these findings and corrected their behavior, the principles would no longer apply. Teams would be much less interested in trading up, so the lucrative market for trading down would evaporate. Why hasn't this happened? One answer is a widely-known psychological bias called the overconfidence effect. As people are given more information, the accuracy of their analysis often hits a ceiling, but their confidence in it continues to increase.
This tendency has been demonstrated in all sorts of areas, from bettors picking horses to psychologists making diagnoses. It's not hard to imagine that NFL general managers — who are given scouting reports on players that cover everything from their body fat percentage to their home life — fall victim to the same sort of overconfidence and, as Massey said, "fall in love" with certain players.
There's also the fact that the sports world as a whole tends to glamorize superstars — leading many to disproportionately attribute a 53-player roster's success to one or two highly drafted players. For a struggling GM, it might seem much easier to trade up and land a guaranteed superstar than patiently fill a roster with competent players.
The problem, though, is that there are no guaranteed superstars — and Thaler and Massey have found that, given a long enough timeframe, no teams are any better at accurately evaluating prospects than others. Sure, a GM might hit a hot streak over the course of a few drafts, but long-term, they estimate that 95 to 100 percent of the difference in teams' odds of striking gold with any one pick is driven by chance.
So the key isn't drafting better — it's just drafting more.
As Cassey noted, there are a few teams out there following his philosophy. In a recent interview, Eric DeCosta — assistant GM of the perennially-successful Baltimore Ravens — dropped a hint about the identity of one of them:
We look at the draft as, in some respects, a luck-driven process. The more picks you have, the more chances you have to get a good player. When we look at teams that draft well, it’s not necessarily that they’re drafting better than anybody else. It seems to be that they have more picks. There’s definitely a correlation between the amount of picks and drafting good players.
No comments:
Post a Comment