From the Vault: Brady vs. Manning, Superclutch

On the cusp of football season, a look back at a comparison of two of the all-time greats. This was the second post in a two-part series,  originally published on January 25, 2012. 

In the last post, we improved on the “game-winning drive” metric by looking at 4th quarter drives when the game was tied or within one score. That post examined drives starting between the 1:00 mark and the 15:00 mark of the 4th (and OT). Let’s define “Super Clutch” drives as those starting in the last five minutes. (Again, excluding desperation drives starting under a minute.)

Below are the Super Clutch results for Peyton Manning and Tom Brady. Included are the other MVP-level QB’s of this era, Drew Brees and Aaron Rodgers:

Super Clutch Years Drive Pts/Dr TD% Score% Avg. Yds Avg. Start Avg. Time TOL % at Home
Tom Brady 03-07, 10-11 19 3.95 47.4% 68.4% 47.6 33.0 02:29 1.68 26.3%
Peyton Manning 03-10 31 3.52 38.7% 64.5% 44.6 32.5 02:31 1.84 45.1%
Drew Brees 06-11 17 2.94 35.3% 47.1% 47.5 26.1 02:55 1.29 41.2%
Aaron Rodgers 08-11 16 1.63 18.8% 25.0% 28.3 27.8 02:57 1.94 25.0%

NB: If New Orlean’s placekicker had converted all of his field goals, Brees would boast a 64.7% score rate and 3.47 points per drive.

So Brady has been better — otherworldly, really — in the final moments of games. This, despite playing mostly on the road. Manning continues to look amazing as well, and so does Drew BreesAaron Rodgers has a long way to go in this regard.

And for those in the mood for some small-sampled data, here are the Big Three QB’s performances on regular clutch drives in the postseason:

Clutch Playoffs Years Pos Pts/Dr TD% Score% Avg. Yds Avg. Start Avg. Time TOL % at Home
Tom Brady 03-07, 10-11 9 2.89 22.2% 66.7% 37.3 33.3 07:06 2.78 11.1%
Peyton Manning 03-10 11 2.09 18.2% 45.6% 33.2 25.3 05:23 2.73 81.8%
Drew Brees 06-11 7 4.14 42.9% 71.4% 36.0 36.1 10:13 2.00 28.6%

And alas we see where Manning earned his (unjustified) reputation as coming up short in the postseason. It’s unjustified because, as anyone who has read this blog knows, the sample size is too small to conclude anything, and these figures don’t control for defensive strength. Brees has been magnificent, scoring on five of seven playoff clutch drives, and Brady has failed on only three postseason drives: early in the 4th against the Colts (2006), the Giants (2007) and in the final minute against Indianapolis again in the 2006 AFC championship.

One final note: After examining a number of clutch performances in basketball, it’s particularly interesting to observe that these quarterbacks seem to perform so well in clutch (and super clutch) situations. In basketball, shooting and scoring typically decline later in games. Yet, at least with Manning and Brady, they’ve performed as well (or better) than the best offenses in NFL history in these situations.

I. Historical Impact: WOWY Score Update

How valuable is a player? How many points per game is he worth? In sports, these are Holy Grail questions that play-by-play data has helped estimate. But how do we compare Magic and Bird when they played before play-by-play was available? How do we compare Russell and Chamberlain when they don’t even have a complete box score?

A few years ago, I circulated a method that takes a stab at these questions by using injuries, trades and free agent signings to compare teams with and without a given player. The result is an historical, (mostly) apples-to-apples comparison of value between players, called WOWY. (There’s a full primer on WOWY attached to the end of this post.) The lineup data — not from play-by-play, but from game-by-game — gives us the same insight into players for the last 60 years.

Take Bill Walton’s legendary rise and fall in Portland. All other things being equal, how did the team fare with and without him in the lineup? It turns out, Walton’s missed time from those years produces the best WOWY score in NBA history. (See the third tab, titled “Top WOWY Runs.”) In other words, Walton had the biggest observable impact of qualifying players (i.e. players who were injured or traded) on any team ever, his presence improving the Blazers by more than eight points per game.

In researching Thinking BasketballI examined hundreds of these WOWY runs. For those familiar with it, I also cleaned up the data, adding controls and incorporating postseason games for over 1,500 instances since the inception of the shot clock in 1954-55. And if we combine those instances for players — only focusing on what I’ve liberally called their “prime” — we can see who left a large impact when in and out of the lineup for an entire career.

Below are the top 10 prime WOWY scores of all-time, with a minimum sample of 20 games missed:

Top 10 WOWY Scores All Time

Indeed, the best combined numbers are from players often found in all-time top-10 or top-20 lists. You can see all the results in this spreadsheet of over 200 qualifying players.

The two outliers — Robertson and West — make most top-20 or top-15 lists. (ESPN had them at 11 and 13, respectively, in their recent top-100 rankings.) While Oscar is largely revered, most people don’t know that his impact was quantifiably enormous, dragging an otherwise inept team in Cincinnati to respectability, then later catapulting Kareem’s Bucks into the upper stratosphere.

Meanwhile, when West was healthy, many of his teams were elite, only overshadowed in history by the dynastic Celtics. Amazingly, West’s teams performed better with him in all 12 lineups that he missed time. Oscar did the same for 11 consecutive lineups. (Note that about one third of WOWY scores on that list are negative.)

538’s Benjamin Morris ran a limited version of this years ago to argue for the greatness of Dennis Rodman, although he only used a minimum of 15-game injury blocks. Rodman’s good, but he clocks in at 16th here. And yes, Kobe (26th) beats Jordan (32nd), but MJ’s number comes largely from 1986 when he broke his foot and missed most of the season. (His 22 missed games from 92, 93 and 95 respectively would elevate him to 25th on the list.)

While this is all valuable data, it’s still limited. It doesn’t help answer our original question for players who don’t miss much time, like Chamberlain and Russell (and even Jordan). We’ll address that issue in Part II of this series on historical impact. For now, I’ll leave you with a WOWY primer below…


What’s WOWY?

It stands for “With or Without You,” and compares the performance of a roster with a given player and without that given player over the course of an entire game. It is an attempt to isolate a player’s impact on that given roster.

I almost always control for players who played at least 25 minutes per game (noted in the control column of this spreadsheet). This typically yields five to seven-man rotations for most teams, depending on how they distribute the minutes. There are some instances where I’ll control for the entire starting 5, even if someone is below the 25-minute mark. Similarly, there are situations that call for including two players at around 23 to 24 minutes per game because there is no clear-cut fifth man on a team.

How is WOWY different from On/Off?

On/Off captures changes within a game. WOWY captures changes from game-to-game. One strength of WOWY is that multi-collinearity is not a problem; in other words, player values cannot be confounded by moving in and out of the game together. In that sense, it is an incredibly pure representation of a player’s value to a given roster, troubled by issues like sample size (major issue), team growth (minor issue), opponent unhealthy lineups (minor) and valuable bench cohorts (minor).

(Note that some lineups have synergistic effects where the whole is greater than the sum of the parts, and removing any player from that equation can disrupt the synergy.)

What’s a WOWY Score?

It’s an attempt to quantify how impressive a given WOWY run is. It takes into account sample size, the distribution of SRS scores in a given era and the quality of the player’s team.

What is “95% +/-?”

It is a confidence interval, based on the SRS-variance of a typical NBA team. For example, from 1977-1978 the Blazers were a -1.2 team in 26 controlled games without Bill Walton. A 95% +/- value of “3.5” means that 95% of the time, the actual full-season SRS of such a team will fall within 3.5 points of that value, or somewhere between -4.7 and +2.3 SRS. (Note: More consistent teams will be slightly penalized by this and more inconsistent teams with benefit from it.)

What is SIO?

It stands for “simple in/out,” and is a basic curving of impact based on the quality of a team. It means that taking a -10 team to -5 is given less value than taking a +5 team to +10.

When combining runs for a “prime score,” why is SIO different than WOWY score?

Uneven samples can provide extremely warped results due to some basic math illusions. Take Michael Jordan, who missed the majority of his games in 1986. His team’s “out” totals will then largely reflect the 1986 Bulls (who were below .500), but his “in” totals will be weighted heavily by the Bulls dynastic teams. So, even if his team performed the same with or without him, his out sample largely be from a -3 SRS team, while his in sample would be teams closer to 9 SRS.

WOWY score was designed to correct this problem — for multiple seasons, it takes the impact (SIO) from a given sample and weighs it accordingly. For instance, if a player makes a team 10 points better in a five-game sample, and then two points better in a 20-game sample, his weighted impact is 3.6 points (because 80% of the sample is from the two-point change).

The actual in and out values are included for posterity, but unless a player played on relatively consistent teams, the numbers won’t reflect the actual impact the player had on his lineups.

Why are there multiple entries for the sample player-season?

The controls are different. Players might miss games from one lineup and then, following a team trade, might play with and without a different lineup.

From the Vault: Are Role Players Worse on the Road

This post was originally published on June 3, 2012. 

On the ESPN pregame show before Game 4 of the Eastern Conference Finals between Boston and Miami, there was a long discussion about why peripheral players tend to struggle more on the road than at home. Which, of course, begs the question…do peripheral players really struggle more on the road than at home?

If we break down player importance by minutes played, we can stratify everyone in the NBA into six categories, ranging from guys who play under 15 minutes per game to those who play more than 35. This is what the results look like for free throw shooting:

Free Throw% Away Home Diff Away % of Home
Players Over 35 mpg .763 .757 -.006 100.8%
30-35 .796 .795 -.001 100.1%
25-30 .774 .768 -.005 100.7%
20-25 .712 .722 .011 98.5%
15-20 .707 .689 -.018 102.6%
Under 15 .677 .675 -.002 100.3%

This confirms another myth-buster we learned at the Sloan MIT Sports Conference this year: teams shoot free throws better on the road, not at home. Players over 35 minutes per game see a small improvement on the road, with home-cooking only reserved for players in the 20 to 25 minute bracket. Those players see a 1.1% decrease in free throw shooting on the road.

Obviously, free throw shooting isn’t a category that tells us much. Instead, let’s look at overall box score statistics that ballpark play based on the basic box stats. First, Game Score (similar to PER) and then Expected Value run only on the box score stats. Finally, points per game and True Shooting% are included.

Game Score Away Home Diff Away % of Home
Players Over 35 mpg 14.6 15.7 1.1 93.2%
30-35 11.1 12.0 0.9 92.5%
25-30 7.7 8.7 0.9 89.4%
20-25 5.9 6.1 0.2 96.8%
15-20 4.0 4.0 0.0 99.8%
Under 15 2.2 2.4 0.2 90.9%
Box EV Away Home Diff Away % of Home
Players Over 35 mpg 4.5 5.4 0.8 84.3%
30-35 3.8 4.4 0.6 87.4%
25-30 2.9 3.4 0.5 84.0%
20-25 2.3 2.5 0.2 92.6%
15-20 1.5 1.5 -0.3 102.2%
Under 15 0.6 0.7 0.1 85.9%
Points Per Game Away Home Diff Away % of Home
Players Over 35 mpg 18.95 19.62 0.7 96.6%
30-35 14.75 15.36 0.6 96.0%
25-30 10.79 11.17 0.4 96.6%
20-25 7.81 7.81 0.0 100.0%
15-20 5.43 5.31 -0.1 102.2%
Under 15 3.18 3.33 0.2 95.5%
True Shooting% Away Home Diff Away % of Home
Players Over 35 mpg .535 .556 .021 96.2%
30-35 .534 .554 .020 96.4%
25-30 .522 .534 .013 97.6%
20-25 .520 .519 -.001 100.2%
15-20 .506 .506 .000 100.0%
Under 15 .471 .485 .013 97.2%

Based on these measurements, lower minute players are actually more consistent on the road than they are away from home. Consider:

  • 25-30 minute players decrease the most on the road by Game Score and Box-based EV
  • 25-30 minute player see no decline in points per game and TS%
  • Under 15 minute players see a Road decline comparable to high minute players in points and efficiency

Otherwise, the players with the biggest drop-off in road performance are the high-minute players! Whether it’s composite box metrics or scoring and shooting, the biggest difference between road and home is typically seen in the key players. In the composite metrics especially, the high-minute players see a significantly larger decline on the road than the low-minute players do.

Keep in mind this is not a definitive study, but a broad examination. We could change the criteria to examine “only All-Stars” or “only All-Stars in the playoffs,” and it’s possible the results look different. But it’s important to note, that in general, it is not the role players who decline more on the road, but the stars.

The Best “Healthy” Offenses of All-Time

In the last post, we looked at the best point-differentials of all-time posted by teams that were “healthy” (when all 25-minute per game players were in action, minimum 41 games played together). But what about isolating the offensive side of the ball?

Since box scores aren’t readily available before 1984, we are limited to teams from the last 32 seasons. But that’s fine — 99 of the top 100 most efficient teams in NBA history played in 1984 or later. (The 1982 Nuggets are the exception.)

Before analyzing the list, a quick disclaimer to keep in mind: Offensive rating is not a perfect representation of offensive quality. Teams can choose offensive-centric lineups at the expense of defense for a net boost, such as crashing the offensive boards instead of retreating in transition defense. Below are the top offenses, relative to the defenses faced, based on these healthy lineup standards:

Healthy Offense Relative ORtg

The 2006 Suns played the second half of the season without Kurt Thomas after running out of big men. As a result, they played one of the most lopsided lineups in NBA history, starting Boris Diaw at center and three wings alongside Steve Nash. They had one big man (Brian Grant) off the bench. While they feasted on defenses, they were defensively compromised and posted an abysmal +6.6 defensive rating during these games. Given that, I wouldn’t rush to crown them the GOAT offense. Some of the Dallas teams on the list also suffer from this kind of lineup tradeoff, slotting Dirk Nowitzki at center next to an offensively-leaning forward.

Offensive rebounding has declined drastically over the last two decades as teams have sacrificed crashing the glass in order to defend transition. Some teams still crash the glass hard, but sometimes individuals are just great offensive rebounders. Dennis Rodman is the ultimate example example of this — he’ll jump a team two tiers by himself. For example, in 37 games without Rodman, the ’96 and ’97 Bulls — with Longley, Kukoc, Jordan and Pippen playing — saw a 5.1% drop in their offensive rebounding rate, which is about the difference between the best offensive rebounding team in the league and an average one.

So let’s expand the list to include components that help place a team’s offensive rating in perspective, such as offensive rebounding rate and turnover percentage. Let’s also include the raw offensive rating and true shooting percentage (TS%) of the team:

Healthy Offense Expanded

First, the 2004 Kings number is shocking because they did it without Chris Webber; they smoked the league with Brad Miller starting in place of him, a phenomenon I discuss in Thinking Basketball.

The ’96 Magic and ’16 Warriors certainly jump out as candidates for the best offenses ever. If you’re eyes thought you’d never seen anything like Golden State this year, you were right; the Warriors shooting efficiency yielded an unheard of 1.19 points per scoring attempt. The ’96 Magic were amazing too, but aided by a shortened 3-point line.

There’s a dark horse in there: The 2016 Cavs, the team that beat the Warriors. Injuries have masked an all-time level offense, led by LeBron JamesKyrie Irving and even Kevin Love. They are not in the upper stratosphere of shooting efficiency, but are a low-turnover offense (11.5%) that benefits from player-specific offensive rebounding by Tristian Thompson.

But how good is an offense that can only take advantage of a fundamentally poor defense? While most offenses perform better against weaker defenses, Cleveland has no correlation between an opponent’s defensive strength and its own offensive production. A linear regression predicts that the Cavs offense will actually perform better against elite defenses than almost every team on this list, including Golden State. Given the small samples, I wouldn’t put too much stock in this, but it is worth noting nonetheless.

Of course, there’s an elephant in the room. Where will Kevin Durant, Steph Curry and the 2017 Warriors place on this list?