Few new “themes” or trends have emerged in the last week (and I refuse to acknowledge what happened last night), so I felt it was useful to review some of the metrics we use, why we use them, and the pitfalls of relying too much on any one metric… or your “eyes” for that matter.
Shooting Efficiency – The Basics
Metric: Field Goal Percentage (FG%)
How its calculated: Field goals made / Field goal attempts (FTM/FTA)
Why you should use it: You shouldn’t. Ever. Ban it.
The drawbacks: It does not recognize the 50% higher value a three pointer has. For example, a player with a FG% of 33% could have the same impact on a game as someone with a 50% FG% if the former only shot threes. That’s a huge difference and why the metric is useless. It also doesn’t recognize a player’s ability to get to the line (where typically they shoot a much higher percentage)
So you should use…
Metric: Effective Field Goal Percentage (eFG%)
How its calculated: (FieldGoalsMade + 0.5 x 3ptFieldGoalsMade) / FieldGoalsAttempted.
Why you should use it: Because a 3 point shot is worth 50% more points than a 2 point shot. You can make less, but contribute more.
The drawbacks: Few, but in some cases True Shooting percentage (below) gives a fuller picture.
or….
Metric: True Shooting Percentage (TS%)
How its calculated: Points / (2 x (FieldGoalsAttempted + 0.44 x FreeThrowsAttempted)
Why you should use it: Because a 3 point shot is worth 50% more points than a 2 point shot. You can make less, but contribute more. Players that are able to draw fouls and shoot a high percentage from the line get “properly credited”.
The drawbacks: None. It gives the player full credit whether they shot a 2, 3 or were fouled.
“Holy Grail” models – Searching for the Ultimate Player Ranking
Metric: Player Efficiency Rating (PER)
How its calculated: Simply. Well, not so much.
Why you should use it: Generally speaking, rankings meet the “eye test”…
The drawbacks: … but the “eye test” is often biased towards scorers. PER has two major issues: 1) it tends to overrate high volume, inefficient shooters and 2) it relies on box score statistics, several of which have flaws – many other contributions to scoring and defence are not recognized in the box score (more below)
Metric: Wins Produced (WP; WP48)
How its calculated: Dr. David Berri, its author, recently updated his methodology here.
Why you should use it: Translates box score statistics into what matters most “how much is Player X contributing to wins”. Attempts to correct PER’s inefficient shooter bias, among other factors.
The drawbacks: 1) Also relies on box score statistics [see below] 2) Makes a team adjustment for defence – making the assumption “defence is essentially a team activity” and no one individual makes above or below average contributions. 3) Values rebounds highly (due to change in possession) and gives full credit to the outcome, but does not give any credit to how that rebound was generated (e.g. perhaps by excellent individual defense that forced the missed shot).
Metric: Adjusted +/-, Regularized adjusted +/-, and others.
How its calculated: Don’t ask…
Why you should use it: It attempts to measure the player’s overall impact on the floor and removes the issues resulting from box score statistics. How does it do this? It “adjusts” for the quality of the players one is playing with as well as the quality of the players one is playing against on the floor.
The drawbacks: It can be very “noisy”. One usually needs a couple seasons’ worth of data for it to be utilized properly.
What are my issues with box score statistics?
Several pioneers in the field of basketball analytics have done an excellent job at focusing attention on possessions and highlighting the pitfalls of past trends like “paying for points”. Researchers like Oliver (Four Factors, Win Shares), Hollinger (PER) and Berri (WP) created new metrics to better recognize other aspects of the game.
However, relying entirely on box score statistics can lead to a few problems.
The simple issue I have is box score numbers are largely “result based”, i.e., you get the “check mark” based on the end result and no (or little) value is assigned to the “components” of, for example, how a field goal was made, or a defensive rebound was obtained. Here are some of my (not unique) concerns about various box score data:
- Assists – a few things: 1) its obvious, but your great passes mean little if your teammates are not efficient shooters/scorers 2) a huge issue I have: you do not get credit for an assist if the play ends in a shooting foul [the exception being “and 1” of course] 3) its still a judgement call by the scorekeepers.
- Blocks – when a defender blocks a ball out of bounds, it does not change the possession. Some feel this makes it overrated. However, it still: results in a “miss”, could result in intimidation (e.g. more hesitant to drive next time), and usually takes seconds off the clock (team forced to inbound the ball and “reset” offence).
- Steals – steals are based on a successful outcome. Thus, a defender could gamble and attempt several steals without being successful. The “missed steal” gamble is can be expensive as the defender is often well out of position.
- Charges – are not recognized, yet change the possession (and often ignite the team/fans).
- Defensive rebound – rebounding is an incredibly important part of the game. However, a defensive rebound is a function of: 1) a missed shot 2) blocking out your opponent and 3) luck [how the ball comes of the rim – direction and distance]
- Personal fouls – context can be important – e.g weak foul 23 feet from the basket, moving screens, intentional at the end of game to stop the clock or a foul as a result of help defence (that would have been an easy basket).
Overall: The largest challenge is there is no adjustment for the quality of players someone is playing with nor against. A player that is productive in “garbage time” may not be nearly as productive in “crunch” time. Points guards playing with efficient scorers will have more assists. Bigs playing with defensive minded wings will have more defensive rebounds.
What is more than a bit odd is a Wins Produced post, with the following argument.
The absolutely most incorrect way to judge a decision is solely by the outcome.
We agree, and also wonder why Wins Produced (largely) only measures outcomes.
The challenge I have with some of these metrics is similar to the Wins Produced authors’ criticism of advanced +/-
You are not measuring what you think you are measuring.
The Curious Case of Reggie Evans
First of all, I’m a fan of Reggie Evans. Despite being a one-dimensional player, he can be very effective in the right lineups. However, he also highlights a challenge with Wins Produced.
Source: data from http://www.thenbageek.com/players
Last year, according to his WP48 score, he should have been considered a top ten player in the league. I wrote about it previously.
The response to some of my curiosity on why he was rated so highly was “so even though the shooting is terrible, there isn’t much terrible happening … It’s perfect: if you can’t shoot, don’t!”
This is the root of the problem. Someone has to shoot the darn ball. I wish I still had the screen grab, but I captured Evans (several times, but this was the best example) with the ball at the top of the key. His defender was no where to be found – he double teammed Bargnani right away. Evans had as clear as shot as one can possibly have and not from a great distance. He also had a fairly clear lane to the basket for a layup. What happened? Well, as WP likes, he didn’t do anything terrible like shoot or turn the ball over. The shot clock continued to count down and Evans had a hard time passing the ball and, because Bargnani was now double teamed, he had one less option to pass the ball. I cannot recall who ultimately got the ball, but 5 or 6 seconds has drained from the clock leaving only a few seconds left. And we know FG% drops significantly in this type of situation.
The challenge is obvious, but not doing anything terrible on paper, its clear his lack of shooting ability hurt the team in this case. He doesn’t get any “knocks” against his WP score, but his teammate likely did.
Simply put: Wins produced has an allocation problem.
After refusing to acknowledge this for some time (The article noted yet again: “I don’t want to get into another rehash of the endless ‘WP overvalues rebounds’ argument”), Dr. Berri (sort of) conceded somewhat and has altered the WP calculation: “although diminishing returns – as detailed in Stumbling on Wins — certainly exists for defensive rebounds (but not for offensive rebounds), the size of the effect is ‘small’.”
However, the “before and after” data shows some fairly dramatic changes:
Dr. Berri massages these large changes with the phrase “the players who ranked towards the top of the league before still rank towards the top now.” Sure, generally speaking, the players are are in the same zip code.
However, the data also suggest rebounds may have been “overvalued” previously. For example, Blake Griffin’s WP48 was 48% higher than after the adjustment. That’s quite material. To use another example, Kevin Garnett was thought to “produce” 24% higher WP48 than Paul Gasol before this adjustment. Now, when rebounds are viewed differently, Gasol is viewed as producing 4% higher WP48. That is pretty significant and somewhat validates previous criticisms.
Ultimately we are encouraged Dr. Berri made the adjustments. However, the issue of allocating some credit to those that may have forced few extra “misses” remains. Nonetheless, the Wins Produced team is appears open to continuous improvement to address its flaws. Continuous improvement will move away from unfortunate posts like calling cancer survivor (no cancer survivor is an idiot – trust me, as I’ve been witness to both positive and negative outcomes), who’s a 1000 win coach and has led his team to Western’s Conference’s second best record.
Bring the Noise! Adjusted +/-
I fully admit to appreciating the concept of adjusted +/-. Why? it attempts to measure ultimately what matters – how much does that player influences the production at both ends of the court, adjusting the for quality of players he’s playing with and against. It indirectly measures all the intangibles: charges taken, tipped balls, help defense, running the lanes, setting good picks etc. The problem is the data is often very “noisy”. It doesn’t mean its worthless, it means you want to ensure the sample size is reasonable. As well, researchers like Joe Sill (who now works for the Wizards, introduced Regularized Adjusted +/-) and Dapo Omidiran (A New Look at Adjusted Plus/Minus for Basketball Analysis) continue to improve upon this analysis.
It would be great for these pioneers to spend less time focused on criticizing each other’s work and more time working through flaws found in each.
So Your “Eyes” are Better then, RIGHT?
No. Don’t get smug yet.
The VAST majority of fans, scouts, GMs etc cannot possibly watch and remember enough tape to accurately access all players. Sure, some get most of the way there by scouting one opponent at a time. But we know that the vast majority remember the “buzzer beater” and yet completely ignore (or most likely forget) that player shot 4 for 12 in the game and had 4 turnovers. Some make more errors in judgment than others.
Eyes Wide Open, Mind Wide Open
The data helps, sometimes a lot. The contributions of Oliver, Hollinger, Pelton, Rosenbaum, Winston, Berri, etc are very significant. Whether you agree with their theories or not, they – at a minimum – enhanced the discussion (even if a few are not overly open to debate).
You may believe your eyes when you say Hedo Turkoglu is clutch when the data allows us to know he isn’t. Or you may accept that Dwight Howard’s contribution to defense is identical to Hedo Turkoglu’s and simply make a “team defense adjustment” – or your eyes tell you Howard is a much superior defender. You may complain about your “go to” player shooting 38% FG%. Or you may realize most of those shots were threes, he got to the line 12 times and was very effective. Your eyes may tell you Russell Westbrook is an all-star based on 22 ppg, but Wins Produced tell you he’s more of an average PG given how often he turns the ball over (4x a game!) and has so few (relative) assists despite having several top offensive weapons. The metrics may look great, but your eyes tell you he’s playing against inferior bench players and it won’t translate to starters minutes.
The best analysis of players and/or teams combine a number of metrics as well as your well trained eyes. Your “eyes” don’t see every play of every game and record it accurately. Nor does any one metric tell the whole story, such that one can make conclusive statements (as much as some may try). Basketball is a complex game and thus anyone who doesn’t utilize several tools is bound to miss the full picture.
Questions? Arsenalist introduced a forum thread dedicated to “Statophile Q&A”. If you prefer to send questions privately, you’re welcome to email me at tomliston [at] gmail [dot] com or find me on Twitter (@Liston).
