Measuring team skillsPosted: January 18, 2010 at 7:49 pm in management, sports
Along the lines of yesterday’s post where I mashed up two different interests of mine (cognitive science and organizational theory), today’s post is about an intersection between basketball and management.
I don’t know a lot about basketball. I watch the game recreationally, but I’ve never played, and don’t have a feel for the sport. I do read a lot about it, though, in part because my favorite sports columnist, Bill Simmons, is obsessed with basketball, to the point where he recently published a 700 page tome called The Book of Basketball.
One of the more interesting debates in basketball these days is how to measure the productivity of a player. You would think it would be easy – take a player’s statistics like points, rebounds and assists, mash them together, and see who does the best. And that’s exactly what ESPN’s John Hollinger has done in his Player Efficiency Ratings to rate every NBA player with a single number.
However, basketball is a team sport. It has been shown repeatedly that five supremely skilled individual players may not mesh well as a team, so there’s a need for team players who fill in the little things that don’t get measured by the traditional stats. One such player is Shane Battier, who was described as the No-Stats All-Star by Michael Lewis in a New York Times story.
So how do you measure team contributions? Basketball geeks are now using something called “plus-minus”. In its simplest form, it’s straightforward – take the team differential in points when a player is on the court. In other words, if the team outscores its opponent by 5 points with the player on the court in a game, then the plus-minus for that player is +5. There are several refinements to adjust for teammates and strength-of-opponent and other factors, but that’s the basic idea.
The reason plus-minus is attractive is that the point of basketball is to have your team outscore the other team, and plus-minus measures the player’s impact on achieving that goal. As it turns out, Shane Battier tends to do well in plus-minus, even though his other stats are terrible (he rates as well below average in Hollinger’s Player Efficiency Ratings). Michael Lewis’s story describes some of the things that Battier does that don’t show up in the box score, particularly on defense, but it’s hard for most people to believe, since “numbers don’t lie”. The flip side to Battier is Kevin Durant, where his stats were eye-popping, but plus-minus said that he was actually hurting his team, in part because of his poor defense and inefficient shot selection.
Based on examples like these, it appears that traditional basketball statistics don’t necessarily measure a player’s contributions to whether the team wins; in other words, teams are measuring the wrong things. Well, not all teams. Daryl Morey, the general manager of the Houston Rockets, has been affectionally dubbed Dork Elvis by Bill Simmons, because he’s figured out a Moneyball-esque approach to measure subtle ways in which players can affect games, and his team is winning with an unconventional set of players despite losing its two All-Stars to injury this season.
Now what’s amazing to me about this is that basketball is a pretty simple game. There are five players per team on the court at a time, and the goal is to score more than the other team. The professional teams spends millions of dollars to find and train the best players, and they’re only now starting to figure out how to measure the impact that a good team player can have on winning or losing.
If basketball teams haven’t figured it out, measuring the impact of a team player in the business world, where the goals are varied and conflicting, where teams are fluid and changing, and where performance evaluation is an afterthought, would seem to be impossible. And yet I’m wondering if some equivalent of plus-minus is possible, in part because I think of myself as a good team player, and would like to see those skills more widely recognized and appreciated.
It would have to start with defining the goals of a team, and having a way to measure the progress towards those goals, the way plus-minus measures team point differential as a metric for winning games. This is already complicated, as a team may have explicit goals (deliver the project successfully) that conflict with implicit goals (don’t contradict the manager). But efforts like the “results only work environment” are starting to move businesses in the direction of focusing on the end goals, rather than the process.
The other component, measuring performance, is also difficult, as any set of metrics will immediately be gamed. Perhaps a better metric of a team player’s effectiveness would be to ask each team member at the end of each project which team member they would most want on their next project. That could favor other characteristics like popularity, but if the team members were going to be judged as a team on the results of the next project, seeing who they want on their team could be a decent proxy to plus-minus.
Wrapping up, I think that plus-minus serves as a useful reminder for managers. If the best players in basketball aren’t necessarily the ones with dominant statistics, it’s possible that the best employees are not the ones with the highest performance evaluation scores. Plus-minus is a reminder to look beyond the numbers, and watch how the team actually works together, to see who is doing the little things that enable the team to function effectively, and to think of new ways of measuring productivity that might more closely map to the desired end-goals of the organization.