- For a given player, find his career Wins Above Replacement. We used bWAR, meaning from Baseball-Reference.com. For batters, their leaderboard is here.
- For that player, find his rank on the EloRater. This is also a Baseball-Reference.com feature, designed to rank individual players by using thousands of head-to-head matchups. The rank for batters is located here, and an explanation for how the EloRater works is linked from that page.
- Based on the player's EloRater rank, determine an expected WAR (eWAR), and find the difference between eWAR and bWAR. This is how many wins the player is underrated (if positive) or overrated (if negative.)
Note that the data was pulled down in January 2012 for this analysis. Over time, EloRater rankings do change.
The issue is how to calculate eWAR. In the end, we did it by averaging the bWAR values for the 3 players immediately ahead and immediately behind the player in the EloRater rankings.
For example, here's the calculation for Wade Boggs:
Boggs is #29 on the EloRater. The three players ahead of him, along with their bWAR, are #26 Shoeless Joe Jackson (62.9), #27 Al Kaline (91), and #28 Charlie Gehringer (80.9). The three players behind Boggs are Joe Morgan (103.5), Eddie Mathews (98.3) and Cap Anson (99.5). The average bWAR of these 6 players surrounding Boggs is 89.35, and so that's Boggs' eWAR. Boggs' actually bWAR is 89, so that's just about dead on and we conclude that Boggs is neither overrated nor underrated.
We completed this calculation for all 1500+ batters on the EloRater, and then found those with the largest differences between eWAR and bWAR to populate our lists of most overrated and underrated. For players very near the top or bottom of the EloRater list, we took an average using fewer surrounding players.
Originally, we worked at fitting an equation to the relationship between the player's WAR rank and his EloRater rank.
Here's a plot of all the batters on the EloRater.
Click on that for a full-size version. As you can see, the data sort of resembles an exponential decay, but it can't be fit very well by an exponential function (i.e. it's close but not a great straight line when plotted on a log scale.)
You can see some of the outliers that represent our overrated and underrated players. For example, you can see two side-by-side points at a bWAR of about 170. These are Babe Ruth and Barry Bonds. Ruth is #1 in the EloRater while Bonds is currently toiling in the 20s. That makes Bonds, technically, "underrated" since he has the second best bWAR ever but ranks behind a couple dozen other players.
Notice around EloRater ranking of 270 there are two points well below the curve around 15 bWAR. These are Bill Buckner and Joe Carter, two guys whose bWAR is far too low for their EloRater ranking. Thus, they show up on our overrated list.
It turns out that even when we fit a series of fancy curves to this data, we got pretty much the same results as the simple moving average method we ended up using. Some players moved up or down a few spots, but overall the results were quite similar.
