Overrated and Underrated in MLB

Over at HHS today, Bryan O'Connor posted a query about who the most over- and underrated players in MLB were.  And specifically, he asked that we (if we were so inclined) devise a method for measuring it.  I came up with separate lists for position players and pitchers, and I want to post the comments I wrote as I came up with my method over here on my own blog.  Why have something so fun only floating around on OTHER people's sites?

Here's what I wrote:

-------------

To mimic fantasy baseball, I took five categories: HR, R, RBI, SB, and H (I took H instead of BA, because I wanted them all to be cumulative). Then I set up a fake fantasy scoring system: 1 pt per hit, 2 pts per RBI or R, 5 pts per HR or SB. This way, everything is (sort of) scaled to H, such that 200 H=100 RBI=100 R=40 HR=40 SB, and a 200 H, 100 RBI, 100 R, 40 HR, 40 SB season is worth 1000 points (that’s a pretty awesome season, I think). Then, for one season, I would divide the number by 2500 – because 1000/2500=.400, which is a batting average representatively awesome enough that it goes with the stats I posted earlier.

Then, I took WAR/25 (in other words, the above season would be seen as equal to a 10 WAR season). You can quibble with my number sense or not, but the goal was to come up with a system, right?
Anyway, for a running three year total, I still just added everything together and divided, but by 7500 and 75, respectively, so the results were scaled to one another. Then I just subtracted the second column from the first (I tried dividing; that didn’t work very well because of negative numbers). I used all players with 1000 PAs in the last 3 years. This made for 226 hitters Fangraphs gave me in the Custom Report I generated for this project.

The Most Overrated Players:
Adam Dunn
Mark Reynolds
Eric Hosmer
Raul Ibanez
Alex Rios
Michael Young
Nelson Cruz
Ichiro Suzuki
Delmon Young
Rajai Davis

The Most Underrated Players
Buster Posey
Mike Trout
Yadier Molina
Joey Votto
Carlos Ruiz
AJ Ellis
Joe Mauer
Evan Longoria
Matt Carpenter
Ben Zobrist

Now, if you ask me, that list looks pretty darn close to right, as far as who sabermetrically-minded folk see as the superstars of baseball, as opposed to what fantasy-focused folk see. This worked pretty well. I’m gonna try something with pitchers, and see what I can do. Be right back…

 -------------

And my second comment (this one has been corrected, since I made an error when I initially posted it on HHS):

-------------


I’m back!

I did something similar for pitchers.

The categories I used were IP, W, S, SO, ER, BB, and H. I shaped them into five “buckets,” and all had to be cumulative, just like the hitters. The five categories I used were IP, 10*W+5*SV, SO, IP-ER, and IP*2-(BB+H). They also had to be scaled to one another. So just as the hitters were scaled to H, I did the same, but to IP. The “ideal” season (and worth 1000 points, just like the hitters) is this: 200 IP, 20 W or 40 SV, 200 SO, 50 ER (2.25 ERA), 200 BB+H (1.000 WHIP). That season would be worth 1000 “points,” just like with the hitters. I used 180 IP minimum, so that the report was roughly the same size as the hitters (I had 226 hitters, 239 pitchers).

I’m not as confident in this method as I am with the hitters; especially using Fangraphs WAR, rather than some combination of B-R and Fangraphs, which would be my preference. Nonetheless, the list actually looks pretty good, I think. So here they are.

The Most Overrated Pitchers
Bronson Arroyo
Ervin Santana
RA Dickey
Tim Lincecum
Yovani Gallardo
Jeremy Hellickson
James Shields
Ian Kennedy
Jason Vargas
Kyle Lohse

The Most Underrated Pitchers
Matt Harvey
David Robertson
Phil Coke
Roy Oswalt
Aaron Cook
Kevin Millwood
Javier Vazquez
Chris Carpenter
Matt Belisle
Hyun-Jin Ryu

There are a lot of relief pitchers on that bottom list. Before you go about saying I’ve overvalued saves, only three of those guys (Holland with 67, Papelbon with 98 and Kimbrel with 138) have more than 4 saves in the last three years. And when I changed the formula to be SV/3 instead of SV/2, it was still the same 10 names – the order only changed slightly.

Again, I’m not as confident about this as the position player list. I don’t know that it’s accurate, but it’s certainly one way of looking at it.

-------------

Specifically, I wanted to post a couple of things here that I thought about as I did this little project.

First, what are some factors that lead to over- or underrating players?

1.  Park factors - Hitters in hitters' parks are overrated; likewise for pitchers in pitchers' parks.
2.  Guys who provide a lot of defensive value - This is obvious; it's not even measured in the former measure.
3.  Positional scarcity - Also not taken into account is position.  To be fair, I generalized to "fantasy players," and fantasy players are usually VERY aware of positional scarcity, so that's not entirely fair.  The mainstream public, though, is not.
4.  DIPS theory - Using Fangraphs WAR, this is going to have a huge impact on who's rated well or poorly.  RA Dickey's numbers suffer from this in particular.  He may actually be a bit overrated (since his performance last year was so bad), but Fangraphs openly acknowledges that FIP-based WAR underrates knuckleballers, so Dickey suffers.
5.  Outs used - Leadoff hitters who don't walk a lot can accumulate MANY more at-bats than players who hit lower in the lineup and take the occasional walk.  Since all of the stats used in the former calculation are at bat-based (well, technically RBI and R aren't, but RBI in particular come pretty scarcely on BBs), that's a major factor.  Plus, if you imagine two guys, each with the same five "basic" stats, wouldn't you prefer one who did all that in 500 ABs, but with 100 BBs and no CS if the other guy also had 600 PA, but no walks and 15 CS?  The first would have created 300 outs; the second would have created 415.  It's an ENORMOUS difference, and it's often not thought of.
6.  Guys who hit lost of doubles - Doubles are the ugly stepsister of hits.  They're better than singles, but the "H" and "HR" column tell you nothing about them.  Ditto for three-baggers; there are just fewer of those, so they're less of a thing to worry about.  Guys who slap a bunch of those hits, though, are tremendously underrated by the "newspaper" stats.

One last thought I should share is just the formulas for each, in long form.

For hitters:

H+2*RBI+2*R+5*HR+5*SB
                 2500

For pitchers:

IP*4+SO+10*W+5*SV-ER-H-BB
                    2500

The idea is that these are equivalent; they're probably not.  But it's really just for fun, so don't worry about it too much!  Any thoughts?