Byron Buxton is having himself a very good month of August, now up to .330/.362/.649 through 27 games in the month. Some credit a new toe tap while others note that he had a very similar hot stretch last season. But whether or not you reference his 4.3 bWAR, 2.8 fWAR, or simply the eye test as he plays center field, one thing has become apparent this season; Byron Buxton is comfortably in the 10% of all baseball players.
But Buxton’s profile is an interesting one. His batting line on the season is .249/.310/.402 and those numbers are… not very good, to say the least. In fact, they’re very close to teammate Eduardo Escobar, who has hit .252/.312/.402 on the season and is a 0.4-0.7 WAR player depending on where you look. If you’d like to isolate Buxton’s season after his horrific 4-49 start to the season, fine, I’ll play that game, even though ALL players would look better if you remove their worst 16% of the season. After Buxton’s 4-49 start to the season, he has hit .275/.338/.449, a line similar to Jedd Gyorko’s .272/.341/.469 line.
These numbers are better. It’s better than most baseball players could do in a season. But Jedd Gyorko hardly comes to mind when people brainstorm the best hitters in the league. What separates players like Buxton and Andrelton Simmons is their defense. If you ask a baseball fan who the best defensive shortstop is, the answer should be Andrelton Simmons. If you ask a baseball fan who the best defensive center fielder is, the answer should be Byron Buxton (Kevin Kiermaier would also be an acceptable answer). But, if everyone is in agreement that Buxton, Simmons, Kiermaier and other players around the league are bar none the best defensive players in the game today, why can’t Baseball Reference and Fangraphs agree on a WAR total for these players? Byron Buxton, for example, has a 4.3 bWAR and a 2.8 fWAR, as noted above. Although there are a lot differences within the two WAR calculations, the answer in Buxton’s case (along with several others) is, unfortunately, very simple: defensive runs saved (DRS) vs. Ultimate Zone Rating (UZR).
Below are a few of the biggest and most notable discrepancies between bWAR and fWAR:
These are huge differences, with Buxton, Simmons, and Kiermaier all varying by 1.5+ WAR each between the two sites. The reason for the discrepancies lies in DRS and UZR, as Baseball Reference uses DRS in their WAR calculation and Fangraph uses UZR in their WAR calculation.
Below are a few of the discrepancies between DRS and UZR:
Buxton, Simmons, and Kiermaier all have a much higher DRS than they do UZR, which explains their much higher bWAR. Xander Bogaerts, on the other hand, has a much higher UZR than he does DRS, and wouldn’t you know it, his fWAR is much higher. And I know what you’re thinking: There’s no way two defensive statistics that vary so heavily could influence WAR that much. And, again unfortunately, you would be wrong.
This is a plot of all 127 qualified position players with the exception of catchers, since there is no UZR for catchers (that’s right, Fangraphs doesn’t even calculate WAR consistently for all position players). Along the x-axis is the player’s difference in DRS and UZR and the y-axis is the difference in that player’s bWAR and fWAR.
When I began plotting these points, I knew there would be some correlation between the variance in defensive statistics and the variance in WAR totals, but this blew me away. The correlation is huge! This season, with an R-squared value of 0.805, over 80% of the variance in (non-catcher) position player WAR between Baseball Reference and Fangraphs can be explained by the variance in DRS and UZR. That’s incredible.
And this (very slowly) leads me to my point; defensive statistics are, in their current state, extremely incomplete thanks to a relative lack of data availability. UZR can vary drastically year-to-year, takes something like 3+ season to actually stabilize, and still “does not necessarily tell you how he actually played just as it does not necessarily tell you what his true talent is” (Fangraphs). DRS also has it’s flaws, including this kind of funny shift error that caused Brett Lawrie to lead the league in DRS in 2012. So why, with the knowledge that these two defensive statistics are incomplete, untrustworthy, and incredibly variable, are we accepting an equally variable WAR as gospel?
I think WAR is an incredible idea, and it has improved drastically over the short time that it has been prominent. In my lifetime, I’m fairly sure WAR will iterate to a point which you can add up all the WAR on every team and almost nail the team wins (or pythag team wins) perfectly. In fact, it does a pretty good job at it right now. But the fact of the matter is this; DRS and UZR are extremely far from perfect. If they were perfect, they would agree numerically, and that is something that will most like never happen. Thus, by extension, WAR is far from perfect as well. And as long as these two, imperfect, defensive statistics CAN cause WAR to differ so greatly, it will also always be an imperfect statistic.