Judging by box-office statistics, most of America has flocked to a movie theater at least once over the past two weeks to see Star Wars: The Force Awakens. Even those who have enjoyed the movie have been forced (no pun intended) to acknowledge that the movie’s plot (semi-spoiler alert!) is strongly derivative of the initial Star Wars movie. J.J. Abrams has many strengths, but finding original takes on classic stories is not among them.
When Travis Sawchik’s Big Data Baseball: Math, Miracles, and the End of a 20-Year Losing Streak published last year, it was legitimately advertised as a sequel of sorts to Michael Lewis’ classic 2003 book Moneyball. Unlike J.J. Abrams, however, Sawchik was able to find an original angle in which to anchor his story, which gives his book a very different emphasis than its predecessor.
Sawchik’s area of emphasis was not just original, but is also broadly relevant to understanding how organizations work. In that respect, Big Data Baseball speaks directly to anyone hoping to effect change, whether in the private or public sector. For that reason, it is one of the most important books anyone can read in 2016 America.
Getting Beyond Moneyball
Moneyball was about the Oakland Athletics and how their brilliant general manager Billy Beane used then-cutting-edge analytics—or “sabermetrics,” a slang which combines “metrics” with the acronym for the Society for American Baseball Research—to overcome the A’s financial disadvantages. Lewis carefully detailed the history of Beane’s analytic insights and various examples of how he acted on those insights.
For much of the public, Moneyball was the first glimpse at how young men with analytical backgrounds, often stereotyped as “Ivy League geeks” or less flattering terms, were taking over baseball’s front offices, much to the delight of sabermetric geeks (such as myself) who had grown up reading Bill James in the 1980s and spent years wondering when baseball teams were going to recognize the obvious insights known to all readers of James’ Baseball Abstracts. The story Lewis told in Moneyball was of revolutionaries overthrowing an old regime whose traditions were little more than superstition—and not just any revolutionaries, but visionaries who had been mocked or ignored and now were proving that they had been right all along.
Thanks to Lewis recognizing the story’s potential and his skill at popularization, the book became a bestseller and spawned a good movie starring Brad Pitt as Beane. The term “Moneyball” spread beyond baseball and became a slang for seeking efficiencies, even in government.
Beane openly spoke of how he and his lieutenants approached baseball decisions as searching for underpriced value and inefficiencies to exploit; it was not a coincidence that Moneyball quickly became mandatory reading at within the asset management industry. Within baseball, where sabermetric-savvy operators were already in the process of taking over teams’ front offices, that trend was accelerated by the popularity of Moneyball.
Less than 13 years after the publication of Moneyball, every team has information and analysis unheard of when Moneyball was published. But this democratization of sabermetrics has made many fans wonder in recent years whether there were remaining inefficiencies poorer teams could use to overcome their relative penury—could they compete with teams like the Yankees and Red Sox, often described as playing “Moneyball, but with money”? The A’s themselves had been up and down in the years since Moneyball’s publication and other teams were still stuck in a toxic combination of low payrolls and perpetual losing—in particular, the Kansas City Royals and Pittsburgh Pirates.
The definitive book on the Royals success in recent years has yet to be written, but Travis Sawchik stepped into the breach with Big Data Baseball, an account of how in 2013 the Pirates broke their 20-year streak of losing more games than they won and finally making the playoffs. Sawchik’s book hits many of the same beats as Moneyball: the inability to compete with bigger-market teams on payroll, the need to find undervalued players, descriptions of the methodologies they used to identify the unappreciated value in such players, and how it all led to the team’s unexpected success.
Yet, Sawchik’s story turns the Moneyball formula on its head. Despite the title, Big Data Baseball is not primarily a story about data. It is about how change actually is implemented in an organization used to doing things very differently.
The heroes of Moneyball were Billy Beane and his team of analysts, who reject the perspective of the team’s scouts and other holdovers and boldly impose new programs from the top down. By contrast, if there is a hero in Big Data Baseball it is manager Clint Hurdle, who uses his traditionalist background to ensure that the sabermetric insights of the Pirate analysts are implemented by the coaches and players.
If there is a runner-up for the “hero” award, it is the Pirates’ chief analyst Dan Fox, but there’s a twist: Fox’s success was more due to his ability to translate and relate quantitative insights to outsiders than because of the analytical superiority of his insights.
Sawchik recognizes the best ideas are not useful unless they can be widely understood and acted upon. Big Data Baseball is a story of how the Pirates came to understand, and act upon, those insights at every level. The book is devoted to answering questions that so many institutions struggle with, such as: How do you get comfortable that a novel idea is worth trying?
Balancing Experimentation and Respect
The answer is experimentation. The Pirates tried radical defensive shifting, based on data about where hitters tended to hit the ball in difference situations, for several years with their minor league team with consistently good results, convincing the organization to try it at the major league level. Given their 20-year losing streak, desperation was also a good motivator. This brings us to another question institutions needing to implement changes have to answer: How do you get buy-in from different segments of the organization, including people heavily invested in the old ways?
Despite his traditional background, Hurdle was convinced that the Pirates’ defensive shifting plan was worth trying. And his old-school background was exactly what was necessary for the plan to be implemented. Hurdle made sure that the Pirates’ analysts who formulated the plan got out of their offices and spent extensive time with the players and coaches, from spring training through the end of the season, eventually even traveling with the team on road trips. He saw that they needed to interact directly with, and gain the respect of, the people who would carry out the plan—otherwise it would never work. And the analysts needed to shed any tendencies to either resort to authority and issue orders, or condescend to non-quants in the manner familiar to anyone who’s had to deal with tech support.
More importantly, each side in this discussion needed to recognize and respect the expertise of the other. The players and coaches needed to recognize that people who’d never played the game could nevertheless have valuable insights, while the analysts needed to recognize that even though most of the players were not quantitative experts, they were not dumb. The Pirates’ staff also recognized that because baseball players have such highly developed visual recognition skills, the players could absorb and act upon huge amounts of information if it was presented in the right visual format. So then, how exactly do you incorporate on-the-ground feedback and adjust accordingly?
The first step is to be open to that feedback. As part of the effort to forge respect between the players and coaches, on the one hand, and the analysts on the other, Hurdle encouraged the analysts to explain as often as necessary the bases for the Pirates’ new plan and the players and coaches to critique the plan as much as they wanted. This was often, given the radical increase in how often the Pirates planned to shift defensively.
The analysts upheld their end of the bargain. Sawchik describes several instances where players or coaches, and particularly pitcher Mark Melancon, observed that the data underlying the Pirates’ plan overlooked various nuances, the analysts took those critiques, realized that the players and coaches were often correct, and modified the plan accordingly. In this case, the analysts needed to respect the unique knowledge of the players and coaches and recognize that the analysts’ numbers did not necessarily contain all the answers.
Beyond Big Data
Too many struggling organizations cannot answer these questions as effectively as the Pirates did—just ask General Motors. It is inspiring to read Sawchik’s account of how the Pirates, one of the worst teams in baseball for 20 years, were able to do so.
In some respects, Moneyball now seems dated. The main specific idea featured in that book—selecting only college players in the amateur draft, rather than high school players—has since been proven to no longer work; the A’s moved away from that strategy soon thereafter and the book’s chapter detailing that strategy now reads like a history of Lamarckian genetics. And the scorn expressed in that book towards the old-school baseball lifers who just didn’t “get it” also is a relic of the past, replaced by the mutual respect and partnership of people with very different perspectives in Big Data Baseball.
Sometimes, cliches about respect, teamwork, and diversity provide a more complete picture than just number-crunching, and the sequel can be more inspiring than the original. The lessons of Big Data Baseball extend well beyond the world of sports—just imagine if the Obama administration had approached implementing Obamacare with half as much sophistication as the Pirates did in implementing defensive strategies, or if a decade ago GM had taken this approach to turning around the auto industry. American institutions that are suffering crises of competence could do much worse than to read Big Data Baseball and really take its lessons to heart.