In many ways, 2016 was a cultural reckoning. The candidacy of Donald Trump brought with it a glimpse into the worldview of the so-called “silent majority,” ushering to orthodoxy ideals like protectionism and chauvinism. But of every new “ism” introduced into the mainstream, the one that has arguably had the most tangible impact is skepticism.
Indeed, Mr. Trump has often bashed the veracity of institutions as specific as The New York Times to as broad as opinion polling itself, eroding the reputation of news media. Of course, this isn’t happening solely because of the president: many outlets have taken blame, too. Now, nearly two years later, they’re trying to move forward.
Two leading data journalism sites, FiveThirtyEight and the Times‘s Upshot, recently announced adjustments in their approaches to presenting political statistics. These changes, they hope, will ameliorate many of the concerns that arose post-2016. Let’s see how they’re doing.
FiveThirtyEight: From Percentages to Odds
Since its launch a decade ago, Nate Silver’s FiveThirtyEight has been considered the gold standard for election forecasting; Mr. Silver rose to stardom by using a statistical model to correctly predict the electoral outcomes in 49 of 50 states in the 2008 presidential race, and all 50 in 2012. After 2016, however, the prognosticator was mocked for failing to foresee Mr. Trump’s victory. In response, Mr. Silver released a series of articles reflecting on coverage of the election, and he came to a conclusion — the media has a probability problem.
The prediction models on FiveThirtyEight’s website are probabilistic in nature. That means they aren’t really “calling” an election (or sports game, or Oscar winner) for any particular entity: if a candidate is ahead 51% to their opponent’s 49%, their chance of winning isn’t automatically bumped up to 100%. In other words, they would still lose about half the time. This concept, Mr. Silver argued, was poorly communicated by the media, and therefore poorly understood by the public.
So, in preparation for their forecast of the 2018 House race, the FiveThirtyEight team overhauled their landing page. Instead of percentages, they were going to show odds.
According to Mr. Silver, this change falls more in-line with how people talk about and perceive probabilities in normal interaction — when chatting with a friend, you don’t say there’s an 80% chance of something happening; you say there’s a 4 in 5 chance, phrasing that better paints the 1 in 5 worlds in which it doesn’t happen.
At face value, the shift to odds from percentages works. I’m sure that, for certain readers, this format successfully presents probabilities as less certain. My biggest gripe with the change is its damage to consistency.
As someone who checks FiveThirtyEight multiple times a day, I’m always interested in seeing how their predictions change over time, especially on a district-by-district level. Under the old format, I could quickly glance at the forecast, reference it to the last one, and move on. This version makes that a lot harder.
Think about it this way: seeing a candidate’s odds go from 13 in 20 to 15 in 21 doesn’t really tell you much off the bat; without doing mental math, I’m barely confident in saying if their chances improved or worsened! Compare that to seeing the numbers shift from 65 in 100 to 71 in 100, a case in which I’d be certain their chances improved. The same applies when comparing one race to another — because there is no common denominator in the new format, it’s difficult to instantly tell if District X is more competitive than District Y. “Spit-balled” odds work well individually, but badly together.
As the above graphic demonstrates, the traditional percentages still appear on FiveThirtyEight‘s forecast, albeit in a smaller font. Going forward, I’d recommend presenting odds by default, while also including a cookie-based toggle that lets users opt in to seeing percentages instead. This system can help please readers of all stripes.
Though FiveThirtyEight’s decision to forgo consistency for colloquialism may frustrate politics junkies and math buffs, it remains net positive. At the end of the day, their work can be understood and appreciated by a wider audience, which is certainly worth a few extra seconds of arithmetic or searching on my part. There might never be a “perfect” way to illustrate these concepts for everyone, but Mr. Silver et al. are on the right track.
The Upshot: Polling For the Whole World To See
The Upshot, a data journalism front started in 2014 by The New York Times, is, in many ways, the newspaper’s response to FiveThirtyEight (that’s probably because the Times once owned the rights to the blog). So, it’s no surprise to see The Upshot also reevaluate its place in the media. Its editors are tackling a different problem, though, and this one’s right up our alley.
Earlier this week, The Upshot partnered with Siena College to provide real-time polling results. The newspaper has long conducted surveys, but this initiative is the first time users can see the process behind it all.
They go beyond just top lines; their breakdowns also delve into things like the change in opinion over time, crosstabs, and how results would look under different weighting techniques. Additionally, The Upshot published numerous articles detailing the methodology behind their surveys, all in an attempt to “demystify polling.”
Honestly, unlike FiveThirtyEight‘s efforts, there are no real complaints about this approach. The staffers at The Upshot aren’t abandoning the quality or depth of their analysis by improving the clarity of their polls. If anything, this is a win-win: inexperienced or skeptical readers learn more about the surveying process, and advanced followers get to geek out.
My only wish — and this is a small one — is for The Upshot to be more comprehensive in their transparency efforts. Again, this would likely be an opt-in system, where users can choose to see additional details, or maybe even download the data directly. Otherwise, the move to real-time polling is beneficial for everyone. I encourage you to check it out.
Conclusion: Why Does This Matter?
The changes from FiveThirtyEight and The Upshot are far from revolutionary, but maybe that’s a good thing. For better or worse, 2016 forced number-crunching journalists to scale back on complication and return to simplicity.
Don’t get me wrong, these sites do an amazing job at analyzing and visualizing the world’s data, yet I worry they often sacrifice readability for rigor. After all, math can be scary, and throwing numbers at readers does little to help them grasp the role that stats play in their day-to-day lives. Thus, it’s great seeing outlets take steps to make data journalism more accessible. However, along the way, writers must also learn to strike a balance that doesn’t leave behind their core base of readership. Only then can this type of reporting be fully realized.
One of the comments I’ve gotten about this blog is that my content comes off as “basic” or “simple.” I agree, but that’s by design: whereas FiveThirtyEight and The Upshot tend to write for math-minded audiences, First Past the Poll‘s aim is to fill the void by presenting political statistics in a way that a broad audience can understand.
If other data journalism sites are truly dedicated to broadening their market, the niche I fulfill may one day become obsolete. And honestly, I hope that happens, because that means they accomplished their goal, and I accomplished mine.
For now, though, we have the same means, but strive for different ends.
First Past the Poll 