Putting Flash Polls Into Perspective

A man walks into an Irish bar in South Boston and calls out “Sully!” Half of the gathered patrons turn their heads to respond. The man concludes that 50 percent of Americans have the last name of “Sullivan.”

If you understand the absurdity of this logic, then you likely also understand the absurdity of the so-called “online” or “flash” polls. In the wake of the third presidential debate, for example, AOL — seemingly a respected news source — published a poll showing that 57 percent of viewers thought Donald Trump had won the debate. Trump seized on these results as well as other flash polls, taking to Twitter to boast, “The results are in on the final debate and it is almost unanimous, I WON!”

Really? In the ensuing days, several other, non-flash polls were released that showed the exact opposite. Hammered by Hillary Clinton in the last debate, Trump’s standings declined dramatically. He and Clinton had been neck and neck on Sept. 18, according to Real Clear Politics’ poll of polls. As of this publication, her lead is about 3 points, in the RCP polling average.

Tell folks that the flash polls are wrong and the typical response is that, well, so are all polls. But that’s not true.

How could the flash polls have been so wrong?

The basic problem is that anyone can participate in flash polls. AOL admits as much, noting its approach is “unscientific.”  The participants, just like that Irish bar in Southie, are self-selected. There’s no effort to get a representative sample of voters. Indeed, the campaigns often encourage their supporters to vote. Flash polls have no more predictive power than does astrology.

So why bother doing them at all? Part of the answer relates to basic human inquisitiveness. We want to know and we want to know now! Any answer — even a wrong answer — seems better than none. But the flash polls also have appeal because, quite frankly, few people understand how opinion polling works. What most people know about polling and statistics comes from the aphorism variously attributed to Mark Twain or Benjamin Disraeli — “There are three kinds of lies: lies, damned lies, and statistics.”

Tell folks that the flash polls are wrong and the typical response is that, well, so are all polls.

But that’s not true. Real polls, the ones where folks use solid sampling techniques and ask fairly framed questions, can give a solid picture of what is going on. The biggest challenge for legitimate pollsters is solving the bar-in-Southie problem: getting the right sample.

The trick is in trying to predict who will actually vote. Most pollsters will look to turnout from past elections and then try to make reasonable adjustments for what they think will be going on this year. So, for instance, if in previous elections women were 53 percent of the turnout, a pollster might expect the same kind of turnout this year. But suppose a pollster decides that women are exceptionally motivated in 2016 and therefore weights his sample to assume that, say, 55 percent of the electorate will be women? Given that women support Clinton over Trump by a wide margin, making that assumption will skew the results in Clinton’s favor. Similarly, another pollster might conclude that low-income white men might this year be inclined to vote in greater numbers than ever. That assumption in turn would skew the results in Trump’s favor.

Given we won’t know turnout for sure until after Election Day, these assumptions are just educated estimates. There are valid ways to test them (e.g., by asking those being polled if they plan to vote or not) but even so, there is no certainty until after the election.

If you’re contemplating making an election wager with a workmate or friend, you’d be smart to simply do this: Believe the real polls and bet accordingly.

But having said there’s no certainty is not the same as saying that well-conducted polls are worthless. There are ways to control for various biases. Folks such as data hound Nate Silver (who runs the FiveThirtyEight website) will look at individual polls and weight them based upon the polling firms’ past successes. Furthermore, as we get closer and close to the election, polls become ever less speculative. It’s easier to figure out who will truly vote. And voters start making up their minds, decreasing the number of “uncertain” voters who are less firm in their convictions.

Thus, it appears increasingly evident what will happen on Nov. 8: Trump will lose, perhaps decisively, to Clinton — and this, despite the FBI’s re-opened investigation of Clinton’s emails.

Naturally, Trump and his supporters resist that conclusion. The Republican is now calling the major polls “phony.” Others will concoct unlikely scenarios — perhaps a surge of last-minute Trump supporters, or perhaps a collapse in voting by minorities. But those are wishful fantasies. If you’re contemplating making an election wager with a workmate or friend, you’d be smart to simply do this: Believe the polls — the real polls — and bet accordingly.

Update: This post has been updated with the latest polling average on Monday.