On Friday CNN announced the rules by which it will determine which candidates make the main stage for the next Republican debate on Dec. 15 in Las Vegas. And while many pollsters, including me, have expressed concern about using polls at all to decide such matters, the new CNN rules are an improvement over criteria used for previous debates. If you’re going to divide the field using polls, this new approach might be the best way to do it of any so far.
Here's what's good about the rules, and one place where problems may still arise.
-- Good: The criteria includes Iowa and New Hampshire, not only national polls.
For better or worse, the two first-to-vote states make a huge difference in candidate viability, and some candidates are banking their entire candidacies on success in one of those states. So including the opinion of the voters who will cast the first ballots makes sense. Previous debate sponsors, including CNN in their previous debate, have used only national primary polling to determine participation, despite the fact that there is no national primary.
In the Dec. 15 debate, CNN will allow candidates that make 4 percent in either Iowa or New Hampshire. This acknowledges the differences between the two states and the different strategic roles each may play. Taken together, they do a good job of covering the ideological spectrum of potential Republican primary voters. What works in religious and conservative Iowa does not work as well in more secular and moderate New Hampshire, and vice versa. Letting candidates qualify based on support in either state acknowledges this reality and theoretically makes room for candidates who appeal to either state's electorate. In practice, it is not likely to make much difference this time around. But it could matter in the future and seems like a good rule to include.
-- Good: It only uses live telephone polls.
Insisting on phone polls done by live interviewers appears to be a sound practice at the moment. The transition to more Internet polling is very much underway, and in future cycles, Internet polls will be the bulk of what’s available. We support Internet polling, and are doing more and more of it for a variety of uses. But online and automated phone polls are showing wildly different results this cycle for primaries, especially as it relates to support for GOP frontrunner Donald Trump. And online polls are still too new to have a track record in presidential primaries. With no track record, and divergent results, we won’t know until after the fact whether online polls are ready for prime time. So for this one purpose, it seems better to stick with phone polls.
-- Good: It specifies the polls that will be used well ahead of time.
CNN is also getting ahead of a potential pitfall by announcing in advance which polls it will include in its calculations:
Polls that will be considered are live interviewer national and state surveys by: ABC News, Bloomberg News, CBS News, CNN, Fox News, Gallup, Marist University, McClatchy News Service, Monmouth News Service, NBC News, The New York Times, Pew Research Center, Quinnipiac University, Time, USA Today, The Wall Street Journal, The Washington Post, The Des Moines Register, the University of New Hampshire, WBUR and WMUR.
Fox Business Channel did not name specific polls in advance, instead taking the four most recent live interviewer polls as the cutoff time. That approached led to controversy when a poll by Investors Business Daily knocked Chris Christie and Mike Huckabee off the main stage for that debate. Had a New York Times/CBS News poll that fielded just after the deadline had been included in the average, both would have made the cut. By explicitly saying which polls it will included in its averages, CNN is minimizing the risk of another such last-minute controversy.
-- Questionable: It won't catch a late Jim Gilmore surge.
If there is one flaw in the CNN scheme it is the very broad time window from which they are accepting polls for the average. CNN will be averaging polls that started fielding on Oct. 29 through polls released on Dec. 13. This means that candidates who had low poll numbers for the last few weeks will remain weighed down by those numbers, even if they rocket upwards in the weeks ahead.
It’s happened before. In its previous debate, CNN ended up bending its own rules to allow Carly Fiorina onto the main stage. Her rapidly rising numbers clearly put her in the top tier, but her average was weighed down by older polls. This could happen again.
For example, if Gilmore (currently averaging 0 percent) were to make an unlikely late surge, the CNN criteria would likely still keep him off the stage.
-- Basing debates on polls is still a dubious proposition.
For many pollsters, including me, the idea of excluding candidates based on polling is still extremely uncomfortable. It’s possible that the difference between making or not making the cut could be tenths of a percentage point, or the opinion of one or two respondents in a single survey. This leaves pollsters pondering unusual questions such as how many decimal places to use in reporting results, and what methodological minutiae or interviewer hiccup might end up changing the debate stage.
Setting a debate by polling will always be a dicey proposition. But if it’s going to be done, CNN’s method seems better than what has been tried with other debates so far. It may be as close to a good solution as we’re going to get this primary cycle.
Steve Koczela is president of the MassINC Polling Group and a regular contributor to WBUR Politicker.
Advertisement
Analysis: New CNN Debate Rules Are The Best Of An Imperfect Lot
On Friday CNN announced the rules by which it will determine which candidates make the main stage for the next Republican debate on Dec. 15 in Las Vegas. And while many pollsters, including me, have expressed concern about using polls at all to decide such matters, the new CNN rules are an improvement over criteria used for previous debates. If you’re going to divide the field using polls, this new approach might be the best way to do it of any so far.
Here's what's good about the rules, and one place where problems may still arise.
-- Good: The criteria includes Iowa and New Hampshire, not only national polls.
For better or worse, the two first-to-vote states make a huge difference in candidate viability, and some candidates are banking their entire candidacies on success in one of those states. So including the opinion of the voters who will cast the first ballots makes sense. Previous debate sponsors, including CNN in their previous debate, have used only national primary polling to determine participation, despite the fact that there is no national primary.
In the Dec. 15 debate, CNN will allow candidates that make 4 percent in either Iowa or New Hampshire. This acknowledges the differences between the two states and the different strategic roles each may play. Taken together, they do a good job of covering the ideological spectrum of potential Republican primary voters. What works in religious and conservative Iowa does not work as well in more secular and moderate New Hampshire, and vice versa. Letting candidates qualify based on support in either state acknowledges this reality and theoretically makes room for candidates who appeal to either state's electorate. In practice, it is not likely to make much difference this time around. But it could matter in the future and seems like a good rule to include.
-- Good: It only uses live telephone polls.
Insisting on phone polls done by live interviewers appears to be a sound practice at the moment. The transition to more Internet polling is very much underway, and in future cycles, Internet polls will be the bulk of what’s available. We support Internet polling, and are doing more and more of it for a variety of uses. But online and automated phone polls are showing wildly different results this cycle for primaries, especially as it relates to support for GOP frontrunner Donald Trump. And online polls are still too new to have a track record in presidential primaries. With no track record, and divergent results, we won’t know until after the fact whether online polls are ready for prime time. So for this one purpose, it seems better to stick with phone polls.
-- Good: It specifies the polls that will be used well ahead of time.
CNN is also getting ahead of a potential pitfall by announcing in advance which polls it will include in its calculations:
Fox Business Channel did not name specific polls in advance, instead taking the four most recent live interviewer polls as the cutoff time. That approached led to controversy when a poll by Investors Business Daily knocked Chris Christie and Mike Huckabee off the main stage for that debate. Had a New York Times/CBS News poll that fielded just after the deadline had been included in the average, both would have made the cut. By explicitly saying which polls it will included in its averages, CNN is minimizing the risk of another such last-minute controversy.
-- Questionable: It won't catch a late Jim Gilmore surge.
If there is one flaw in the CNN scheme it is the very broad time window from which they are accepting polls for the average. CNN will be averaging polls that started fielding on Oct. 29 through polls released on Dec. 13. This means that candidates who had low poll numbers for the last few weeks will remain weighed down by those numbers, even if they rocket upwards in the weeks ahead.
It’s happened before. In its previous debate, CNN ended up bending its own rules to allow Carly Fiorina onto the main stage. Her rapidly rising numbers clearly put her in the top tier, but her average was weighed down by older polls. This could happen again.
For example, if Gilmore (currently averaging 0 percent) were to make an unlikely late surge, the CNN criteria would likely still keep him off the stage.
-- Basing debates on polls is still a dubious proposition.
For many pollsters, including me, the idea of excluding candidates based on polling is still extremely uncomfortable. It’s possible that the difference between making or not making the cut could be tenths of a percentage point, or the opinion of one or two respondents in a single survey. This leaves pollsters pondering unusual questions such as how many decimal places to use in reporting results, and what methodological minutiae or interviewer hiccup might end up changing the debate stage.
Setting a debate by polling will always be a dicey proposition. But if it’s going to be done, CNN’s method seems better than what has been tried with other debates so far. It may be as close to a good solution as we’re going to get this primary cycle.
Steve Koczela is president of the MassINC Polling Group and a regular contributor to WBUR Politicker.
Advertisement
More from WBUR