In the days ahead of Tuesday’s election, 10 different opinion polls from national and local firms sought to determine the favored candidate in North Carolina’s Senate race.
Most determined that Sen. Kay Hagan was leading by between 1 and 4 percentage points – the same figures polls had been reporting for months. Two had the candidates tied.
Only one put Republican Thom Tillis in the lead. Tillis ultimately won by 47,236 votes, a margin of 1.64 percentage points and an outcome that had many wondering where the polls went wrong.
“In this election, across the board, you saw a Democratic bias” in polling numbers, said Elon University Poll director Kenneth Fernandez, whose final poll had Hagan up 4 percentage points. “It’s not easy to be a pollster.”
The same problem plagued polls in other states’ Senate and gubernatorial races: Democrats led in opinion polls but fared poorly on Election Day. Even Nate Silver, who rose to fame for accurate predictions in 2012, had given Hagan a 69 percent chance of victory. Silver’s method weighs polling data alongside demographics and local political dynamics.
The failure stems from Tuesday’s lower-than-expected turnout numbers, pollsters and political analysts say.
“They were making an assumption that it would have been a presidential level of turnout,” said Gary Pearce, a Democratic strategist and blogger. “It really was just a regular midterm turnout.”
In North Carolina, 44 percent of registered voters cast ballots. That’s the same turnout as the last midterm election in 2010, when Sen. Richard Burr topped the ballot in a far less competitive campaign. Turnout fell well below the presidential election year 2012, when 68 percent of registered voters participated.
Despite Hagan’s repeated criticism of the state legislature under Tillis’ leadership, Democrats didn’t go to the polls in the numbers the party had hoped.
Early-voting totals showed that 49 percent of the 110,000 ballots were cast by registered Democrats, with only 31 percent registered as Republicans. But on Election Day, exit polls found that only 36 percent of voters were Democrats.
“There was just a very healthy turnout for Republicans and lackluster turnout for Democrats,” Fernandez said.
Pearce Godwin, director of Raleigh-based, conservative-leaning American Insights, said polling this year failed to predict the wave that swept Democrats out of office at all levels across the country.
“With survey response rates in the single digits and 41 percent of Americans now cellphone only, gaining a representative sample and accurate data on a population is an increasingly steep challenge for pollsters in all industries,” he said in an email.
Throughout the Hagan-Tillis race, the polling difference between the candidates fell within the margin of error. That’s typically 3 to 4 percentage points, depending on the poll. And even the final polls showed that up to 6 percent of likely voters remained undecided days before the election.
With so much in flux, most polls phrased their question carefully: Who would you pick “if the election were held today”?
‘A pretty elaborate model’
Predicting turnout is one of the biggest challenges that pollsters face. When questioned on the phone, many people say that they’ll plan to vote – even if going to a polling place isn’t a high priority come Election Day. “Yes” just seems like the right answer when asked about a civic duty.
Most polls work to screen out registered voters with a less-than-stellar participation record. In the Elon University Poll, the caller asks a series of 10 questions to establish voting history. Did you vote in the last midterm election? Can you name your polling place? Do you know neighbors who vote?
“It’s a pretty elaborate model to figure it out,” Fernandez said.
The High Point University Poll uses public records to determine whether someone voted in the 2010 or 2012 elections. “We tried to base our likely voters on that, plus people who had registered since then,” said Martin Kifer, director of the poll, which showed Hagan and Tillis tied in late October. “That reduces the size of your population a bit.”
Polling firms also vary in their approaches to reaching younger voters and minorities, who are less likely to use easy-to-track landline phones. Many polls have been increasing the number of cellphones they call, and some use Census data to give more weight to responses from underrepresented demographics.
But some pollsters have gone too far in weighting responses, said Francis De Luca, president of the conservative Civitas Institute, which regularly conducts polling. “People who try to compensate are ignoring the fact that older people are more likely to vote,” he said.
Older people, in turn, are more likely to vote Republican. One national exit poll Tuesday found that 22 percent of voters this year were over 65, while just 13 percent were between ages 18 and 29.
Getting it right
Only one poll put Tillis in the lead in its final pre-election survey: Harper Polling, a 2-year-old firm led by former Republican strategist Brock McCleary and based in Pennsylvania.
Harper’s Oct. 28-30 polling data showed Tillis with 46 percent of the vote to Hagan’s 44 percent. The data also had bad news for Hagan: 19 percent of Democrats said they’d vote for Tillis, and he had a 10 percent lead among independent voters. (Harper’s accuracy on those details won’t be known until the N.C. Board of Elections releases a detailed breakdown of Tuesday’s voters.)
Tillis, Harper pollsters discovered, was getting a broader base of support than many had anticipated.
“It appears that what we got right that others didn’t was the share of Democrats that Thom Tillis got,” McCleary said. “You can’t win statewide without getting some meaningful share of Democrats to vote for you.”
And while other polls still showed a slight Hagan lead or a tie, they saw responses that didn’t look good for Hagan. The senator’s campaign had sought to make the election about the Republican-led state legislature.
Many voters, however, had their eyes on Washington, D.C. And like Republican candidates across the country, Tillis was constantly hammering his opponent for supporting President Barack Obama.
“What we did see was that the people who were most energized were people who had negative views of the president and thought there should be a change,” De Luca said. “Consistently, except for a couple blips, they were saying they were going to vote on national issues.”
Most polls had forecast a close race, rarely putting Hagan in the lead by more than a percentage point or two.
“I don’t think this was a polling fail,” said Tom Jensen, director of Public Policy Polling, a Democratic-leaning firm in Raleigh. “That’s why polls have a margin of error. It is unusual that all were off in the same direction.”
FiveThirtyEight, which aggregates political polls, found there was a 2.2 percent bias in North Carolina’s Senate race in favor of Democrats and a 4 percent bias toward Democrats in Senate polls nationwide. Bias reflects the difference between the advance polling average and the actual election results.
Campaigns saw ‘close race’
With polling numbers within the margin of error, experts say neither campaign was basing their strategy off a perception of Hagan as the front-runner.
“I don’t think anybody was rushing out to Las Vegas to bet on that polling data,” said Carter Wrenn, a veteran Republican strategist and blogger, adding that the campaigns likely had far more detailed, expensive polling data that was never publicly released.
Ben Ray, a Democratic strategist who worked with the Hagan campaign, said if publicly released polls on Oct. 1 had accurately shown Tillis’ margin of victory, “I just would have said, ‘Oh, it’s going to be a close race.’
“I feel very comfortable with the strategic guidance that we got from our pollsters,” Ray added. “You see a campaign here that, relative to the rest of the country, significantly outperformed (other Democratic candidates), that outran the presidential approval rating.”
And while key Democratic groups didn’t turn out in the expected numbers, experts say the Hagan campaign did the best it could to get out the vote.
“I don’t think a field operation can create enthusiasm – it can mobilize enthusiasm,” Pearce said.
Pollsters say it will be hard to use this year’s mistakes to create a better election forecast in 2016 and 2018. “You can learn some lessons, but those lessons might not be applicable in the new political environment that we see in future elections,” Fernandez said.
Even McCleary – whose poll was also spot-on in the Pennsylvania governor’s race – isn’t banking on the techniques his firm used this year.
“I think the industry is changing so much that a good year does not mean that you can stop innovating,” he said. “Staying ahead of the curve in polling will be difficult for a long time to come.”