Making Sense of the Numbers

Kristi Pronovost Merit Shop Spokesman Blog

By: Brent Sailhamer, Director of Government Affairs, ABC Keystone

It’s no secret that voters have been lying to pollsters for years about their actual intent in the ballot box, but that fact was even further magnified in 2016. Nearly every poll conducted predicted a massive victory for Hillary Clinton in the presidential election, and while she did win the national popular vote, final results by state varied drastically from the polls conducted earlier in the year.  As voters now focus on the Pennsylvania gubernatorial race, where polls consistently show Scott Wagner behind by double digits against incumbent Governor Tom Wolf, Republican supporters are pointing to 2016 as a testament to the inaccuracy of polling. So are the polls believable?  Maybe and maybe not.  Whether you believe the final result or not, here are five secrets to reading between the lines that can help you better understand what’s really going on.

  1. Sample size. When a poll is released, the sample size, or number of respondents, can make a big difference. Generally, the larger the sample size, the more broad the trend tends to be.  Say, for instance, a poll is released with a sample size of 200 people that shows favorable results for a candidate. In Pennsylvania, that means that the poll is a representation of only three voters in each of the Commonwealth’s 67 counties. While three people is hardly a representation of an entire county, each county also does not contain the same amount of voters. So three supportive responses in Forest County, with a total population of 7,400 people, is more accurate (albeit only slightly) than three supportive responses in Philadelphia County, with a total population of 1.56 million people. The more people are contained in the sample size, the more accurate the poll tends to be, particularly in larger geographic areas.
  2. Equal representation. While most credible polls focus exclusively on likely voters – that is, voters who identify themselves as likely to vote in the upcoming election and not just registered – another critical component to the poll is the party breakdown. While some would argue that a fair result would be a 50%/50% split in respondents between Democrats and Republicans, Democrats maintain a voter registration advantage of nearly 900,000 people in Pennsylvania. For a poll to be relatively accurate, that poll should admit a slight advantage for Democrats in a statewide race. Any poll that seeks to keep the respondents close to an even split inherently skews the poll toward Republicans, promoting inaccuracy. While it’s impossible to predict how many registered voters will actually vote, theoretically, self-identifying as “likely voters” should help to provide more accuracy.
  3. Recent events. Remember, polls are a snapshot in time, or a reflection of how the respondent feels at that particular moment. Credible polls rarely ask a respondent to speculate about the future (e.g. Who will you be voting for in November?), but rather ask about the current feeling of the election (e.g. If the election were held today, who would you vote for?). Not only is the general result valuable, but these snapshots also show the efficacy of recent events. Take, for example, President Trump’s recent visit to Pennsylvania in support of Rep. Lou Barletta and Scott Wagner. Prior to the visit, polls showed both candidates roughly 12 points behind in their respective races.  Polls taken just days after the visit showed roughly the same result – 12-13 percentage points behind. While the end result is valuable, the polls also show that President Trump’s endorsement carried little weight with self-identified likely voters. For campaigns, this trendline is helpful in late game strategy – whether to more heavily emphasize the endorsement of surrogates or to downplay that support.
  4. The popularity contest. Most people focus on the general election match-up, or “head to head,” in a poll. The pollster narrows the choices for the respondent, asking him/her to choose between two or more candidates or if they are still undecided. While the portion of respondents who are still undecided is enormously valuable to campaigns, polls often ask a follow-up that can provide further clarity. By studying the favorable opinions of a candidate, where respondents are asked to choose their level of favorability ranging from Very Favorable to Very Unfavorable, campaigns can gain insight in the weaknesses of an opponent. For instance, if a candidate is leading his/her opponent in the head to head but shows high unfavorables, it’s reasonable to assume that respondents have a low opinion of that candidate’s personality but, given the options, are willing to vote for him/her nonetheless.
  5. Putting the work in. Again, because polls are a snapshot in time, stringing results together to study the trendline can be helpful for campaigns as they promote various messages and strategies. For instance, a candidate who is slightly behind and spends two weeks campaigning on a narrowly-focused fiscal message and finds little change in poll results could reasonably conclude that the message is not resonating with voters. The longer the trendline is consistent, the less ability a campaign has to modify the likely results. In 2014, incumbent Governor Tom Corbett found himself far behind in polls early on.  For weeks, he highlighted various fiscal policies that he had implemented, but found little change in poll numbers. He then focused on his campaign to end corruption in Harrisburg and his role in “Bonusgate” convictions, which also found little traction. As months ticked by, each strategy failed to close the gap, pointing to a fatal flaw in the candidate himself.

September 25, 2018