Online polls have become ubiquitous across various websites, news sites, and forums, including on the Kerrville Daily Times, Kerrville Breaking News Forum on Facebook, and other local websites. But what value do they provide? Should we place any weight on these opinion polls?
In recent months we’ve seen online polls purporting to measure the approval rating for City Council and Commissioners Court, the popularity of various candidates running for Kerr County Sheriff, opinions on the Animal Shelter, opinions on climate change, a survey of Kerrville’s greatest needs, and approval rating of the Kerrville 2050 plan. The polls have become a mainstay on the Kerrville Daily Times website and the results are often featured in weekend editorials. But what do these polls actually tell us?
Online Polls vs. Statistically Designed Polls
We are surrounded with polling data. Every day, news organizations, universities, blogs, random websites, and research institutes around the world launch countless opinion polls on everything from the brands of clothes you prefer to the job approval rating of the President to which contestant should win Dancing with the Stars. But the accuracy of a poll depends on how it’s conducted. And unfortunately, accuracy is often not the goal of the survey at all.
When professional researchers launch an opinion poll that they wish to be statistically accurate and reflective of the opinions of an entire population of interest, the scientists go through a rigorous design process based on complicated mathematical formulas to ensure that their sample is representative and random. The results of the poll are published with heaps of additional data, assumptions, and other information that helps the end user understand the methodology, the results, and the limitations of the poll. Other scientists then critically examine the results to determine whether or not the polling is scientifically rigorous and useful.
National polls are a highly controversial subject, but news organizations rely on them almost every day to report on public opinion, economic indicators, and other matters of public concern. On a smaller scale, local news organizations incorporate their non-scientific online surveys into editorials and newspaper articles, giving them weight and significance when no scientific rigor has gone into their design or implementation.
“My advice would be to avoid opt-in polls that provide little value to the public… Otherwise, the news organizations run the risk of being irresponsible and possibly harmful.”
Andrew Seaman, Society of Professional Journalists
Online polls, such as those conducted by the Kerrville Daily Times and Kerrville Breaking News Forum, have a few critical flaws that make them inherently inaccurate and problematic:
- Online polls are not random. Anyone can participate, regardless of where they live or who they are. Non-random sampling is problematic.
- Online polls do not measure a representative sample of the population. Only those with internet access can participate, and often, only those with the right type of social media account that login at the right time. Certain demographic segments or economic classes are either missed entirely or over-represented.
- Online polls are open to manipulation by groups that want to “stuff the ballot box.” The polls become a popularity contest or a contest of which interest group can mobilize the most respondents to answer the poll. Also, web apps are easily available to allow one person to vote multiple times, essentially stuffing the ballot box for a particular viewpoint.
- Online polls are skewed to reflect the opinion of those motivated enough to vote (and vote often). Groups that have strong opinions are much more likely to vote than those who do not have strong views.
Frank Newport, the editor in chief at Gallup, an organization that performs scientific polling, warned that most reader-response surveys on news sites are more for the benefit of the news agency than for the benefit of the public, increasing views and clicks on the agency’s website. An ethics advisor for the Society of Professional Journalists, Andrew Seaman, says, “My advice would be to avoid opt-in polls that provide little value to the public… Otherwise, the news organizations run the risk of being irresponsible and possibly harmful.”
Designing Scientific Polls
What makes a poll scientific? What is a population, sample size, margin of error, etc.? Why do they matter?
Statisticians rely on complicated mathematical formulas and research to determine how many people must be polled, what types of people should be polled, and how the questions should be posed to generate quality results. Without getting above our pay grade, let’s discuss some of the science behind these more rigorous polling projects.
We’ll use Kerrville as an example. The entire population of Kerrville is around 25,000. But if we were polling to find out which candidate is favored in a city council election, we wouldn’t care as much about the entire population of Kerrville. Residents under 18 cannot vote, so we’d want to exclude those folks. Second, not all persons over 18 are legally able to vote for various reasons, so we wouldn’t want them in our population of interest. We might want to limit our population of interest to just eligible voters. But in the last city council election, only about 3,000 people cast a ballot — about 20% of the 15,000 that are registered to vote. So our poll would be more valuable if we only asked likely voters how they felt, rather than asking every single person in Kerrville, regardless of their voting eligibility or their likelihood of casting a ballot. We can identify the 3,000 most likely voters by reviewing voter roll data from previous elections.
So let’s plug those numbers into an online service that estimates what size sample you need to get reasonably accurate results. If we assume our population of interest (likely city voters) is about 3,000, and we can live with a 5% margin of error (plus or minus 5% on the final results), and we needed a confidence level of 95% (a high degree of certainty), we would need at least 341 respondents, chosen randomly, out of our population of 3,000 likely voters. We can reasonably predict that if we do this right, we can be within five percentage points of the actual opinions of the entire population we are studying.
But even with a pretty large sample size of randomly chosen individuals, our poll is susceptible to errors such as bad randomization, biased questions, and other problems. And these days it’s not easy to reach folks by phone, so it makes accurate polling even more difficult. So even with our carefully designed poll, we may not get perfectly accurate results. Science is hesitant to state that the results of any one experiment are absolute truth, and the margin of error and confidence interval figures are how they relate that uncertainty.
Low Response Rate
Let’s compare our hypothetical poll with the sample size of a recent Kerrville Daily Times poll. On September 29, the newspaper asked readers, “What’s your opinion of the Kerrville 2050 plan?” According to the results, 109 people submitted a response. So even if the respondents were randomly selected across the population and adjusted for proper demographics, we’d only have about 1/3 of the number of respondents we’d need to get the opinions of likely voters.
Kerr County has a much larger population than Kerrville, of course, with nearly 50,000 residents and approximately 35,000 registered voters. The November 3 KDT poll asking about the job performance of Commissioners Court only garnered 95 responses, but the results were touted on the editorial page the following weekend as evidence that the residents disapproved of the county’s actions. It may very well be true that the majority of county residents disapproved of the Court, but this poll cannot tell us that definitively given such a small sample size and due to the other problems inherent in online polls.
Many of the online polls we looked at suffered from this very low response rate.
Poorly Designed Questions
In one poll that’s currently online at the Kerrville Daily Times, readers are presented with the following survey:
Here is the problem… These choices are not mutually exclusive. We have covered the Animal Shelter issue extensively on Kerrville United and our readers have suggested several different options for the Commissioners Court to consider regarding the hours and days of operations. Most of our readers seem to think that the shelter should open on Saturdays, and have suggested closing on another weekday to offset the staffing hours. However, the Times is asking readers to make a singular choice among these options, when in fact, a reader may think several of these options actually achieve the same thing. This type of poll does not provide valuable information because the choices are ambiguous and confusing.
The Dangers of “Voodoo Polling”
Voodoo polling is the pejorative term for polls that have no scientific basis, such as most online polls. As we mentioned above, online polls have problems: self-selected, not representative of the entire population, and easy to cheat/game. But if a newspaper or online poll is inaccurate, what is the harm? There are several potential consequences when users rely on inaccurate polling numbers.
- Bad Public Policy. If an inaccurate poll claims that the vast majority of the public supports or opposes an issue that is being considered by a governmental body, that governmental body may make decisions that are, in fact, not in the best interest of the entire community. For instance, with the animal shelter poll discussed above, the elected officials could be confused about what the public really wants. If only 0.2% of the population says they disapprove of something, such as with the Commissioners Court approval poll, does that really send a valuable message to Commissioners?
- Influencing Voters. Some voters may be influenced by “groupthink” when a poll shows overwhelming support for one candidate over others. In the recent poll of favored Sheriff candidates, an overwhelming number of respondents chose Larry Leitha and Eli Garcia. Will other voters be swayed by a self-selecting poll that could have been influenced by a motivated group of supporters, when in fact, the sentiments of the rest of the voters may not be counted?
- Driving the Conversations. Because so many people rely on the newspaper and read online forums in our small community, many of these polls tend to steer the discussion around the proverbial water cooler at our workplaces, watering holes, and social gatherings. If the information in the polls is not accurate, how does that affect the conversations about solutions and candidates and issues? Are we even discussing the right issues?
- Toxicity. Some online polls are intentionally misleading or divisive. Certain online forums have become notorious for toxic comment sections where readers tear one another down and use highly charged rhetoric that further divides us rather than uniting the community around shared values. A recent poll on Kerrville Breaking News Forum asked, “Is it ok to hate people because they are different than you?” (June 15, 2019. https://www.facebook.com/groups/TexasBreakingNews/ ) Is this type of poll helpful or harmful?
Can Kerrville benefit from scientific polling?
Contributors to (and readers of) Kerrville United are often news junkies and politics enthusiasts, and many of us would be thrilled to have scientifically credible polling for certain issues affecting our community. Unfortunately, due to the expense of conducting statistically accurate polls in a community our size, very little useful polling is available.
There is one notable exception. The City of Kerrville has participated in the National Community Survey program since 2016. This poll is conducted using a scientific sample of 1,800 randomly-selected households and the responses are weighted to represent community demographics. In short, it’s perhaps the only statistically reliable poll conducted in Kerrville at this time. The broad results of this poll can be found on the City of Kerrville’s website.
Community leaders can and should rely on valid polling to gauge the community’s sentiment about certain issues. But until such time as accurate polls are readily available, we should all be careful to look at these online polls with a grain of salt. Are online polls interesting and entertaining? Most of us would say yes. But are they valuable? I think we must admit that they’re not.
Takeaways
Online polling is inherently inaccurate, and in some cases, dangerous. But the polls are only dangerous if institutions and the public at large take them seriously. When major news sources rely on unscientific surveys to represent the opinions of an entire community, our community suffers. Because these polls drive traffic and revenue to these sites, they are not likely to disappear anytime soon. And this website has not been totally immune from the attraction of online polls. We have referenced them in previous articles. But as informed citizens, we must recognize the limitations and shortcomings in these polls, and shouldn’t rely solely on them to make judgments or decisions about the issues that face our community.
More Information
- Online polls are everywhere: here’s why we should be wary before trusting them
- Open-access poll (Wikipedia)
- Why Online Voting Is a Danger to Democracy
- Why You Shouldn’t Trust ‘Polls’ Conducted Online
Authored by Aaron Yates
