Q&A: Do AI and bogus respondents threaten polling’s future? – Pew Research Center

Numbers, Facts and Trends Shaping Your World
Read our research on:
Full Topic List
Read Our Research On:
|
At first glance, recent news stories may appear to spell doom for the polling industry.
Instead of asking real people for their opinions, some companies are asking artificial intelligence what people would think. In other cases, bad actors are using AI to fake survey responses at scale. Some polls also face serious data quality issues because of bogus respondents who don’t take surveys in good faith.
It can be difficult for people outside the polling industry to understand what’s going on and which polls to trust. In this Q&A, Courtney Kennedy, Pew Research Center’s vice president of methods and innovation, answers some common questions about the current polling landscape in the United States.
No. We only interview real people. We don’t use AI to tell us what the public thinks. There are ethical and scientific concerns with using AI to replace humans in public opinion surveys.
Polling is fundamentally about humans – what they’re thinking and experiencing. Polls give the public a voice in politics, business and other areas. In the political realm, they let leaders know what hardships people are experiencing and what they’d like the government to do differently. If we stop polling people and just assume AI knows the answer, we risk misunderstanding what’s actually happening in the public.
There are scientific concerns, too. Many studies have explored how AI performs as a replacement for interviewing humans. We’ve done some of this research ourselves – for experimental learning only, not for reporting.
These studies have found that AI estimates tend to stereotype groups of people, have a harder time representing Republican viewpoints than Democratic ones, and understate the level of disagreement in public opinion.
In time, AI may more accurately imitate answers real people would give. Even so, a core philosophical point of polling is that we ask real people about their views.
No. That threat applies to “opt-in” surveys. Those are polls that people can proactively sign up to take – for example, by responding to social media ads offering a reward for taking a survey. Since it’s easy to adopt a fake identity online, opt-in polling opens the door to AI and bad actors seeking to commit fraud.
We don’t use opt-in surveys at Pew Research Center. We use probability-based sampling instead. That means we select real people in real life, not online. We start with a giant list of all U.S. home addresses and randomly select some of them. We initially contact people by snail mail, and each year we invite only a carefully selected sample of the public to take our surveys. Any one person’s chances of being selected are tiny, and you can’t self-enroll or nominate yourself to take our surveys. That means bad actors don’t have the ability to self-select into our panel.
They could, but what matters is scale. Since anyone can sign up for opt-in surveys, bad actors can create multiple fake accounts and take dozens or even hundreds of surveys each day to maximize the financial rewards they get.
Let’s say someone creates five AI bot accounts online and uses them to take 200 surveys a day, with a reward of $1 per survey. That person could hypothetically haul in $30,000 a month by using AI to speedily complete lots of opt-in surveys.
With a probability panel like the one we use at Pew Research Center, that kind of large-scale fraud isn’t possible. You can’t create multiple accounts or take surveys all day. Our respondents each have a single account, take an average of fewer than two surveys per month and receive an average of $11 per survey. Someone using AI to answer our surveys could hypothetically earn $22 per month – not exactly a huge payday for cheating.
Would someone intent on defrauding pollsters want to make $30,000 per month, or $22?
Bogus respondents are survey-takers who make no effort to answer questions truthfully and instead try to complete surveys as quickly as possible to get monetary rewards. One hallmark of bogus respondents is that they tend to give positive answers, like “yes” or “approve.” This tendency of bogus respondents has led to false conclusions and news organizations needing to retract stories based on opt-in polls.
The fundamental reason bogus respondents exist is because opt-in surveys generally encourage anyone interested to sign up. Rigorous surveys do the opposite – the researcher carefully recruits people. People can’t just sign up on their own.
No. In some cases, opt-in polls can produce results similar to those produced by probability-based polls. If the poll is only being used to measure the opinion of all adults on a given topic – say, approval or disapproval of the president – opt-in polls and probability-based ones may give similar results.
However, opt-in polls have been shown to generate erroneous data for young adults and estimates of relatively rare behaviors. Those rare behaviors include things like belief in conspiracy theories, identifying with an Orthodox Christian church, military service and support for political violence.
Not necessarily. It’s important to assess how respondents were recruited, but it’s not all that matters. For example, in recent elections, there have been examples of probability-based polls that were not weighted properly and, as a result, were way off.
To be trustworthy, a poll has to be well-designed from start to finish. Using a probability-based sample is the best possible start, but other practices are important, too.
Our surveys are more expensive because getting participation from a randomly selected sample of Americans is time- and labor-intensive. We recruit people offline, in real life, via letters mailed to home addresses. We use random sampling so that nearly all U.S. adults have a chance of being selected for our surveys. And we allow people to answer our questions by web or phone (because research shows that some groups of Americans are reluctant to take surveys online). All those efforts to be rigorous cost money.
Topics
John Gramlich is an associate director at Pew Research Center.
901 E St. NW, Suite 300
Washington, DC 20004
USA
(+1) 202-419-4300 | Main
(+1) 202-419-4372 | Media Inquiries
ABOUT PEW RESEARCH CENTER Pew Research Center is a nonpartisan, nonadvocacy fact tank that informs the public about the issues, attitudes and trends shaping the world. It does not take policy positions. The Center conducts public opinion polling, demographic research, computational social science research and other data-driven research. Pew Research Center is a subsidiary of The Pew Charitable Trusts, its primary funder.
© 2026 Pew Research Center
We use cookies and other technologies to help improve your experience; some are necessary for the site to work, and some are optional. Learn more in our privacy policy.
These cookies are required for the website to function and cannot be switched off.
These cookies are used to collect information about how you use our website.
These cookies are used to track your activity and show you personalized content.
These cookies allow the website to remember choices you make.

source