Skip to content
Sign in Subscribe free

AI can now fake survey responses. What to ask before you trust a poll.

Five questions to ask before you trust a poll in the age of AI

AI can now fake survey responses. What to ask before you trust a poll.

A new study finds that AI bots can complete online surveys virtually undetected.

The barrier to entry for malicious actors is low. Each fake respondent costs only about 5 cents to deploy, according to a Dartmouth College study published in the Proceedings of the National Academy of Sciences. A real survey respondent typically earns about $1.50 per completion, meaning a bot operator could profit while flooding a survey with fabricated data.

The paper's recommended fix? A return to controlled recruitment methods, such as address-based sampling used by Lumaris Research.

Our surveys are built for the age of AI: the Minnesota Community Survey recruits by mail from Minnesota addresses chosen at random. Read how the Minnesota Community Survey works.

The snapshot

Sean Westwood, a Dartmouth political scientist, built what he calls an "autonomous synthetic respondent" from a roughly 500-word prompt. Across 43,800 evaluations spanning 139 questions and 6,700 trials, it fooled nearly every safeguard that survey platforms use to catch bots and bad actors.

These are not crude bots that can be easily screened out. Westwood's AI:

  • Includes plausible typos and corrections to appear more human
  • Simulates realistic reading times and writing styles based on education level
  • Mimics human mouse movements and keystrokes
  • Passes standard attention checks — 99.8% across 6,000 trials, with 10 total errors
  • Works even when its prompt is written in Russian, Mandarin, or Korean, producing fluent English answers

It would also adjust its answers based on its demographic persona. For example:

  • If assigned a lower income level, it would report paying lower monthly rent
  • When asked to name all 50 U.S. state capitals, it named fewer capitals correctly when assigned a less-than-high-school education (30% accuracy) than when assigned a postgraduate degree (96% accuracy)

The primary model tested was OpenAI's o4-mini, with the core results validated across eight additional LLMs including Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Preview, and Llama 4 Maverick. It's worth noting that AI models have dramatically advanced since then.

Why this counts

Survey data shapes real decisions: where foundations invest, what legislators prioritize, and what journalists report as fact. And an increasingly common source for that data is opt-in online panels, where anyone can sign up with minimal verification. These surveys are fast and cheap, but the same low barriers that make them accessible also make them vulnerable to exactly the kind of fraud Westwood describes.

The data quality problem predates AI. Before 2019, open online surveys typically yielded more than 75% usable responses, according to a peer-reviewed analysis of 36 studies in Frontiers in Research Metrics and Analytics. After 2019, usability predominantly fell into the 0–25% range — a collapse driven by professional survey-takers and traditional bots well before AI agents entered the picture.

How vulnerable are opt-in panels to AI specifically? According to Westwood's study, it would have taken just 10 to 52 fake AI responses to flip the predicted winner in seven major national polls conducted during the final week of the 2024 presidential campaign. At roughly 5 cents per fake respondent, that's less than $3.

The AI could also infer what a researcher was trying to prove and produce answers that artificially confirmed the hypothesis. That raises a separate concern for academic research. In one replication of a published political science experiment, hypothesis-confirming answers increased by 22 percentage points compared to human responses.

The study called AI respondents "a potentially existential threat" to unsupervised online research. Westwood warned that synthetic respondents could bias public opinion measures, warp electoral strategies, and erode trust in democratic institutions.

Worth noting

In a follow-up published in PNAS in February 2026, Westwood and a colleague at the University of Pennsylvania ran an empirical audit on Prolific, a major online survey panel. Two independent AI-detection methods flagged 4.4% of respondents as likely AI — 128 out of 2,898.

Several survey panel companies have since published their own research. Prolific researchers flagged fewer than 1% of about 4,800 responses across a dozen providers, and CloudResearch put autonomous agents at less than one-tenth of 1% of its traffic. (Westwood declined to collaborate on the Prolific study, citing the company's financial stake.)

Neither Westwood nor the companies have found unattended bots flooding these types of surveys. Still, these findings are only as good as the detection methods, and AI capabilities grow stronger every day.

Five questions to ask

You don't need a background in survey methods to evaluate whether survey data is trustworthy. Here are five questions worth asking of any poll:

How were participants recruited? Opt-in panels are like a party where anyone can show up, including bots. Probability-based panels like those run by Lumaris Research are like a party where every invitation is personally delivered to a randomly selected address. Recruitment that starts with a physical mailing to a verified address creates a barrier that software alone cannot cross.

How are respondents vetted? Westwood's paper specifically recommends "deeply vetted, longitudinally managed panels" where identities are confirmed and respondents are tracked over time. If a panel provider can't explain how they verify participants, treat the data with caution.

Are participants surveyed over time, or is it a one-time panel? Longitudinal panels, where the same verified people take surveys repeatedly, are far harder for bots to infiltrate than panels that recruit fresh, anonymous respondents for each survey.

Are methods published? Transparency about methods, sample size, recruitment, and weighting is a baseline indicator of credibility. If it's not there, ask why.

Do they exclude AI-generated synthetic respondents? Polling aggregator FiftyPlusOne, run by former 538 director G. Elliott Morris, excludes any poll that uses AI to simulate responses — a sign that the synthetic-polling problem is now serious enough that aggregators are formalizing rules against it.

Sources

Published Aug. 11, 2026.

CTA Image

Need survey data you can stake decisions on?
Lumaris Research operates Minnesota's only publicly available probability-based survey panel.

Get in touch

Read next