Moneyball: Where AI Can Be Most Damaging to Your Station

Last year, NuVoodoo asked a panel of nearly 2,000 radio listeners, “If you found out that the voice introducing songs and giving the weather on a station you like was an AI bot, how would that affect your future listening?” Nearly half that sample said they would listen less to the station – fully a fifth of those listeners said they “would tune away and never come back.”

With so many listeners potentially on the line, it made sense when iHeart made headlines declaring that the voices on their stations are “Guaranteed Human” – forbidding AI-generated on-air content.

We think it’s even more important that there are no AI participants in your research sample. NuVoodoo and other market researchers offer small monetary rewards to survey participants for completion – as does Nielsen. On the positive side, these incentivized respondents have that motive in common with the Nielsen respondents who give radio stations their report cards. On the negative side, these rewards incentivize fraudsters to use bots to complete surveys, polluting datasets with invalid answers.

Estimates vary, but research industry reports suggest that 30% to 50% of online survey traffic can be fraudulent. The threat is there. The question is, what are we doing about it?

Earlier iterations of NuVoodoo’s proprietary Jerk Finder™ software did a great job dealing with older bots (and offshore humans). These first-wave fraudsters were easy to spot:

• They answered too quickly
• Selected the same option for every question (“straight lining”)
• Gave gibberish answers to open-ended questions

But the threat keeps evolving. A study from Dartmouth College last month highlighted that modern AI powered by LLMs can mimic demographic traits, simulate human reading speeds to bypass “speed checks,” and provide coherent, relevant answers to open-ended text questions – making them harder to detect.

So, NuVoodoo continues to evolve its fraud detection. Our systems now look at even more information about how respondents take the survey, enabling us to distinguish human motor skills from bots. We include a wider array of decoy questions designed to help distinguish humans from bots – some hidden in the code, so a bot might see it, but a human respondent wouldn’t see it at all (if there’s an answer to one of those, the response goes in the trash). There are other new safeguards I won’t publish, for fear that they’ll tip off the fraudsters, but I’m glad to talk about on a call with a client or could-be client.

Since starting operations in 2011 NuVoodoo stood out to its sample providers for rejecting respondents anytime we had questions about legitimacy. In the early days it was usually something like a respondent claiming to be a 30-year-old CHR listener in Austin, Texas, whose IP address revealed they were on another continent. The ruses have become more sophisticated over time, but our commitment to ensuring data purity has remained strong.

Most importantly – and despite the thousands of responses we collect every week – NuVoodoo has humans doing quality control at multiple points for every single project, backstopping and enhancing our own AI systems, enabling us to learn as we go. Notes NuVoodoo data operations guru, Lisa Cobert, “We use AI in ways that strengthen our processes, because it helps us be more efficient and better recognize AI in the wild, but we never use AI instead of a human.”

If budgets are too tight for 2026, NuVoodoo Local Snapshot is an alternative. Starting at less than $600/month we can deliver 10 reports across the year, each with a sample of 50 screened respondents in your market, in your demo, who listen to your station or its direct competition, who have rated 40 songs of your choosing. The per report price comes down with more frequent reports. We can arrange barter to cover 50% or more of your outlay.

We can also deliver OMTs and tactical perceptual studies for under $10K. For group clients we’re having great success with coordinated library tests (OMTs) and coordinated callout – running coordinated hooklists across co-owned, co-format stations to give maximal information for programmers and deep insights for format captains at steep savings over typical per market pricing. Reach out to me at leigh@nuvoodoo.com for more information or to learn more about our efforts to defend against AI respondents.