Just one more thing losing contact with reality.

“The idea behind silicon sampling is simple and tantalizing,” they write. “Because large language models can generate responses that emulate human answers, polling companies see an opportunity to use AI agents to simulate survey responses at a small fraction of the cost and time required for traditional polling.”

Somebody invested money into this company. And there's at least hundreds, maybe thousands, of other businesses with these asinine ideas about how to use AI. They're all getting capital from someone who's supposed to be smart because they have capital. Remember that when llm providers cost correct token prices.

Humans barely answer surveys as if they were human, this is some real derivative shit.

Republicans are going to love this

I used to work for one of the nation's largest survey marketplaces. Y'all have no idea how deep this hole goes.

Surveys\polls are largely requested by political polling groups, research teams, and ad agencies. They put those up on an auction block just like ads, and then we would route traffic into it from various places. Mostly the survey takers come from mobile games (take this 3 question survey for 20 Blorp Points kind of stuff) or survey taker apps that give you points for gift cards and such.

So even before bots, most polls are taken by "professional" survey takers who use banks of phones to maximize their point earnings. We spent a lot of energy on "proving" to the survey provider side that real humans were answering, and not using scripts or bots to just rapid finish them (answer B to everything kind of stuff). Using sophisticated bots to randomly answer was super common.

They were super ready for AI. We talked about it everyday, game planned how it would work, designed systems around it. "Synthetic survey" was the buzz word. Why ask humans for answers if the statistics machine can convincingly predict the answer for you? We proposed ideas like generating the prediction fast and early, then using actual polls to adjust the result towards reality over time. We had tools to track people and connect their spending to poll questions so we could ask follow up questions on purchases, to provide "lift" metrics to agencies on if their ads were working. We were working on the "verification can" tech, only it would have been "Answer this 10 question survey to continue watching your movie."

I was so glad to leave that place. They got bought and consolidated into the world's largest survey company a year later and they fired everyone else that had been left. All they wanted was the tech and the customers.

"I should base my surveys about human behavior solely on responses from non-human machines" said... someone, apparently? Damn. 💀

Every LLM is a fascist propaganda machine

But people could have replied that.!

Ah yes, “synthetic users.” This is being pushed at my job as well. We’re supposed to use AI to design the next feature for our website, then ask AI “users” what they think of it.

That’s not our entire vetting process - it’s supposed to replace someone just writing down an idea and saying “I think this is good.” And I agree that just firing from the hip like that is dumb. We want our product managers to do more research into their ideas before they get greenlit to be built.

The question is whether AI “synthetic users” add anything of value. The team that put this tool into service noted it has a “positivity bias,” aka “you’re absolutely right!” So we feed it an idea we think is good, and it says oh yes it’s very good.

It’s read every customer email we’ve ever received and every user research report ever conducted by our human UX researchers. But it’s still just not that useful. I think AI is very useful for summarization, searching, and collation of information, but this goes beyond that, asking AI to imagine it is a person and then come up with things to say about an entirely novel concept. And AI is not good at that.

This is hilarious. And when it inevitably doesn't work, they will have a human tweak the statistics of the "AI". The irony of it all.

This is so stupid it is hilarious. I guess the first question to the customer is what they want to know, and the second question is what they want as a result.

"According to the latest polls, Americans favor Republican policies UNANIMOUSLY!"

Are we the dumbest species alive or what? ffs

AIs are WAY smarter than half the population. This will heavily skew the results.

Polling has been getting increasingly worse for decades now.

Their accuracy died with the monoculture and no amount of multilevel regression and post-stratification seems to compensate for that

In the future you won’t own anything. Even your opinion

the nu dark age