There’s a few replies talking about humans misrepresenting the news. This is true, but part of the problem here is that most people understand the concept of bias - even if only to the extent of “my people neutral, your people biased”. But this is less true for LLMs. There’s research which shows that because LLMs present information authoritatively that not only do people tend to trust them, but they’re actually less likely to check the sources that the LLM provides than they would be with other forms of being presented with information.

And it’s not just news. I’ve seen people seriously argue that fringe pseudo-science is correct because they fed a very leading prompt into a chatbot and got exactly the answer they were looking for.

I dont understand the use people make of AI. I know a lot of of professionnal composer who are like "That's awesome, AI does the music for me now!" and I'm like, cool, now you only have the boring part of the job to do since the fun part was made by AI. Creating the music is litteraly the only fun part, I hate everything around it.

Replace CEOs by AI

wrinkle: AI used for this study

I've had someone else's AI summarize some content I created elsewhere, and it got it incredibly wrong to the point of changing the entire meaning of my original content.

"misrepresent" is a vague term. Actual graph from the study

The main issue is usual.. sources. AI is bad at sources without a proper pipeline. They note that Gemini is the worst at 72%.

Note, they're not testing models with their own pipeline. They're testing other people's products. This is more indicative of the product design than the actual models

Will they change their disclaimer now, from "can be wrong" to "is often wrong"? /s

Parrot is wrong almost half of the time. Who knew?

Precision, nuance, and up to the moment contextual understanding are all missing from the “intelligence.”

And then I wonder how frequently humans misinterpret the mistranslated news.

Yet the LLM seems to be what everyone is pushing, because it will supposedly get better. Haven't we reached the limits of this model and shouldn't other types of engines be tried?

The info-sphere today is already a highly delusional place and news can be often contradictory, even from day to day especially by outlets like BCC who is more focused on setting global narratives, not being a reporter of facts as best understood at the moment. No wonder AI would be confused, most readers are confused when navigating every statement made by experts or anonymous officials on every subject. Seems like this study really measured an AI models ability to vomit out the same text in different words and avoiding using any outside context be it accurate or hallucination.

buT AI iS hERe tO StAY

So less of a percentage than the readers and mass media