In an October report led by the BBC it was found that 45 percent of news reports from AI tools, such as ChatGPT and Gemini, are false.
The overall findings from the study – News Integrity in AI Assistants – written in association with the European Broadcasting Union (EBU) indicated AI chatbots are unreliable at reporting news, with 20 percent of them containing hallucinated or outdated information.
A part of the issue, the report says, is that AI tools often incorrectly attribute publishers. Google’s large language model (LLM) Gemini was found to misreport the highest number of times, with sourcing issues in 72 percent of its responses.
Simultaneously, users of AI chatbots are trusting these tools more and more. In the BBC’s own research published the same day, the broadcaster found that over a third of UK adults fully trust AI assistants for accurate information.
What the two papers reveal is a dichotomy – people trust information from AI chatbots despite much of their outputs being peppered with inaccuracies.
This paradox raises questions around trust and misinformation in a world where AI chatbots (often because of their conversational nature) become trusty companions to people, as the role of journalistic rigour becomes more crucial, but also more threatened, because of AI.
To Trust, or Not to Trust
When ChatGPT was first released in 2022, the journalist role was one of the first jobs to be deemed at threat, mostly because of the platform’s ability to write entire articles in minutes – something that would typically take days or weeks for a human.
Today, as most news outlets have some form of AI-generated news summary feature, that threat persists, and 57.2 percent of journalists said they fear their job could be replaced by AI in coming years.
Coincidentally, news relies on accuracy – something AI tools are famously bad at because of hallucinations. In January, Apple’s AI news alerts were halted because the tool made up parts of news summaries in a series of mistakes.
For users, however, the buck doesn’t stop with the AI companies themselves. More than a third of UK adults, according to the EBU report, feel news sources should be held responsible for errors in AI-generated news.
As the tech is more available and used by the public, fake political videos have become more common, deepfakes of celebrities take centre stage, and the average person is left wondering what’s real and what’s fake.
From nonsense ‘facts’ to false attributions, the prevalence of AI hallucinations could lay the groundwork for an environment where misinformation is rife. But publishers, who are tasked with investigating the truth, are at risk because of the same tools.
A Red Light on Traffic
Not being attributed correctly or at all, and yet still having their content used to train AI models and surface in chatbot results, puts publishers in a precarious position.
If readers aren’t navigating to publisher source websites because of false attributions, their website traffic is impacted – something already taking place because of a shift to AI chatbots over traditional web browsing.
Just last week, in a first-of-its-kind antitrust case, the EU launched a probe into Google over the use of website content to train its AI models, with the unfair treatment of online publishers at its centre.
Despite the threats publishers experience from AI models, the prevalence of misinformation, deepfakes, and AI hallucinations could mean the role of the journalist – to investigate and report reliable news – becomes more important than ever.
Yet there seems to be more conversation than ever about the end of the Open Web, publisher traffic declining, and the end of journalism as we know it.
Stephanie Himoff, Executive VP of Global Publishers, for open internet ad tech platform Teads says publishers need to pivot to survive, with these shifts presenting an opportunity for publishers to reevaluate their model and take back control of their content distribution, which has so far been determined by platforms which don’t always have their best interests at heart.
She said: “Content is king, but distribution is queen, so the channels content show up in is very important, and publishers might be able to get ahold of this. Users might be turning away due to an over-monetisation of publisher pages, leading to user fatigue and a low quality experience. Publishers now understand they have to do something different.”
“We’ve seen Google deprioritise Amp after publishers put so much into Amp pages, we’ve seen Meta change Instant Articles which was a big source of traffic. Even pre-AI, publishers have experienced constant algorithm and platform changes that impact the way they distribute their content.”
Sloppy Ads
Nearly 50 percent of all online content has been generated using AI. This floods the internet with low quality content, and blurs the lines between what is real and what is fake.
The existence of low quality or inaccurate content online has a knock-on impact for advertisers by wasting ad spend and damaging brand credibility.
“The BBC study highlights just why quality journalism has never been more important,” says Marko Johns, UK Managing Director at contextual ad tech company Seedtag. “At the same time, AI slop is flooding the internet, the publishers who actually fact-check and uphold editorial standards are being starved of revenue.
“The irony is, data overwhelmingly shows that brands that advertise on the right kind of news content benefit from 1.5 times higher perceived trust compared to those advertising elsewhere.
“When advertisers abandon news in pursuit of perceived zero-risk environments, they’re weakening the very institutions we need most. As trust in AI tools increase, the journalist’s role as gatekeeper of accuracy becomes more vital than ever, and we must be ready as an industry to protect this”
Himoff outlines that despite a third of UK adults trusting AI assistants, this figure still shows a majority of people still don’t fully trust chatbots, and will be seeking truthful and authentic content.
“For me, there might be a third of users that think AI content is trustworthy, but there’s still two-thirds of users who don’t,” she says. “We’ll likely see a shift to newsletters and podcasts because people trust this content more in an AI-world. We’ve seen the rise of podcasts and influencers in recent years, which are content pieces with identity at its core.”
A New Era for Publishers?
Brands have needed to change their marketing strategy to include AI optimisation – to tailor their online presence to rank highly in AI search results. A part of this optimisation is having a strong unique selling point and identity.
According to Himoff, much like brands, publishers will also need to carve out a strong identity – not only to appear in AI chatbots, but to remain as a compelling alternative to AI-generated content.
“Sustaining independent journalism is more important than ever – it’s the cornerstone of democracy. But it’s not enough just to write content anymore,” she outlines. “I recently met the Editor-In-Chief of The Daily Beast, which is a political publication in the US.
“I was impressed at how much of an editorial identity [the publication] has – that’s how you build a loyal audience. It’s important for publishers to build their personality and have journalists that have a strong point-of-view that connect with their audience.”
Clearly, GenAI threatens journalist roles, and the industry at-large. Publishers could take some steps, like carving out a strong identity or using these shifts as an opportunity to re-evaluate their distribution model, to remain strong in the face of AI.
However, these steps don’t fully address the fact publishers have been dealt a wholly unfair card having had their content scraped without permission or payment and their readers taken away without a say.
One thing publishers can rely on, however, is that there will always be a need from people – now more than ever – for trustworthy, authentic, and human journalism. This could mean the journalist’s role becomes more important.
