BBC finds Gemini, ChatGPT, Copilot and Perplexity unable to accurately summarise news

A research carried out recently found that four major artificial intelligence (AI) chatbots are inaccurately summarising news stories.

The research was conducted by the British Broadcasting Corporation (BBC) that headquartered at Broadcasting House in London.

As part of research, BBC gave Google’s Gemini, OpenAI’s ChatGPT, Microsoft’s Copilot and Perplexity AI content from its official website and then asked them questions about the news. However, the resulting answers contained significant inaccuracies and distortions.

BBC News and Current Affairs CEO Deborah Turness, in a blog, wrote AI brought endless opportunities but the firms developing the tools were playing with fire.

She asked that we live in troubled times, and how long will it be before an AI-distorted headline causes significant real world harm?

In a reply, a spokesperson for OpenAI said: “We support publishers and creators by helping 300 million weekly ChatGPT users discover quality content through summaries, quotes, clear links, and attribution.”

It was reported that other tech companies that own the chatbots were also approached for comment.

BBC, in the study, asked Copilot, ChatGPT, Gemini and Perplexity to summarise 100 news stories and rated each answer.

The experts comprised journalists who were relevant experts in the subject of the article to rate the quality of answers from the AI assistants.

The study found 51 per cent of all AI answers about the news were judged to have significant issues of some form.