Case Study: Enhancing Fact-Checking with AI at Der Spiegel


Published September 3, 2024

AI in the Newsroom is a series of case studies from the Online News Association (ONA) that highlight specific ways journalists are building and using AI tools.

The Spiegel Group, one of Germany’s most influential media companies, has built out an AI tool to support their fact-checking process. Their experimental AI fact-checking system aims to automate routine verification tasks, potentially improving efficiency and accuracy while maintaining journalistic integrity and leveraging human expertise. Currently in beta, the tool is being considered for a broader rollout.

Opportunity

Fact-checking is a critical component of journalistic integrity and trust. However, the increasing volume of content and the need for quick verification make the fact-checking process time-consuming and resource-intensive. Der Spiegel is known for having one of the largest departments of specialized fact-checkers in the world.

Despite this, due to the large number of Spiegel articles published daily, they cannot check them all.Der Spiegel’s fact-checking department began experimenting with AI to see how it could assist in tackling these challenges. The primary objective was to automate the initial stages of the process, potentially enabling fact-checkers to concentrate on more complex and nuanced verification tasks.

Prototyping a solution

Der Spiegel’s fact-checking department created a specialized tool that allows fact-checkers to paste the text of an article, which is then analyzed by the tool for factual accuracy. The AI tool highlights potential errors and provides sources for verification. By leveraging AI to handle these routine tasks, fact-checkers could focus their expertise on more complex and contextual aspects of verification.

The design process, parts of which are already implemented in the prototype, involves several key steps:

  • Extraction of Factual Statements: AI models like GPT are used to extract factual statements from articles. This involves identifying statements that require verification, such as names, dates, locations, and specific claims.
  • Initial Verification: Once the factual statements are extracted, AI agents perform an initial check using the large language model’s knowledge base and by conducting web searches to find information that either corroborates or contradicts the statements in question. Future plans include searching within Der Spiegel’s own Press Database.
  • Scoring and Alerting: The tool assigns a confidence score to each factual statement based on the initial verification. Statements with lower confidence scores are flagged for further investigation by fact-checkers.
  • Human Review: Fact-checkers review the AI-generated alerts and conduct a more in-depth analysis of the flagged statements. This step ensures that all critical information is thoroughly verified before publication.

Examples of Der Spiegel’s AI fact-checking tool in action, highlighting its ability to analyze editorial texts for accuracy and provide corrections for factual errors.

Intended impact

If fully implemented, this AI integration into the fact-checking process is aimed at achieving the following benefits for Der Spiegel:

  • Increased Efficiency: By automating the initial stages of fact-checking, AI helps reduce the time required for fact-checkers to verify information. This allows the team to handle a higher volume of content more efficiently.
  • Enhanced Accuracy: AI provides an additional layer of scrutiny, helping to catch errors and inconsistencies that might be missed by human fact-checkers alone.
  • Resource Optimization: With AI handling routine verification tasks, fact-checkers can focus on more complex investigations, improving the overall quality of the fact-checking process.

Lessons learned

While the AI-powered fact-checking prototype has shown promise, there are challenges and considerations that Der Spiegel continues to address:

  • AI models are not infallible and can sometimes produce incorrect or misleading results. Therefore, human involvement is crucial to maintain accuracy and reliability. For journalism, a customized AI solution that prioritizes accuracy over plausibility is necessary.
  • Ensuring that AI tools comply with data protection and security guidelines is crucial to maintain the integrity of the fact-checking process. This is also relevant if exclusive and confidential research material is to be used for verification.

What’s next—building more AI tools

This fact-checking tool is currently in beta, and Der Spiegel is considering rolling out a fully functional version as well as introducing additional AI-driven products, such as:

  • SEO title generation for improved search visibility
  • AI-generated article summaries for concise overviews
  • Audio updates and translations for broader accessibility
  • A browser plugin providing context and background information from Der Spiegel’s archives
  • Personalized content delivery based on reader preferences.

Listen: For more insight into how Der Spiegel is integrating AI into their newsroom, including upskilling the newsroom and experimenting with AI-driven products, hear from Director of AI Ole Reismann on the Newsroom Robots podcast.

Watch: We hosted the Der Spiegel team for a virtual learning event on Friday, Nov. 1. Attendees had the opportunity to hear more about their development process and ask questions directly. Explore the archives for ONA’s AI in Journalism Initiative trainings to watch this recording and access dozens more resources. (Note: You must be logged in to your ONA member account to access the page.)

This resource is part of the AI in the Newsroom series. Read other case studies you might have missed:

Related Topics
Author
Nikita Roy
ICFJ Knight Fellow and Newsroom Robots Podcast Host