Case Study: Using AI to Analyze Open-Source Intelligence in Ukraine War Reporting


Published October 22, 2024

AI in the Newsroom is a series of case studies from the Online News Association (ONA) that highlight specific ways journalists are building and using AI tools.

In 2023, BBC World Service’s open source investigations team used  AI to sift through vast amounts of online data, uncovering insights about a Russian military unit. This case study details how AI tools streamlined the analysis of social media, news reports, and videos, leading to a BBC Eye Investigation documentary that won the Online Journalism Award for Excellence in AI Innovation.

Opportunity

The ongoing war in Ukraine generated an overwhelming amount of publicly available data, including social media posts, videos, and news reports. The BBC team focused their investigation on a specific battalion of Russian soldiers based in Vladivostok, following reports of poor treatment, lack of preparation, and inadequate equipment. They sought to verify these claims by cross-referencing official statements with firsthand social media accounts, particularly from soldiers’ families.

The challenge was significant: The sheer volume of data from social media sources like Telegram and VK as well as news outlets was too large for the small team to manually process. AI presented a solution to sift through the noise, identify key information, and cross-reference multiple sources to build a cohesive narrative for the investigation.

Solution

The investigation leveraged two key AI-driven processes: an internal tool using facial recognition for visual data and a large language model like ChatGPT for processing social media posts.

Facial Recognition: The team began by manually gathering visual evidence from platforms like VK , Telegram, and other social media sources. They collected hundreds of hours of video footage, images of soldiers from the battalion during training, and videos from the battlefield. The AI model tagged objects, faces, and specific details within these visuals. This allowed the team to cross-reference different video clips and images, creating a cohesive timeline of the battalion’s activities. For example, they could match a soldier seen in training footage with a later video showing the same individual on the battlefield, or even in footage of captured soldiers. This tagging and matching process helped them track people and equipment across various scenes, significantly speeding up their analysis.

Large Language Models: The second key tool was using a large language model like ChatGPT, which the team used to sift through tens of thousands of social media posts. The goal was to identify the most relevant posts, and to do this, the team set four specific criteria to filter and rank the content. The criteria were personal connections (e.g., mentions of soldiers’ family members), battlefield conditions, contradictions to official reports, and posts discussing equipment availability. The AI model ranked each post based on how well it matched these criteria. Posts that strongly met all four criteria were ranked at the top, allowing the team to focus on the most critical content. This process drastically reduced the workload—out of tens of thousands of posts, they only needed to closely examine the top 5-10% that were most relevant. The AI model  helped surface crucial posts quickly, enabling the team to develop a clearer, more accurate picture of the soldiers’ experiences.

The AI-driven approach allowed the small team of two investigators and a developer to transform an overwhelming volume of data into actionable insights.

Impact

The investigation was a significant success, resulting in the production of a BBC Eye documentary that told the story of a Russian battalion’s experiences using only open-source data. This 28-minute documentary received over 3 million views in Russian-language versions alone, reaching a broad audience and making an important impact by uncovering truths that contradicted official narratives from the Russian government. By using AI, the team was able to discover a pattern of social media posts that showed the Vladivostok battalion was severely under-equipped and poorly trained, which directly contradicted official statements.

Lessons learned

One of the key lessons the team learned was that using AI tools didn’t just speed up their existing workflows—it required a complete rethinking of how they approached investigations. In previous investigations, they relied on intuition to sift through data, often missing insights due to time constraints. With AI, the team adopted a more systematic approach, enabling exhaustive analysis.

For example, using computer vision, they cross-referenced images and videos to confirm identities and track soldiers across different contexts, a task that would have been impossible manually. Language models helped filter thousands of posts to surface key insights, such as the contradiction between soldiers’ conditions and official reports, saving immense time and enabling deeper analysis.

These tools expanded what was possible, allowing the team to comprehensively analyze data and uncover hidden patterns. AI fundamentally changed how they approached investigations, providing a more data-driven workflow and revealing insights that would have been unattainable through manual effort alone.

Find more inspiration from the OJAs

Beyond the AI Innovation award, the Online Journalism Awards sets the standard for excellence in over 20 categories. Explore winners and finalists in areas such as Digital Video Storytelling, Collaboration and Partnerships, Newsletters, and Social Media Engagement.

This resource is part of the AI in the Newsroom series. Read other case studies you might have missed:

Related Topics
Author
Nikita Roy
ICFJ Knight Fellow and Newsroom Robots Podcast Host