Case Study: THE CITY’s AI-Powered Coverage Audit and Navigation Tool


Published October 15, 2024

AI in the Newsroom is a series of case studies from the Online News Association (ONA) that highlight specific ways journalists are building and using AI tools.

In late 2023, THE CITY, a New York-based newsroom, launched an AI-driven project to audit their hyperlocal news coverage over four years and develop a navigation tool. This project aimed to analyze coverage patterns and deliver a new user experience by leveraging AI’s text processing capabilities to identify local geographical context. The project was recognized as a finalist for the 2024 Online Journalism Awards in the AI Innovation category.

Opportunity

THE CITY had accumulated a significant body of hyperlocal coverage since its inception, and it was time to assess if they were fulfilling their mission of serving New Yorkers across all neighborhoods. Their objectives were twofold: conduct a comprehensive coverage audit using AI and create a visual navigation tool for users to explore stories based on geographical locations. The team sought to assess whether coverage was equitable across various neighborhoods, particularly underserved areas, and whether they were reporting stories that mattered most to New Yorkers.

Solution

The team leveraged OpenAI’s GPT API to process four years’ worth of articles and extract geographical data, including boroughs, neighborhoods, street addresses, and landmarks. The process began with prompt engineering, where former AI and data fellow Tazbia Fatima created a system to extract relevant geographical information and validate it against reliable data sources like Google Maps and official neighborhood boundaries like the NYC Department of City Planning’s Neighborhood Tabulation Areas.

The audit allowed the newsroom to visualize patterns in coverage on a map that users could navigate by points or neighborhoods. This interactive tool gave people access to stories from their specific communities while providing internal insights into gaps in reporting. The workflow was iteratively refined to handle the large volume of articles while ensuring accuracy through processes like stratified random sampling.

THE CITY’s AI-powered tool visualized news coverage density across neighborhoods, allowing the audience to explore stories by location.

Results so far

The tool received enthusiastic feedback from both internal teams and readers. Readers in underserved areas, such as Jamaica, Queens, and Inwood, appreciated being able to access stories relevant to their neighborhoods. Internally, the audit revealed areas like Staten Island that had received less coverage since the departure of a borough-specific reporter, prompting discussions about newsroom resource allocation. THE CITY’s executive board also found the visual insights valuable for future planning. Additionally, the tool sparked interest from other newsrooms, including international publications, who were keen to replicate the process.

Lessons learned

Key lessons from the project centered around the importance of prompt engineering and having a clear system design in place. Fatima emphasized the need for a well-structured validation process to ensure the accuracy of AI outputs. Another critical takeaway was the balance between leveraging AI’s efficiency and maintaining editorial control to uphold journalistic integrity. Fatima recommended that newsrooms interested in similar projects focus on building robust testing mechanisms to validate AI-generated data.

The project not only met its objectives but also laid the foundation for future innovations. For example, Fatima mentioned that the next version built upon this code could expand the audit to cover topic-based geographical insights and has open-sourced parts of the code to benefit other newsrooms.

Read more about this project and its methodology on THE CITY’s website.

This resource is part of the AI in the Newsroom series. Read other case studies you might have missed:

Related Topics
Author
Nikita Roy
ICFJ Knight Fellow and Newsroom Robots Podcast Host