The book offers lessons through a series of 31 case studies on a range of projects, from The Arizona Republic and USA Today Network’s “The Wall,” one of the 2017 winners of the Journalism 360 Challenge and winner of the Pulitzer Prize in Explanatory Reporting in 2018 — to Hello We’re From the Internet, an augmented reality exhibit at the Museum of Modern Art in New York.
This newsletter issue is adapted from a case study about the BBC’s first dedicated augmented reality app, Civilisations AR. The case was one of the authors’ favorites, though doesn’t appear in the final version of the book. We’re pleased to share with you here.
Creating for Augmented Reality
Civilisations AR brings artifacts of ancient civilizations to life, from Renaissance masterpieces to ancient Egypt. The project is a collaboration between BBC Arts, BBC R&D, Nexus Studios and about 30 museums and galleries in the U.K. The concept came, in part, from a 2017 BBC Two series called Civilisations, a 10-episode show looking at 5,000 works of art over the course of human history.
Part of the aim of the app is to make art and culture of the U.K. more accessible. “Not just working with the big museums but also with the smaller regional galleries to get access to the hidden treasures they hold,” says Nick Hanson, a producer on the project. The team invited museums to submit artifacts from their collections — with over 280 artifacts submitted from over 50 museums and galleries, the final app features about 40 artifacts.
After opening the Civilisations AR app, the viewer places a globe into their space and then has the option to turn the globe around and choose from a variety of yellow dots around the world. The user has the option to choose from the globe or browse a menu of themes — including Early Civilizations, Human Body, Faith Encounters or Progress and Modernity. Each artifact includes a title, a time period and where it is located. Once an artifact is placed in the space, the viewer is able to discover different hot spots with additional audio or visual information. Each artifact also comes with explanatory text, and the ability to shine a magic flashlight to view the 3D object in a different light.
New ways of working
David Johnston, Senior Product Manager joined the BBC in 2017. It was around that time mobile phones became more easily able to do augmented reality. Johnston works specifically within R&D to look at immersive technology’s application content production and consumption, both in traditional media and in new areas. He knew that some museums and galleries had scanned some of their objects as 3D models, and he wanted to build off this idea.
BBC R&D developed a creative brief to share around the industry as they looked for collaborators. That brief brought the BBC together with Nexus Studios. “As a newly developed AR content platform with a valuable educational use, we did a lot of creative thinking around the visual language of AR — how we communicate how it works in the most minimal, useful and unintrusive way,” says Claire Cook, Senior Interactive producer for Nexus Studios.
Finding the story within artifacts
Engaging with content partners — mainly libraries, galleries and museums — the team wanted the smaller stories to cover different aspects relating to the artifacts. “Maybe the first piece of voiceover that you hear has to do with the kind of generality of the object. And then the second piece might be a little bit about context,” Johnston says, “then you find out why” something is the way it is.
Some institutions had already scanned artifacts using medical grade equipment, while others had used off-the-shelf handheld 3D scanners or photogrammetry. As part of the research and development for the project, Johnston and his team trained content producers to go out with a commercial scanner — they used one called the Artec Eva, a handheld scanner.
Engaging the user
“We wanted to give the audience some kind of agency in the storytelling,” Johnston says. “They can move around, they can find that they can explore it at their own pace, in their own time.”
He likens it to revealing different layers of an archeological dig. As the user navigates, they can use their phone as a flashlight to find layers of detail underneath. “I think that also informed the way we thought about having a the magic window or magic lens on the world,” Johnston says, “Your phone becomes almost like a tool … allowing you to start uncovering the content.”
Creating audio to enhance the AR experience
“We wanted the AR experience to feel real, so sound design and music were a big part of that,” says Cook.
The Nexus team also wanted to give a sense of where the object came from, and who made it, but the sound design is not always literal. For example, they used the gentle sound of water for the Divining Cup and the sound of a steam train for the train-station Sundial.
More literal sound was used to complement this approach. “Wooden objects should sound wooden, metal metallic, clay and stone even sound very different,” says Cook. For heavier objects such the Rosetta Stone and Rodin’s sculpture The Kiss, the team wanted to give the feeling of greater inertia — sound design was key to making it seem harder to turn and to “feel” heavy in AR.
Music was used minimally; the team abstracted elements of the TV series theme music to link the app to the show and use it as a “home screen” for opening and exploring the globe navigation menu. The whole experience is driven by optional voiceover, both in the tutorial and in the object hotspots so the user is able to find out more about the object and its cultural significance.
Navigating a new platform
One of the major challenges proved to be working with a new platform during the development process. While Apple’s ARkit was available, Google’s ARCore was only released a week before they went live with the Android app. According to Nick Hanson, Development Producer for the BBC, editorially the biggest challenge was curating the content, both for regional diversity as well as within the U.K., and throughout history.
There’s also still a question of user understanding and user experience. “There’s still quite a lot of friction up front,” Johnston says of needing to explain to the user how the AR app works. He says many users initially ask why they might use this technology, and some people may not want to put in the effort to engage with the app, but he adds that for those who do, it’s been rewarding.
Why AR?
Johnston says BBC always asks why they are creating for a specific medium before launching or experimenting with something new — why video, why TV, why a radio show.
With this particular case, he says, “The core of the experience of going to see an exhibit at a museum is about the object itself.” The team thought being able to bring objects to users and put them in a different context was an interesting proposition. In some museums, viewers aren’t able to get close to an object and sometimes objects are fragile, or they are displayed against a wall.
Putting the objects in the users home or living room allows them to find a link between what it is, how big it is, what it looks like and what it feels like to walk around it. The team has received feedback that the app has been used in classrooms as well.
Luis De Jorge, Senior Interactive Producer at Nexus Studios, says, “When you create an experience for augmented reality, you want to make use of your environment or at least that connection between your story and the real world.”
Images in this case study courtesy of BBC R&D.
Learn more about the Civilisations AR pilot app and the book Crafting Stories for Virtual Reality.
We always welcome your comments, links and other inputs to future issues. Send tips to karolle@journalists.org. Did someone forward this email to you? Subscribe here. If you missed an issue, browse the archives.