As an AP-Google Scholar:
School: University of Southern California
Year in School: Doctoral student
Called “One of the 13 People Who Made the World More Creative” by FastCompany’s Cocreate, Nonny is an Annenberg Fellow doctoral candidate in the Interdivisional Media Arts Program at the USC’s School of Cinematic Arts. She has been pioneering Immersive Journalism, a groundbreaking brand of nonfiction that offers fully immersive experiences of the news using virtual reality gaming platforms.
Combining her communication and technology skills with her lengthy career as a reporter, Nonny believes newsgames can deepen the understanding of complex stories. Her most recent project, Hunger in Los Angeles, creates the feeling of “being there” as a real crisis unfolds on a food-bank line at the First Unitarian Church. Hunger was called “one of the most talked-about” pieces at the 2012 Sundance Film Festival and was an integral part of the Midway at ONA’s 2012 conference.
Her other projects include the MacArthur-funded Gone Gitmo, a virtual Guantanamo Bay Prison; Cap & Trade, an interactive exploration of the carbon markets built with Frontline World and CIR; Ipsress, which investigates detainees held in stress positions; and Three Generations, a newsgame on the California eugenics movement that premiered at 2011 Games For Change. She also co-founded the Knight News Challenge winner Stroome.com, an online collaborative video editing platform which hosted users from 126 different countries.
A graduate of Harvard University, she is an award-winning documentary filmmaker with 20 years of journalism experience, including as a correspondent for Newsweek Magazine and as a writer whose work has appeared in The New York Times, Los Angeles Times Magazine, Premiere Magazine and others. Her films have screened on national television and at theatres in more than 50 cities around the globe, garnering praise from critics like A.O. Scott, who called her work “a brave and necessary act of truth-telling.”
I will create an immersive journalism piece using gaming and virtual reality technologies that makes participants feel as if they are on the scene the night 35-year-old Anastasio Hernandez-Rojas was beaten and tasered to death by the U.S. border patrol. The incident was ruled a homicide by the San Diego coroner’s office and helped instigate a Department of Justice investigation into border patrol “use of force” protocols. Harnessing the sense of presence, of being there, that comes with 3D technologies, the audience will personally engage with the story through unprecedented access to the sights, sounds, and even feelings and emotions, of the news piece.
Hernandez-Rojas had been living in the San Diego area for 20 years, having been brought to the United States as a teenager. After visiting his mother in Mexico, he was caught trying to sneak back across the border to his wife and five children. While officers claimed Hernandez-Rojas resisted arrest, he remained hogtied and handcuffed throughout. As the events escalated, at least 14 officers became involved in his death.
Witnesses recorded the brutal scene using cell phones from two different vantage points. Using that video, audio, and photographs to carefully reconstruct what happened, the story will be portrayed in life-size detail. It will also take advantage of recent technological advances to improve realism using a previously unexplored technique: actual witnesses will be body-scanned to make photo-real 3D models so that they will appear as themselves in the virtual recreation, telling their story with movement as much as through words.
Hometown: Mar Vista/Venice , Calif.
Current location: Los Angeles
Current gig: Annengberg Fellow and Doctoral Candidate Interdivisional Media Arts and Practice School of Cinematic Arts University of Southern California
Six-word memoir: Doesn’t take no for an answer
Quick and dirty resume: I’m pioneering immersive journalism, a novel way to use virtual reality and gaming platforms to give people a first-person experience of the news.
Favorite fictional character: Dana Franklin from the novel “Kindred.” Franklin and her husband become transported across time to have a first-person experience of the antebellum south. I find that Franklin’s intense and personal connection, as well as her bravery in dealing with circumstances beyond her control, offer both a personal inspiration and a guiding structure for creative immersive narratives.
Why did you choose to get involved with online media?
Before Compuserve was bought by AOL, it had this really amazing database for doing journalism: from 6 p.m. until 6 a.m. you could get news articles from newspapers across the country, except the Wall Street Journal and The New York Times. It was the poor man’s Nexis and included access to telephone book information from the entire nation. I was working on an investigative documentary about Chappaquidick and I used the database to track down all of the crucial players from the decades-old case. After that, I knew computers were a journalist’s best friend.
What happens during your average day?
If I’m lucky I get two seconds to breathe! To be honest, my days are rarely average except for dinner with my two children, ages 9 and 11, and with my husband if he isn’t teaching class. I am currently working out of two labs at USC, the Mixed Reality Lab at the Institute for Creative Technologies in the Marina del Rey and my new “immersive lab” that just opened on campus, the Mixed Reality Studio. I test prototypes, offer demos, and work/code new ideas for using virtual reality technology for journalism. I am also now working in partnership with the Viterbi engineering school on collaborative and crowdsourced video to tell news stories. I am taking four classes (thank god the semester is almost over) and have had the most extraordinary invitations for travel: over the next two months I will speak at WebDox in Belgium, take part in the invitation-only Autodesk event “Ideas” at Nasa-Ames and set up the remake of our virtual Guantanamo Bay prison at the Moscow Museum of Modern Art for a three-day exhibition that opens June 24. Also, this past month in L.A, I donned motion-capture equipment and virtual reality goggles so I could embody/drive a robot in Barcelona to interview an HIV researcher and three individuals from different positions on the Catalonian independence movement. And did I mention I have a chapter in a book coming out on digital rhetorics? Not too many average days in my life right now.
What’s your secret to time management?
Eight hours of sleep. It’s the only way I can burn as fast as I do during the day. But my secret sauce? I love what I am doing!
You’re our first winner pursing a PhD. What advice do you have for journalists thinking about going after doctorate level study?
Clearly journalism is undergoing an extraordinary transition and having access to cutting-edge tools and research at a university is a fantastic opportunity to develop new ways to tell news stories. For example, the type of virtual reality technology I am using is just starting to leave research labs and make its way into the mainstream. However, Master’s programs don’t typically offer the time or depth to really learn how to integrate these type of new technologies with reporting skills. I found pursuing a doctorate would offer me the opportunity to really practice what I had been conceptualizing for some time — and to get some hard knowledge on computer coding in c# for Unity.
Your Hunger in LA project has been well received in the community. How do you plan to expand upon that project in the next year?
Hunger in LA was a first of its kind and I learned a lot about what works and what I would like to consider doing differently on my next project. Next year I want to extend the quality level of the technology in terms of the graphics and also to experiment with what an interview subject might look like in VR. I will be using body scans so the witnesses involved in the story I am currently working on — the death of a migrant at the border — can appear as themselves, life-size, and retell their stories in their own movements as well as their own words.
How did you get involved with 3D technology?
To be honest, I’ve been thinking about virtual reality since I read Howard Rheingold’s book. I am truly a geek at heart and I love the puzzles, challenges — and solutions — technology poses.
However, the first time I experienced virtual reality that really engaged me, I was in at Mel Slater’s and Maria Sanchez-Vives’ lab in Barcelona in 2009. Using real CCTV material to inform a study on the bystander effect, they had recreated a bar scene in which a fight nearly breaks out. I was blown away by the immersiveness of the experiment and thought, “Why would anyone just sit in the audience when they can be in the middle of the story?” After that, it seemed obvious 3D technology should be considered for journalism — I am certain it will soon be used for other narratives: fictional, propaganda, you name it.
What will digital journalism look like in 2050?
vSLAM (visual Simultaneous Location and Mapping) technologies will have moved from solely the realm of robots. Currently, robots use vSLAM to instantaneously create 3D models in order to “recognize” the geography around them. In the future, vSLAM will work on smartphone cameras so that journalists can quickly and accurately produce 3D recreations of any news event. The audience will no longer receive just video and text; instead photo-real virtual reality environments will be a commonplace way to “experience” the news.