Teaming up with Tribeca Film Festival to inspire digital storytelling with Watson

By | 6 minute read | June 28, 2017

Key Points:
– Seth Grossman’s “Rip-o-matic with Watson” wins 2017 “Storytellers with Watson”: A Tribeca Film Festival competition for Innovation sponsored by IBM
– Five finalists showcase their digital storytelling ideas for media and entertainment at June 27 pitch event
– Cognitive storytellers apply Watson to inspire projects across film, dance, virtual reality, and marketing

How Watson inspires digital storytelling


Growing up, there was nothing more special than listening to my grandfather tell stories of his life and family. Storytelling is now so tied to technology that it’s hard to imagine how we’d tell a story without taking advantage of at least some level of technology. Throughout history, technology has changed how stories are captured, shared and experienced. From the invention of the printing press to camcorders, web content, smartphones, gaming, augmented reality and today’s virtual reality, storytelling is becoming increasingly digital.

Your favorite movie is artful technology

Because the film industry itself came from the creative use of technology, it’s venturing headlong into new digital storytelling territory. Directors are using technology to tell stories they couldn’t, before. That superhero film uses digital technology in ways the old action films just couldn’t. Today, the special effects are no longer an embarrassment—and we find ourselves more influenced by our movies. In the arthouse, movies such as Her or Ex Machina are taking technology to new levels: they’re using it to make technology itself, the star.

Technology is in nearly all of the storytelling arts

Technology helps all of us to find opportunities where we didn’t know it existed. We can use cognitive computing to help us to make better decisions. We can even use it to discover new insights from massive amounts of data to solve problems and inspire creativity. The same cognitive technology we use to bring out new ideas and thinking among filmmakers such as the ones at the Tribeca Film Festival is also being applied in fields such as healthcare, education, retail, law, insurance and more.

Cognitive technology is so illuminating, it’s making us want to tell more stories. IBM Watson and IBM Cloud is already helping professionals design and innovate in fields such as fashion (with Marchesa to create the “cognitive dress”), cuisine (with Chef Watson to create new culinary combinations), music (via Watson Beat and Grammy award-winning producer Alex Da Kid) and architecture (with SOFTlab to create a sculpture inspired by Antoni Gaudi and the history of Barcelona.

Storytellers with Watson: The Finalists

Cognitive technology has become a tool of the arts that can help us to tell better stories. That’s why IBM has partnered with the Tribeca Film Festival for the launch of “Storytellers with Watson: A competition for Innovation sponsored by IBM.” Over the last two months, the competition has encouraged new thinking among media and entertainment developers to create original stories with the help of IBM Watson and the IBM Cloud.

1. “Building a Better Rip-o-matic”: Recognizing meaning in images & language for video editing

Producer, filmmaker and journalist Seth Grossman created a proposal that addresses the arduous task of searching and finding the short clips or “rips” from videos that can appropriately represent a filmmaker’s vision—ultimately all “rips” would be spliced together to make a preview called a “Rip-o-matic.” By recognizing information in images as well as finding and classifying their meaning in sets of written information, Watson would uncover desired content, including specific quotes, time periods, and locations to produce multiple clips that effectively represent the filmmaker’s vision. Seth also wants Watson to transform an entire script into a finished Rip-o-matic, by learning to recognize slug-lines or settings, and other classic elements of a preview’s script.

2. “ScriptAloud”: Helping casting directors and producers quickly review film scripts

Using Watson Text to Speech and Tone Analyzer, Sadaf Amouzegar, a software developer and data scientist, wants to transform written scripts into audio files available for casting directors, producers and others to easily download and listen, avoiding the burden of flipping through giant books of script. By harnessing these Watson speech and language capabilities such as emotive and expressive tag languages for controlling text to speech cadence, timbre, inflection and intonation, Sadaf envisions that automated voices in the transformed audio file will be able to convey the human emotion of the script’s original lines. 

3. “Watson Dance Assistance”: Dance choreography visualization

Professional dance choreographer, Mary John Frank seeks to explore the ways Watson can help choreographers create a visual script of their dance pieces to reduce the time and cost typically needed of a traditional choreography preview. “Watson Dance Assistant” would Watson Visual Recognition to recognize the information in the images of a single dancer to assist in designing group dance formations. She wants to use Watson to inspire new dance moves and edit dance choreography.

4. “Human Conversation Project”: Real-time translation with Watson Language Translator

Founder of Moth + Flame VR and Image Theory 3D Animation, Kevin Cornish has worked with Watson in the past making the Lions Innovation shortlist at Cannes Lions with his virtual reality conversation project, Fall in Love VR. In his new project proposal for the Storytellers with Watson contest, Kevin would use the Watson Language Translator service to connect people all over the world in a real-time—a virtual and augmented reality experience. Wearing Watson-powered headsets, users would be able to converse in their native tongues to each other, enabling conversation between two people living in opposite sides of the world.

5. “Watson Brand Studio Model: Powering the Content Brands of Tomorrow”

Billee Howard, founder and CEO at creative consultancy, Brandthropologie Media, aims to use AI to find the key elements of successful marketing campaigns—and to use those insights to help companies build campaigns based on “emotional levers”—or shared focal points that truly win the customer. Watson would be trained on campaign data, language, and images, to recognize topics in social media and other news sources, using Watson Natural Language Classifier. She’d also use Watson to understand the human personality driving campaign significance. The result would synthesize Watson’s insights to bring the most powerful marketing messages to the surface.

On behalf of the jury panel, we think each of these concepts utilize Watson in the real world – from the idea to the business positioning – each addressed pain points in the entertainment industry using storytelling with Watson, its unique qualities, and challenging Watson in new directions. The jury would like to give an honorary mention to Sadaf Amouzegar for her innovative approach, and taking a problem she saw on her job and finding a solution with a breakthrough idea. The jury selected Seth Grossman for his vision to tell a story, taking away barriers with a strong vision to push the envelope of Visual Recognition utilizing Watson. Congratulations, Seth Grossman!

The Tribeca Film Festival community represents a wellspring of creativity and imagination, and this competition has only just begun to show the types of stories the media and entertainment industry can tell together with cognitive computing.

(Finalists photo credits: Seth Grossman, Sadaf Amouzegar, Mary John Frank, Kevin Cornish, Billee Howard)

Try Watson’s APIs to build your next smart app or solution in minutes


Most Popular Articles