Cognitive Computing

IBM Research Takes Watson to Hollywood with the First “Cognitive Movie Trailer”

Share this post:

How do you create a movie trailer about an artificially enhanced human?

You turn to the real thing – artificial intelligence.

20th Century Fox has partnered with IBM Research to develop the first-ever “cognitive movie trailer” for its upcoming suspense/horror film, “Morgan”. Fox wanted to explore using artificial intelligence (AI) to create a horror movie trailer that would keep audiences on the edge of their seats.

Movies, especially horror movies, are incredibly subjective. Think about the scariest movie you know (for me, it’s the 1976 movie, “The Omen”). I can almost guarantee that if you ask the person next to you, they’ll have a different answer. There are patterns and types of emotions in horror movies that resonate differently with each viewer, and the intricacies and interrelation of these are what an AI system would have to identify and understand in order to create a compelling movie trailer. Our team was faced with the challenge of not only teaching a system to understand, “what is scary”, but then to create a trailer that would be considered “frightening and suspenseful” by a majority of viewers.

As with any AI system, the first step was training it to understand a subject area. Using machine learning techniques and experimental Watson APIs, our Research team trained a system on the trailers of 100 horror movies by segmenting out each scene from the trailers. Once each trailer was segmented into “moments”, the system completed the following;

1)   A visual analysis and identification of the people, objects and scenery. Each scene was tagged with an emotion from a broad bank of 24 different emotions and labels from across 22,000 scene categories, such as eerie, frightening and loving;

2)   An audio analysis of the ambient sounds (such as the character’s tone of voice and the musical score), to understand the sentiments associated with each of those scenes;

3)   An analysis of each scene’s composition (such the location of the shot, the image framing and the lighting), to categorize the types of locations and shots that traditionally make up suspense/horror movie trailers.

The analysis was performed on each area separately and in combination with each other using statistical approaches. The system now “understands” the types of scenes that categorically fit into the structure of a suspense/horror movie trailer.

Then, it was time for the real test. We fed the system the full-length feature film, “Morgan”. After the system “watched” the movie, it identified 10 moments that would be the best candidates for a trailer. In this case, these happened to reflect tender or suspenseful moments. If we were working with a different movie, perhaps “The Omen”, it might have selected different types of scenes. If we were working with a comedy, it would have a different set of parameters to select different types of moments.

It’s important to note that there is no “ground truth” with creative projects like this one. Neither our team, or the Fox team, knew exactly what we were looking for before we started the process. Based on our training and testing of the system, we knew that tender and suspenseful scenes would be short-listed, but we didn’t know which ones the system would pick to create a complete trailer. As most creative projects go, we thought, “we’ll know it when we see it.”

Our system could select the moments, but it’s not an editor. We partnered with a resident IBM filmmaker to arrange and edit each of the moments together into a comprehensive trailer. You’ll see his expertise in the addition of black title cards, the musical overlay and the order of moments in the trailer.

Not surprisingly, our system chose some moments in the movie that were not included in other “Morgan” trailers. The system allowed us to look at moments in the movie in different ways –moments that might not have traditionally made the cut, were now short-listed as candidates. On the other hand, when we reviewed all the scenes that our system selected, one didn’t seem to fit with the bigger story we were trying to tell –so we decided not to use it. Even Watson sometimes ends up with footage on the cutting room floor!

Traditionally, creating a movie trailer is a labor-intensive, completely manual process. Teams have to sort through hours of footage and manually select each and every potential candidate moment. This process is expensive and time consuming –taking anywhere between 10 and 30 days to complete.

From a 90-minute movie, our system provided our filmmaker a total of six minutes of footage. From the moment our system watched “Morgan” for the first time, to the moment our filmmaker finished the final editing, the entire process took about 24 hours.

Reducing the time of a process from weeks to hours –that is the true power of AI.

The combination of machine intelligence and human expertise is a powerful one. This research investigation is simply the first of many into what we hope will be a promising area of machine and human creativity. We don’t have the only solution for this challenge, but we’re excited about pushing the possibilities of how AI can augment the expertise and creativity of individuals.

AI is being put to work across a variety of industries; helping scientists discover promising treatment pathways to fight diseases or helping law experts discover connections between cases. Film making is just one more example of how cognitive computing systems can help people make new discoveries.

IBM Fellow, Manager, Multimedia and Vision

[…] John R. Smith erklärt in einem Blogbeitrag wie genau das […]

[…] nécessité une triple analyse : visuelle, sonore et de composition. C’est ce qu’explique John R. Smith, un cadre d’IBM chargé de la vision par ordinateur et du […]

[…] IBM-Blog auf asmarterplanet.com erläutert Projektleiter John R. Smith: „Unser System konnte die Momente auswählen, aber es […]

[…] Volgens IBM was dit vooral een experiment om te kijken wat de uitkomst zou zijn. Uiteindelijk heeft het team één moment dat door Watson was geselecteerd niet gebruikt, omdat het niet bij de rest van de trailer zou passen. Het team beweert verder dat het maken van de trailer hierdoor in 24 uur voltooid kon worden. […]

[…] O resultado você confere no vídeo abaixo, que também acompanha falas dos profissionais da IBM no projeto e do próprio diretor, Luke Scott(em inglês). Caso queria dar uma olhada no post original no blog da IBM, clique aqui. […]


Chris Briggs

The question isn’t whether AI systems can create. The question is whether they would want to.

[…] По материалам FastCoCreate, Asmarterplanet. […]


Buttercup

This is Amazing, a new step in AI world. good job IBM Watson project.

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, ‘what is […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, ‘what is […]

[…] de composer une bande-annonce –ce qu’elle est parvenue à faire–, explique John Smith, sur son blog, qui a piloté le […]

[…] différemment en fonction de chaque spectateur »,  explique John Smith sur le site d’IBM. « Il fallait donc que Watson apprenne à reconnaître ce qui était […]

[…] and understand in order to create a compelling movie trailer,” wrote IBM’s John R. Smith, in a blog post. […]

[…] Source: IBM […]


Laleph : Actualite alternative

[…] de composer une bande-annonce –ce qu’elle est parvenue à faire–, explique John Smith, sur son blog, qui a piloté le projet. Pour autant, Watson n’est pas parvenue à «monter» ces extraits […]

[…] Watson was able to make the trailer by first analyzing 100 horror movie trailers, studying each scene, and looking for common ground. Next, it was fed the 90-minute movie, and chose 10 […]

[…] Watson was able to make the trailer by first analyzing 100 horror movie trailers, studying each scene, and looking for common ground. Next, it was fed the 90-minute movie, and chose 10 scenes that it thought would fit in  the trailer. An editor then pieced together the scenes. […]

[…] down the film to just six minutes of footage. Of course, not everything it picked was suitable, but according to IBM, it offered a different way of evaluating […]

[…] Fox approached IBM and asked them if they’d be willing to let Watson have a go at creating a trailer for the movie. The movie was, after all, about a sentient machine. IBM agreed. […]

[…] Fox approached IBM and asked them if they’d be willing to let Watson have a go at creating a trailer for the movie. The movie was, after all, about a sentient machine. IBM agreed. […]

[…] Fox approached IBM and asked them if they’d be willing to let Watson have a go at creating a trailer for the movie. The movie was, after all, about a sentient machine. IBM agreed. […]

[…] pas pu tout faire toute seule. John Smith, l’ingénieur d’IBM en charge du projet indique donc qu’il a fallu ensuite procéder à un montage et à des finitions plus abouties, bel et bien […]

[…] September 2; check out the trailer below for a hint of what the movie is all about (and even check out an AI-created movie trailer!) Discuss further over in the 3D Printed Morgan forum at […]

[…] 20th Century Fox has partnered with IBM Research to develop the first-ever “cognitive movie trailer” for its upcoming suspense/horror film, “Morgan”. Fox wanted to explore using artificial intelligence (AI) to create a horror movie trailer that would keep audiences on the edge of their seats.” (Lesen mehr unter IBM.com) […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, ‘what is […]

[…] с научно-исследовательским подразделением IBM Research приказали когнитивной системе IBW Watson изготовить трейлер с для […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, ‘what is […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, ‘what is […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, […]

[…] с научно-исследовательским подразделением IBM Research приказали когнитивной системе IBW Watson изготовить трейлер с для […]

[…] movie trailer,” John R. Smith, IBM Fellow and manager of multimedia and vision said in a post announcing the project. “Our team was faced with the challenge of not only teaching a system to understand, […]

[…] с научно-исследовательским подразделением IBM Research приказали когнитивной системе IBW Watson изготовить трейлер с для […]

[…] son blog, John Smith, qui a piloté le projet pour IBM, explique comment Watson, l’intelligence […]


Luca Marchi

What are the Watson services/APIs that were used in this project?

[…] do filme Morgan. O filme fala sobre um super ser humano com modificações cibernéticas; por isso, a equipe de pesquisa da IBM usou a plataforma de inteligência artificial Watson para editar o “trailer perfeito” para o filme. Ele pode ser visto no final desta […]


Maria Capello

I can’t wait to see this movie. I love Thrillers. I wonder if IBM will do a movie premiere? or even get discount tickets to the movies?

[…] The good news it that it doesn’t look like computers will be replacing filmmakers or editors anytime soon, but if you want to know more about the process of how this was accomplished, check out a full write-up over here. […]

[…] to make movie trailers. 20th Century Fox has tapped into the supercomputer’s powers to create the first AI-made trailer for its upcoming thriller film Morgan. It’s a fitting start for […]

[…] The good news it that it doesn’t look like computers will be replacing filmmakers or editors anytime soon, but if you want to know more about the process of how this was accomplished, check out a full write-up over here. […]

[…] трейлер для фильма ужасов "Морган". Как сообщается в блоге IBM, студия попросила создать трейлер, который "будет […]

[…] трейлер фільму жахів "Морган". Як повідомляється в блозі IBM, студія попросила створити трейлер, який "буде […]


Lifeboat News: The Blog

[…] beyond their control. Fitting, then, that they’d employ the help of America’s AI sweetheart IBM Watson to build the film’s […]

[…] Morgan opens September 2. For more on the IBM team’s process check out their PR blog post here. […]

[…] make the film, an IBM blog post explains, Watson analyzed the trailers of over 100 horror and thriller film trailers to understand what […]

[…] make the film, an IBM blog post explains, Watson analyzed the trailers of over 100 horror and thriller film trailers to understand what […]

[…] make the film, an IBM blog post explains, Watson analyzed the trailers of over 100 horror and thriller film trailers to see understand what […]

[…] make a film, an IBM blog post explains, Watson analyzed a trailers of over 100 fear and thriller film trailers to know what sounds, […]

[…] make the film, an IBM blog post explains, Watson analyzed the trailers of over 100 horror and thriller film trailers to understand what […]

[…] make the film, an IBM blog post explains, Watson analyzed the trailers of over 100 horror and thriller film trailers to see understand what […]


Sebastian

¿Who was the filmmaker?

[…] better way to ignore that rule than to have IBM task an AI to create the first-ever “cognitive movie […]

Comments are closed.

More IBM Watson stories

IBM Watson Goes to the GRAMMYs

The recording industry has a certain rhythm to it. Throughout any given year, there is a steady drumbeat of new releases from established artists. There are the debuts of fresh new faces and innovative sounds. There is the heartbreaking loss of some of our most beloved musicians. And every year, this complex symphony of events […]

Continue reading

Future Designers Unleash Creativity with AI

At Tommy Hilfiger, we have always been industry pioneers and driven by our vision to break conventions. We are constantly seeking new ways to evolve and respond to our consumer’s expectations. In this age of newness, it’s more important than ever to evolve and deliver on the instant gratification that consumers are looking for. Two years ago, we had the unique […]

Continue reading

IBM Patents: the Foundation of Innovation and Discovery

The strategic use of intellectual property has been at the core of IBM’s success throughout our 107-year history. We have always sought an approach to patenting that balances proprietary and open innovation. Patents not only allow us to grow and thrive as a company but they also enable us to advance the cause of technology […]

Continue reading