For its new video on the IBM solutions that help power Formula 1 (F1) team Red Bull Racing, IBM Spectrum Computing wanted to highlight cognitive computing and the shared commitment to innovation.
IBM Spectrum Computing called in the team behind IBM Watson Beat—an AI experiment in computational creativity—to unleash its neural network on writing an exciting soundtrack.
Knowsmusical theory through an advanced machine learning system
Createdoriginal soundtrack from basic ideas and specification of mood
Assistedhuman musician in coming up with fresh new music
Business challenge story
Making music that moves
As an innovation partner for the Red Bull Racing team, IBM Spectrum Computing knows all about pushing the boundaries of technology to deliver success in the real world. Backed by IBM Spectrum solutions, Red Bull Racing gathers, stores and analyzes data from on-car sensors to power real-time data-driven decisions and assist in the continuous redevelopment of the more than 7,500 components that constitute an F1 race car.
To promote its partnership with Red Bull Racing, IBM Spectrum Computing created a short video to showcase the technology and focus on the concepts of speed and the convergence of big data analytics and cognitive technologies in F1. Because the team already had a video interview of the Red Bull Racing team explaining the benefits of the IBM technologies, the new video was an unnarrated collage of behind-the-scenes footage. Of course, great visuals always have more power if they are backed by the right soundtrack, and the IBM team wanted to do more than just tag on some stock music.
Tutiya Teevan, IBM Spectrum Computing Program Manager, comments: “We are really proud of the innovation that is the foundation of our relationship with Red Bull Racing, and we saw the video soundtrack as an opportunity to build on this. We had read about the work of an IBM Research team in computer-assisted composition, and we were excited to get them on the case!”
Catching the beat
IBM Watson Beat started life as a hobby project for Janani Mukundan, who has a PhD in computer engineering from Cornell University and is a member of the IBM Research team in Austin, Texas. Mukundan is intrigued about the potential for IBM Watson artificial intelligence (AI) technology to be computationally creative, and recognized that music could be the ideal domain. She comments: “Music combines mathematical rules and structures with human creativity, so I felt it would be a great starting point for an experiment in machine learning. I set out to build a neural network that would ultimately compose original music.”
After several months of laying the groundwork, Mukundan brought in a professional musician and software engineer—Richard Daskas—to help her teach the new system the fundamentals of composition.
Says Daskas: “To teach the system, we broke music down into its core elements, such as pitch, rhythm, chord progression and instrumentation. We fed a huge number of data points into the neural network and linked them with information on both emotions and musical genres. As a simple example, a ‘spooky’ piece of music will often use an octatonic scale. The idea was to give the system a set of structural reference points so that we would be able to define the kind of music we wanted to hear in natural-language terms.”
The IBM team gave the system—named IBM Watson Beat—a comprehensive set of audio samples and digital instruments, and the self-learning agent started work on creating its own compositions. Further refinement and steering by the human team brought IBM Watson Beat to the point where it could reliably produce coherent original music to order.
Mukundan comments: “To use Watson Beat, you simply provide up to ten seconds of MIDI music—maybe by plugging in a keyboard and playing a basic melody or set of chords—and tell the system what kind of mood you want the output to sound like. The neural network understands music theory and how emotions are connected to different musical elements, and then it takes your basic ideas and creates something completely new.”
Enter the IBM Spectrum Computing team with a demanding brief to create the perfect soundtrack to match the jarring, jump-cut visuals and distorted look-and-feel of its latest video. Doug O’Flaherty of IBM Spectrum Computing explains: “It’s very well-known that IBM technologies power many of the world’s industry leaders, from major banks and insurers to governments, retail giants, global manufacturers, energy companies and healthcare providers. What’s perhaps less obvious is all the innovative work we do with fast-moving startups and in new industries.
“For our Red Bull Racing video, the key guiding principle was therefore the idea of the ‘unexpected’, so we actually wanted something that was a little different and that would surprise any viewers unfamiliar with this side of IBM. And right from the start, we knew we wanted to do something different on the music front.”
By varying the playback speed for different clips within the video, the director set out to communicate the idea of “altering time”—using vast computational power to accelerate analysis and design, but also to stop the clock and enable split-second in-race decisions based on big data. The brief to the IBM Watson Beat team was to complement the visuals by combining classical music and dance beats, and to wrap everything up with the same degree of distortion seen in the video transitions.
Cutting the track
Working with some initial musical ideas from Daskas, IBM Watson Beat rapidly came up with a stunning original composition, perfectly timed to the working cut of the video. The output was in the form of a finished audio file and the corresponding editable MIDI music data.
To finalize the composition, Daskas applied some tweaks and mixed in F1 engine sounds to add further texture. “We went through a couple of stages of refinement with the video team, and Watson Beat gave us 80 or 90 percent of the final piece—the rest was just some realignment,” he says. “As a composer, it’s really interesting to work with Watson Beat because it comes up with a whole new way of doing things. We see it as a creative assistant tool to help human composers rather than trying to replace them: you give the system a starting point, it comes up with a path to follow, and then you apply your own ingenuity and instinct to finish the piece. It’s also great if you have a case of writer’s block!”
With the Red Bull Racing soundtrack complete, the IBM Watson Beat team is continuing to build out the system’s capabilities, adding new genres and moods. The solution will move to the IBM Cloud, and will soon be made available to the public under an open source license.
Elizabeth Transier, IBM THINKLab Director, outlines the future vision: “We’re really excited about the potential for Watson Beat, both as a tool for enabling non-musicians to become more musical and as an intelligent assistant for composers. If you’re a musician, you could almost think of it as another member of the band: someone to bounce ideas off and to throw you a few curveballs that get you thinking in new creative directions. Watson Beat is a great example of how IBM cognitive technologies are starting to augment human capabilities and help us reach new capabilities.”
About IBM Watson Beat
IBM is one of the world’s leading technology companies. Working together, our mathematicians, engineers, coders and designers outthink challenges and make discoveries that alter industries. Our research division continues to push the boundaries of what we know and what we can do, developing advanced cognitive technologies that enable new partnerships between people and computers.
Take the next step
To learn more about IBM Watson Beat, please contact your IBM representative or IBM Business Partner, or visit the following website: ibm.biz/watson-beat
To learn more about IBM Spectrum Computing, please contact your IBM representative or IBM Business Partner, or visit the following website: ibm.com/spectrum-computing