Making “Cognitive Music” with IBM Watson

Share this post:

It’s no surprise to find music at the center of many important moments of our lives – think of a wedding march, a “Happy Birthday,” or even a funeral procession. Music is deeply tied to the human experience; each song and each moment is personal to the listener. Over the last year, Watson has been studying up on music themes, theory, moods, and emotions – and how they correlate and connect with each other.

Now, Watson is inspiring musical creativity with Grammy-winning music producer Alex Da Kid, who used Watson’s technology to inspire his new song about heartbreak, “Not Easy.” This is the first song in Alex’s 4-track EP in collaboration with Watson.

For this partnership, Watson analyzed the last five years of culture and music data to uncover new emotional insights to augment Alex’s creative process. To identify the most pervasive themes, the team used the Watson Alchemy Language API to read and understand Nobel Peace Prize speeches, New York Times articles, Billboard song lyrics, movie synopses and more.

The Watson Tone Analyzer API then ingested more than 2 million lines of related social content to understand the emotional sentiment surrounding these themes. Combined, these APIs helped us to build the “emotional fingerprint” of each of the last five years.

Alex then tapped into Watson BEAT to examine and better understand popular musical trends. And Watson BEAT is more than your average music-suggestion engine. Today, music-recommending apps use genre terms and ratings to suggest similar songs we might like. For example – if you like Wiz Khalifa’s 2015 hit “See You Again”, you might also like Maroon 5’s 2012, “Payphone” (featuring Wiz Khalifa). They recommend based on other songs, not how a person feels in the moment.

Using machine learning algorithms, Watson BEAT is capable of learning from songs by deconstructing a song’s pitch, time and key signatures, note sequence, and note velocity (how hard a note was struck).

Combined with theories about emotional responses to music, Watson BEAT can generate completely new musical scores based on a variety of preferred moods (joyful, devastated, anxious etc.) or feelings (spooky, mysterious, cheerful, etc.).

Using this technology, Alex could create new songs – even snippets of songs, like a bass line – until he found a sound that inspired him. And while “Not Easy” is an original work, Watson BEAT gave Alex many new ways to experience musical themes and human emotions.

To dive even deeper into the insights, the IBM team used the Cognitive Color Design Tool to create an interactive visualization of the data based on the images, album artwork and colors that inspire Alex. The tool blends together understanding of psychological effects of colors, the interrelationships between emotions, and image aesthetics to create a custom color palette, tailored to the individual.

At the heart of this collaboration is the power that artificial intelligence brings to human expertise. ‘Cognitive Creativity’ – augmenting human creativity with the power of AI, allows us to create ideas, stories and concepts previously unimagined. Watson’s ability to understand, reason and learn at scale using natural language is emerging as a source of creativity in a number of industries – from the culinary world (Chef Watson), to fashion (Cognitive Dress) to entertainment (Cognitive Movie Trailer).

Today, Watson is working in music – the team at IBM is already imagining where Watson’s ability to inspire will go next.

Distinguished Engineer, IBM Research

More stories

Houston, We’re Live Streaming

Since people around the world first gathered around black-and-white televisions to watch the 1969 moon landing, the general public has been fascinated by space, craving more visual information about the final frontier. At NASA today, we are on a mission to find new ways to share our discoveries on Earth and in space. Before the […]

Continue reading

An Open Letter to the President-Elect

International Business Machines Corporation Office of the Chairman, President and Chief Executive Officer One New Orchard Road Armonk, New York 10504-1783 November 14, 2016 Mr. Donald J. Trump Office of the Presidential Transition 1800 G Street, NW Washington, DC 20006 Dear Mr. President-elect: Congratulations on your election as the 45th president of the United States. […]

Continue reading

How New Algorithms Are Helping to Predict Food Dangers

Thanksgiving in the United States, like other holidays around the world, places food at the center of a joyful celebration — and at the forefront of our minds. As we plan, shop, prepare, cook and share our celebratory meals, we are careful to include not only flavors that our loved ones enjoy, but also foods […]

Continue reading