Making “Cognitive Music” with IBM Watson

Share this post:

It’s no surprise to find music at the center of many important moments of our lives – think of a wedding march, a “Happy Birthday,” or even a funeral procession. Music is deeply tied to the human experience; each song and each moment is personal to the listener. Over the last year, Watson has been studying up on music themes, theory, moods, and emotions – and how they correlate and connect with each other.

Now, Watson is inspiring musical creativity with Grammy-winning music producer Alex Da Kid, who used Watson’s technology to inspire his new song about heartbreak, “Not Easy.” This is the first song in Alex’s 4-track EP in collaboration with Watson.

For this partnership, Watson analyzed the last five years of culture and music data to uncover new emotional insights to augment Alex’s creative process. To identify the most pervasive themes, the team used the Watson Alchemy Language API to read and understand Nobel Peace Prize speeches, New York Times articles, Billboard song lyrics, movie synopses and more.

The Watson Tone Analyzer API then ingested more than 2 million lines of related social content to understand the emotional sentiment surrounding these themes. Combined, these APIs helped us to build the “emotional fingerprint” of each of the last five years.

Alex then tapped into Watson BEAT to examine and better understand popular musical trends. And Watson BEAT is more than your average music-suggestion engine. Today, music-recommending apps use genre terms and ratings to suggest similar songs we might like. For example – if you like Wiz Khalifa’s 2015 hit “See You Again”, you might also like Maroon 5’s 2012, “Payphone” (featuring Wiz Khalifa). They recommend based on other songs, not how a person feels in the moment.

Using machine learning algorithms, Watson BEAT is capable of learning from songs by deconstructing a song’s pitch, time and key signatures, note sequence, and note velocity (how hard a note was struck).

Combined with theories about emotional responses to music, Watson BEAT can generate completely new musical scores based on a variety of preferred moods (joyful, devastated, anxious etc.) or feelings (spooky, mysterious, cheerful, etc.).

Using this technology, Alex could create new songs – even snippets of songs, like a bass line – until he found a sound that inspired him. And while “Not Easy” is an original work, Watson BEAT gave Alex many new ways to experience musical themes and human emotions.

To dive even deeper into the insights, the IBM team used the Cognitive Color Design Tool to create an interactive visualization of the data based on the images, album artwork and colors that inspire Alex. The tool blends together understanding of psychological effects of colors, the interrelationships between emotions, and image aesthetics to create a custom color palette, tailored to the individual.

At the heart of this collaboration is the power that artificial intelligence brings to human expertise. ‘Cognitive Creativity’ – augmenting human creativity with the power of AI, allows us to create ideas, stories and concepts previously unimagined. Watson’s ability to understand, reason and learn at scale using natural language is emerging as a source of creativity in a number of industries – from the culinary world (Chef Watson), to fashion (Cognitive Dress) to entertainment (Cognitive Movie Trailer).

Today, Watson is working in music – the team at IBM is already imagining where Watson’s ability to inspire will go next.

Distinguished Engineer, IBM Research

Al Green

Please i beg you, tell me what mixing console is being used in this commercial. it looks like a custom SSL Duality console.
Watson, where are you when I need you?

Watson please send the answer to my email below.
I dream of having a console like that

Dolores Catherino

It seems important to clarify that the words intelligence, creativity and cognitive are being used metaphorically in computer science.

AI generatively recycles and recombines massive amounts of data provided by humans, and this is what gives it an appearance similar to individual and collective intelligence. In a related way, the expanding use of audio sampling/looping in music provides a digital simile of physical instruments and musicians only because its database approximates, recycles and recombines the prior recordings of human musicians.

In either instance, these similies of human intelligence and creativity are dependent on massive quantities of encoded, discrete, snippets of human content (and human generated algorithmic recombination patterns) for their approximation. Thus words like intelligence and creativity are being used metaphorically when characterizing AI as being anything more than an appearance/simulation of the qualities of consciousness and sentience.

Current trends in technology and AI are replacing an increasing number of human activities. In this context, it seems important to distinguish the word ‘creativity’ in (at least) two senses: creative process (composer) versus generative recombination (compositor).

Ultimately, it seems imperative to value and develop human creativity as a process to which new technology is designed to assist; as an area distinct from training the human creative process for the role of assisting (editing, arranging) a generative technological process.

Comments are closed.

More Uncategorized stories

One Girl’s Journey into Programming

Like a lot of high schools, mine has a wide range of extracurricular activities, from academic clubs, such as Math Honor Society, to sports teams like outdoor beach volleyball. But there’s one that’s fairly unique: an organization of driven, smart young ladies called All Girls Code. The All Girls Code club was started a couple […]

Continue reading

Houston, We’re Live Streaming

Since people around the world first gathered around black-and-white televisions to watch the 1969 moon landing, the general public has been fascinated by space, craving more visual information about the final frontier. At NASA today, we are on a mission to find new ways to share our discoveries on Earth and in space. Before the […]

Continue reading

An Open Letter to the President-Elect

International Business Machines Corporation Office of the Chairman, President and Chief Executive Officer One New Orchard Road Armonk, New York 10504-1783 November 14, 2016 Mr. Donald J. Trump Office of the Presidential Transition 1800 G Street, NW Washington, DC 20006 Dear Mr. President-elect: Congratulations on your election as the 45th president of the United States. […]

Continue reading