Skip to main content

Shalisha and Shonda
Witherspoon

AI researchers making music more accessible

Shalisha Witherspoon

Software engineer

Identical twins and software engineers Shonda and Shalisha Witherspoon share more than an office in IBM Research, where they work to expand the capabilities of AI and edge computing. In 2022 they were jointly named the Most Promising Engineer by the Black Engineer of the Year Awards (BEYA) STEM Conference, and were recently granted a patent inspired by their love of K-pop music. Shalisha and Shonda interned at IBM in 2017, and joined the company full-time after graduating at the top of their class from Florida International University — with identical GPAs.

How would you describe your work?

Shonda: We both work as software developers for IBM Edge AI, which is an emerging technology that enables AI applications to have fast, streamlined access to the data they need, wherever that data resides.

Shalisha: When AI systems need to analyze large quantities of data, and that data is housed on distant servers, that can slow the process down and use a huge amount of bandwidth. With Edge, the computations are done closer to where the data is being originated, which makes things much more efficient so more AI applications can be enabled.

How is your work advancing the field of AI?

Shonda: AI is still a developing field and, with more and more applications using machine learning, we have to make sure these models are trained well to do their tasks — that they are not biased. It’s all about making sure the data is right and the models are correct.

Shalisha: For example, take self-driving cars. The models have to know that a red light means stop and a green light means go. If the models aren’t trained properly, disasters can ensue. My focus is on OOD: Out-Of-Distribution detection. What that means is, how can we know that our models are right, that we can trust our models, even when things in the real world are different from the ideal? For instance, if a stop sign has a bumper sticker on it, the model still has to be able to recognize it as a stop sign.

What about your work makes you excited to get up in the morning?

Shonda: Working in a true research environment. A lot of the things that we’re trying to accomplish have a lot of unknowns. I like the idea that I can contribute to finding the answers, and if an answer I come up with doesn’t work, we can find another one. There’s that mystery-solving aspect to working in a research environment.

Shalisha: So many of those same reasons are mine also. Learning something new that is out of my comfort zone is always exciting.

What do you think of as your first big success?

Shalisha: I have to say it was winning the title of Most Promising Engineer, awarded at the Black Engineer of the Year Awards (BEYA) STEM Conference. The award is for developers who were within the first three to 10 years of their careers, and we literally just made the three-year mark that month.

I understand you were recently granted your first patent? Can you tell me about that?

Shalisha: We love K-pop music. We’ve even created two volumes in a book series, K-Pop Idol Diaries.

Shonda: But sometimes people will say, “Why do you like K-pop? You can’t even understand what they’re saying.” There are “fanslations” out there, but you can tell they’re taken straight from Google Translate, because they really don’t make sense — they don’t capture the idioms and other cultural subtleties.

Shalisha: We got to thinking that we could use machine learning to achieve our goal. So, we designed a system and method that allows users to input a song in one language, choose a target language to output it to, and have it translate in a way that captures the meaning of the song, the lyrics that are best suited to the purpose.

Our goal was to create something that would translate the songs in a way that meets the user’s needs the best. What if you don’t care so much about the idioms and cliches? Maybe you want to preserve rhyming, or the number of syllables in a verse. We tried to create an app that will come up with the quickest way of producing the version of the song that a user might be looking for — a translation that meets their needs.

How does your design accomplish that?

Shonda: Once you know what you’re after, the next step is to apply a deep-learning approach. This is basically where the system learns the voices that are inside this audio file. For example, let’s say the singer is Michael Jackson. You want to be able to recognize Michael Jackson’s voice so that you can synthesize it, so that if you provide the vocal model and the translated lyrics, the final track is in Michael Jackson’s voice.

Shalisha: You know the idea of deep fakes? Like, you basically show a Barack Obama speech, but you manipulate it to make the vocal model speak like Obama, but it’s not him, and it’s very scary because it sounds just like him. Unfortunately, it’s a technology that’s being used for malicious purposes in some cases. For this invention, we’re using that technology to make the vocals sound as close to the original singer’s voice as possible.

The next step in this is the text-to-song. The idea here is that it’s not just that the AI is learning how Michael Jackson sounds, but it’s learning how he sings. So, for example, if you listen to Billy Jean or something like that, you want the translation to mimic how he sings the song.

And finally, what you have is your audio. You have your translated lyrics, and you just re-create the song by placing the audio overlay on the original instrumentation, and then the song is ready to give back to the user.

What do you hope will be the outcome of your music translator?

Shalisha: Music is a universal language, and our hope is that our creation will let people enjoy more music from around the world in new ways, and allow artists to break through the language barrier to share their message.

Besides your work in advancing AI technology, what other advances would you like to be part of?

Shonda: I always think about how the work I do could impact society for the greater good. Like STEM outreach — how can I make sure there are more women in science? How can I make sure more Black people pursue it? I want to have an impact on diversity and inclusion initiatives, and on how we can use technology to make our communities safer, especially Black communities.

Shalisha: There are so many opportunities to get involved in tech for good. We both participated in Call for Code for Racial Justice, which is something that could really impact a change in society. That’s the kind of work I hope to get more opportunities to participate in.

Explore AI