“It’s not just a pet; that dog becomes a part of them,” says Lorraine Trapani, a member of IBM’s strategy and leadership team for Government and Regulatory Affairs. Trapani is also a puppy raiser for Guiding Eyes for the Blind, a non-profit organization in Yorktown Heights, NY, that provides more than 170 service dogs for people with vision and other disabilities every year.
AI helps grow a better guide dog
Puppies are probably not the first thing that comes to mind when someone mentions AI. Maybe robots, maybe self-driving cars, definitely not puppies. But thanks in part to the efforts of IBMer Lorraine Trapani, AI is playing an important role in turning adorable puppies into confident, competent guide dogs.
Trapani became interested in guide dogs when her late husband lost partial sight, and has been a puppy raiser for Guiding Eyes since 2011. “Michael learned to adapt to the loss of his vision, but during that time I saw how challenging it is, not just physically for the individual with vision loss, but emotionally for their families. All of the puppies I raise, I raise in Michael’s name.”
“After Michael died, a friend invited me to a graduation ceremony at Guiding Eyes. These ceremonies are open to the public and celebrate the partnership between a guide dog and its recipient. They also mark the beginning of Guiding Eyes students’ new lives with their dogs, and salute the hundreds of volunteers who raise these puppies and provide loving homes for the breeder dogs. I was hooked.”
Trapani was at a Guiding Eyes lecture in 2016 when the presenter, Jane Russenberger, spoke about efforts the group was making to improve the success rate of puppies that go through the program. It costs around US$50,000 to get a dog from breeding all the way to graduation, and most don’t make it that far. For some the job is too stressful; others don’t have the confidence needed to make a thousand micro-decisions daily for their owner; some just don’t enjoy the work.
Russenberger, the agency’s senior director of genetics and breeding, said Guiding Eyes has been collecting data on their dogs since 1990. The data comes from veterinarians, puppy testers, puppy raisers and guide dog trainers — basically everybody who passes through a guide dog’s life before it graduates.
“She said they were trying to analyze that data to improve the dogs’ success rate, so I asked her if she had ever heard about Watson, and offered to send her a link to try it,” said Trapani. “Their board had already approached IBM about moving all that data onto IBM Cloud, making the data available for analysis and sharing. The groundwork was laid, and it just took off.”
A perfect match between Watson and Guiding Eyes
Guiding Eyes migrated more than half a million canine health records and more than 65,000 temperament records on thousands of dogs to the IBM Cloud. The resulting analysis helped Guiding Eyes better predict which dogs would make it through the program.
Researchers then used IBM Watson Personality Insights and Natural Language Processing on a preliminary dataset to find patterns, traits and characteristics — both human and canine — that would help create successful matches between dogs and trainers
“I was just blown away,” said Russenberger. “They were able to predict if a puppy raiser would be successful with a particular dog. Watson was able to use the natural language comments in our records to do that, which is something we hadn’t been able to do at all.”
Getting all the variables right is critical, not only for the puppies and the people they will eventually serve, but for the puppy raisers as well. “It’s very expensive to put a dog through the program, and it’s all supported by private donations,” Trapani said. “So it’s costly to fail, and it’s heartbreaking to fail, because you have people like me who devote a year of their time and emotionally invest in the success of a puppy, and when that puppy doesn’t succeed, you feel like you failed them somehow.”
I want these puppies to go off and help someone in my husband’s name.
Next steps with Watson
Those first studies led Russenberger to wonder if AI could help even more. “The real question,” she said, “is whether AI can give us a reliable prediction of success early in a dog’s life.”
In 2017, Guiding Eyes began working with computer scientist David Roberts and electrical engineer Alper Bozkurt, researchers from North Carolina State University who had been recognized by the Obama White House for their work on animal-centered computing, which seeks to design computing systems that allow animals to apply their intelligence in interesting, creative and health-improving ways.
Guiding Eyes has a long-established process where they test their puppies early, at 7 ½ weeks, to evaluate their temperament and stress response.
Bozkurt described that early test: “They have puppies climbing stairs, walking across different surfaces; they turn on a vacuum cleaner, or open an umbrella next to them, and they observe their behavior.”
Then throughout the growing process, puppy raisers and trainers take note of significant traits, fears and behavioral quirks, while also conducting ongoing temperament testing.
“The data collection was very subjective,” said Roberts. “Self-reports from raisers, from experts, from other people who were in the dog’s life. When Jane heard about the work we were doing, she asked if our technology could help them in that process.”
It takes a village
The UNCS team working with Guiding Eyes (left to right): Marc Foster, Sean Mealin, Simba, Alper Bozkurt and Evan Williams. Not pictured: David Roberts, Timothy Holder and Zach Cleghern.
Photo by Marc Hall (NCSU)
“Alper and I looked at each other and said ‘Absolutely, we can do that!’ — not knowing at the time how complicated it was going to be. Tackling issues like, you’re working with 7 ½ week-old-puppies. How do you design sensor systems to work effectively so you don’t impact the behavior that you’re trying to measure?”
The solution was an elastic chest band the puppies wear for a few minutes during that early temperament testing.
“We put a small inertial measurement unit on the band to track the puppy’s movement, and we designed an EKG system for heart rate detection that uses 3-D printed electrodes,” said Bozkurt. A grad student working with the team designed a video application to record the testing sessions, and all of that data – along with notes compiled by Guiding Eyes’ expert observers, is synchronized and uploaded to the IBM Cloud nightly via a Kubernetes architecture. Roberts and Bozkurt then analyze the data using Watson machine learning.
“Ultimately,” said Roberts, “our goal is to be able to predict very efficiently and very early whether or not a puppy is likely to be successful as a guide dog.”
“We’re still early in the process to have outcome data for enough puppies,” he added. “As a proxy for being able to predict whether a puppy will be successful, we’re trying to predict behavioral checklist (BCL) scores. The puppies were scored on their reactions to 29 stimuli to assess things like fear of strangers, concern about stairs and anxiety in new situations. It’s the gold standard in the working dog community for describing temperament, so until we have enough outcome data from puppies we’re working with — ideally, around 1,000 puppies — our goal is to be able to accurately predict the BCL scores for these puppies, based solely on the electrical signals coming from the elastic bands. Thus far, we have been achieving a 93% accuracy in predicting BCL scores.”
Smart new collars for smart new dogs
Roberts and Bozkurt also developed a collar module that older puppies wear. “It has environmental sensors in it that tell us about the dog’s environment once they go to live with the puppy raisers,” said Bozkurt. “It records barometric pressure, light, temperature. There’s also a small microphone in there to record barking and environmental noises, and a device that records activity levels.”
Engineering a device that could withstand the rigors of being attached to an active puppy was a tougher challenge. “Our first prototypes were failing in two or three days,” remembered Roberts. “We were getting feedback like, ‘My dog ate your device.’”
Once they connected with a company experienced in making durable dog collar devices, they were able to begin manufacturing working models
One of the biggest hurdles, said Roberts, was “solving the intellectual challenge of putting sensors on a biological organism in a way that doesn’t affect their lives, but allows us to collect data. We’re very careful to always act in an ethical manner that doesn’t harm the dogs.”
The collar measures a wide array of factors, from the dog’s stress responses to socialization patterns. Even information as basic as the dog’s walking speed is helpful — a dog with a faster natural gait might be a better fit for a visually impaired person who is more active.
Trapani said the hope is to use the data to tailor a puppy’s experiences to its disposition. “If the puppy test data show that loud noises make her heart rate increase, indicating stress, they can tell the raiser to introduce the puppy to things like fire trucks gradually, perhaps by bringing the dog to the local fire station to become acclimated to the trucks in a non-threatening situation. This way her first experience with a fire truck isn’t when it’s flying by with the siren screaming.”
By customizing each dog’s training experience, Russenberger said the agency hopes to see more dogs make it successfully through graduation. “There’s such an investment of time and money that goes into each puppy, and if this research helps us increase the percentage of dogs that graduate, even by 10%, it will be huge.”
IBM’s sponsorship is funding an order of 300 more smart collars to be given to puppy raisers in the near future.
“We’re incredibly grateful for IBM’s support,” said Roberts. “They’ve been a force multiplier on this project.”
She loves them enough to let them go
Trapani is now raising Jackie, her seventh Guiding Eyes dog and the second sponsored by IBM. TJ, her first, was a star in puppy training, but sadly, an uncontrollable impulse to chase birds kept him from living out his life as a guide dog. He was adopted by a colleague of Trapani’s, and spends time with her when his new family goes on vacation.
In her role as an AI ambassador for IBM, Jackie has met with members of Congress, including Sen. Thom Tillis and Rep. David Price, both from North Carolina.
“We wanted to bring Jackie to D.C. to show members of Congress the positive power of this technology,” said Sarah Minkel, of IBM’s policy communications team. “It’s a great example of tech for good.”
After eight to twelve months with a puppy raiser, and a minimum of six months with a professional trainer, a puppy is ready to meet the person it will spend virtually every moment with for the next eight or so years.
Trapani said that even though Jackie is her seventh Guiding Eyes dog, giving them up never gets easier — but it’s all worth it.
“The weeks that lead up to their return to Guiding Eyes are hard. I think it must be like sending a kid off to college. You miss them every day, but you want them to succeed in life.
“It’s heart-wrenching … but the thinking side of me says this puppy can’t achieve what I want her to achieve in Michael’s honor if she stays with me.
“A man with five daughters, who received a guide dog I had raised, said, ‘Ed is the son I never had. With Ed, I’m doing things I haven’t done since I lost my sight. He gave me my independence back.’ Another recipient told me he lost his vision at 16 years old. He said, ‘I was just learning to drive, and hadn’t yet started dating.’ He said the following few years were very difficult, but ‘then I got a guide dog, and I’m sitting on a park bench and someone sits down and tells me what a beautiful dog I have, and suddenly I’m going for coffee, then I’m going for dinner, and then I’m getting married … to the woman my guide dog most liked!’”