UC researchers develop speech therapy video game

Speech language pathologist Suzanne Boyce, PhD, is the principal investigator of a research team that has received a patent and a National Institutes of Health (NIH) grant to develop video game biofeedback for speech learning. The video game builds off Boyce’s previous work in using ultrasound to overcome difficulties with speech.

Her team includes postdoctoral researcher Sarah Dugan, PhD, who also is a speech language pathologist; Michael Riley, PhD, whose work in human perceptual-motor behavior inspired the game; T. Douglas Mast, PhD, a biomedical engineering professor who has helped program the ultrasound tracking software; and Renee Seward, an endowed associate professor of communication design in the College of Design, Architecture, Art, and Planning (DAAP) who has provided her expertise in making visual displays that are understandable and actionable.

“Different kids respond differently to using the ultrasound for speech therapy, and we expect that to happen,” Dugan said. “While it is not a magic wand, we are seeing overall that after using ultrasound, kids are able to produce ‘R’ in words that were originally a challenge for them to say.”

Speech disorders are most common in children, but they can persist into adulthood, with approximately 2% of college-age students having difficulties with pronunciation, said Boyce, a professor in the Department of Communication Sciences and Disorders within UC’s College of Allied Health Sciences.

In English, “R” is one of the sounds that most commonly causes issues. Many speech language pathologists refer to the condition as “the dreaded ‘R’.”

“Basically, two parts of your tongue have to move in opposite directions, which is not something you get very often with other speech sounds,” Boyce said about why “R” is so hard for some people. “Basically, it’s a new skill that children need to learn to do with their tongue, and for some reason, some children seem to just miss that skill in the course of learning to talk.”

Researcher Dugan, who received master’s and doctoral degrees from UC in speech language pathology, was inspired to create visuals to aid in speech therapy after watching UC professor Riley’s research on ACL injury prevention.

Riley studies human movement with a focus on rehabilitation and human performance. He has worked with athletes in collaboration with Cincinnati Children’s on augmented neuromuscular training.

To learn to prevent ACL injuries, athletes wore sensors on their bodies along with goggles over their eyes. As they did squats, the sensors tracked their movements and created shapes that were displayed in the goggles. When they incorrectly performed the exercise, the shapes would deform, helping them to implicitly learn how to move their bodies.

“I was watching this, and I was thinking, we could absolutely do that with ultrasound,” Dugan said. “We could take this complicated tongue movement and transform that into a simple, actionable object that kids could follow and understand.”

At that time, UC already was using ultrasound, which is commonly associated with checking a baby’s development during pregnancy, to assist with speech therapy. Since 2012, the UC Speech & Hearing Clinic has offered a speech therapy program using ultrasound. An ultrasound wand is placed under a patient’s mouth, displaying tongue movement on a screen while the patient talks.

For many kids, the visual feedback helps them make progress because they can see how their tongue is moving for the first time. For others, deciphering an ultrasound can be difficult.

“It’s kind of hard to understand what you’re seeing for a while,” Dugan said. “A lot of kids figure it out, but there are some kids who struggle to make sense of the ultrasound image on the screen.”

When people learn implicitly, when they’re not focusing on their body, they tend to learn a complex motor skill faster and more robustly.

Sarah Dugan, PhD Postdoctoral researcher and speech language pathologist

The UC research team created the video game to make it easier for kids to understand the visual feedback created by ultrasound.

Their game has gone through multiple iterations based on feedback from children. The current version is a side-scrolling game, like classic Super Mario games by Nintendo, but this game features a goat on roller skates that jumps from platform to platform in the mountains.

The patient’s tongue acts as the controller, allowing them to make the goat move and jump by making the correct movements with their tongue.

“A lot of this detail has been given to us by kids who wanted different things, and we have a laundry list of things we can still implement, as you can imagine,” Dugan said.

Gamifying speech therapy has multiple benefits.

For one, the therapy seems more fun and less like work when it’s turned into a game.

Also, games can help with implicit learning, or learning without intention, which has shown to be an effective method.

“When people learn implicitly, when they’re not focusing on their body, they tend to learn a complex motor skill faster and more robustly,” Dugan said. “If you’ve ever played golf or tennis: the moment you start focusing on your body, you tend to mess up the thing you are trying to do. The challenge with speech language pathology is: how do we tell someone to change something that they’re doing with their tongue while not talking about their tongue?”

The UC researchers will test the effectiveness of their speech therapy game this summer.

They have scheduled two two-week camps during which children ages 7 to 17 will receive therapy focusing on “R” sounds. One week will use ultrasound technology alone while the other week will incorporate the game.

If the methods yield positive results, they could be used for other speech disorders beyond the “R” sound and also could help adults.

“The technique is applicable to other sounds, but our design has focused on one sound because if you can demonstrate it for the most commonly mispronounced sound, that makes it more likely that we can fix other sounds,” Boyce said.

The researchers would be interested in expanding the technology beyond UC.

As graduates of UC’s Venture Lab, members of the research team have learned about creating a startup business.They are actively seeking a business partner to make the project commercially available to other speech-language pathologists, hospitals and schools.

More than 25 students from the UC colleges of Allied Health Sciences, Engineering and DAAP have contributed to the project. Seward and her DAAP students received an honorable mention award from the Design Management Institute for their contribution to the project.

“The students will come up with something, and they’ll say, ‘Let’s see if I can do it,’” Dugan said of the students’ contributions. “They figure it out. They have that drive to solve problems that others have found to be impossible, and they do it every time.”

Innovation Lives Here