A recently developed brain-computer interface can turn brain signals into speech with an impressive accuracy of up to 97%. Researchers have implanted sensors in the brain of a man whose ability to speak was severely hindered by amyotrophic lateral sclerosis (ALS). Remarkably, he was able to articulate his intended speech just minutes after the system was activated.
The innovative brain-computer interface (BCI) created at UC Davis Health is capable of converting brain signals into speech with a remarkable accuracy rate of 97% – making it the most precise system of its kind.
The researchers installed sensors in the brain of a man suffering from severely impaired speech due to ALS. After activating the system, he could communicate what he wanted to say within minutes.
A study detailing this breakthrough was published today in the New England Journal of Medicine.
ALS, often referred to as Lou Gehrig’s disease, impacts the nerve cells responsible for controlling movement throughout the body. This condition results in a progressive loss of the ability to stand, walk, and use limbs, and can also impair the muscles used in speech, leading to unintelligible speech.
This new technology is aimed at helping individuals who are unable to speak due to paralysis or neurological disorders like ALS. It can decode brain signals generated when the user attempts to speak and convert them into text that is then “voiced” by the computer.
“Our BCI technology has enabled a man with paralysis to communicate effectively with his friends, family, and caregivers,” said David Brandman, a neurosurgeon at UC Davis. “Our findings demonstrate the highest accuracy of any speech neuroprosthesis (device) documented to date.”
Brandman is a co-principal investigator and co-senior author of this study, serving as an assistant professor in the UC Davis Department of Neurological Surgery and co-director of the UC Davis Neuroprosthetics Lab.
The innovative BCI bridges communication gaps
When an individual attempts to speak, the new BCI device converts their brain activity into text displayed on a screen, which the computer then reads aloud.
To develop this system, the team included Casey Harrell, a 45-year-old man diagnosed with ALS, in the BrainGate clinical trial. At the time of his participation, Harrell experienced weakness in his limbs (tetraparesis) and his speech was often incomprehensible (dysarthria), requiring assistance from others.
In July 2023, Brandman surgically implanted the experimental BCI device, positioning four microelectrode arrays in the left precentral gyrus, a brain area associated with speech coordination. Each array is designed to gather brain activity data from 256 cortical electrodes.
“We are essentially detecting the user’s effort to control their muscles and speak,” explained Sergey Stavisky, a neuroscientist and assistant professor in the Department of Neurological Surgery. He co-directs the UC Davis Neuroprosthetics Lab and is a co-principal investigator of the study. “We are monitoring the brain area that sends signals to the muscles and translating these patterns of brain activity into a phoneme—like a syllable or speech unit—and then into the words the person is trying to say.”
Quicker training, improved results
While BCI technology has made progress, previous communication systems have often been slow and inaccurate because machine-learning algorithms needed significant amounts of time and data to operate effectively.
“Earlier speech BCI systems frequently encountered word errors, making it challenging for users to be consistently understood, which hindered communication,” Brandman noted. “Our goal was to create a system that allowed individuals to be understood whenever they wished to speak.”
Harrell utilized the BCI in structured prompts and natural conversations. In both scenarios, speech decoding occurred in real-time, with ongoing updates to maintain accuracy.
The words decoded by the system appeared on a screen and, astonishingly, were vocalized in a voice resembling Harrell’s voice prior to his ALS diagnosis, generated using software trained on recordings of his voice before the onset of the disease.
During the initial speech training session, the system reached an incredible 99.6% word accuracy within just 30 minutes while using a vocabulary of 50 words.
“The first time we tested the system, he was overcome with emotion as the words he wanted to say appeared correctly on the screen. We all were,” Stavisky recalled.
In the second training session, the vocabulary expanded to 125,000 words. After just 1.4 additional hours of training data, the BCI achieved a word accuracy of 90.2% with this broader vocabulary. Following continued data collection, it maintained an accuracy of 97.5%.
“Currently, we are accurately interpreting Casey’s intended speech around 97% of the time, which surpasses many commercial smartphone apps aimed at recognizing a person’s voice,” Brandman stated. “This technology is revolutionary as it offers hope for individuals who yearn to communicate but are unable. I aspire that technology such as this speech BCI will enable future patients to engage in conversations with their loved ones.”
This study documents 84 data collection sessions spanning 32 weeks. In total, Harrell employed the speech BCI for over 248 hours to communicate in person and via video chat.
“The inability to communicate can be incredibly frustrating and disheartening. It feels as though you’re trapped,” stated Harrell. “An advancement like this can reintegrate individuals into life and society.”
“Witnessing Casey regain his ability to speak with his family and friends through this technology has been immensely gratifying,” remarked Nicholas Card, the lead author of the study and a postdoctoral scholar in the UC Davis Department of Neurological Surgery.
“Casey and our other BrainGate participants are remarkable individuals. They deserve immense recognition for participating in these early clinical trials, not for their personal gain, but to help us create a system that can restore communication and mobility for others.” said Leigh Hochberg, co-author and investigator for the BrainGate trial. Hochberg is a neurologist and neuroscientist affiliated with Massachusetts General Hospital, Brown University, and the VA Providence Healthcare System.
Brandman serves as the principal investigator for the BrainGate2 clinical trial, which is currently enrolling participants. For more details about the study, visit: https://www.braingate.org/