Decoder device translates brain activity into speech

July 5, 2019 0 By FM

Gopala K. Anumanchipalli et al have developed a speech decoding device that can transform brain signals into speech. The neural decoder translated directly recorded human cortical activity into representations of articulatory movement which are then transformed into audible speech. The researchers worked with five participants who had electrodes implanted on the surface of their brains as part of epilepsy treatment. Brain activity of the participants was recorded as they read hundreds of sentences aloud. The researchers combined these recordings with data from previous experiments that determined how movements of the tongue, lips, jaw and larynx created sound. The team developed a deep-learning algorithm on these data, and then incorporated the programme into the decoder. The device could transform brain signals into estimated movements of the vocal tract and turn the movements into synthetic speech.

The scientists enrolled people to listen to the synthesised speech and transcribe them. People who listened to 101 synthesized sentences reported to have understood 70% of the words on average. The decoder could also synthesize speech when a participant silently mimed sentences. The findings show the clinical viability of speech neuroprosthetic technology to help restore spoken communication in paralyzed individuals.

Source: Nature volume 568, pages493–498 24 April 2019