Cloud Hosting

By Jason Arunn Murugesu

Andrew Ostrovsky/Getty Images

An artificial intelligence can accurately translate thoughts into sentences, at least for a limited vocabulary of 250 words. The system may bring us a step closer to restoring speech to people who have lost the ability because of paralysis.

Joseph Makin at the University of California, San Francisco, and his colleagues used deep learning algorithms to study the brain signals of four women as they spoke. The women, who all haveepilepsy, already had electrodes attached to their brainsto monitor seizures.

Each woman was asked to read aloud from a set of sentences as the team measured brain activity. The largest group of sentences contained 250 unique words.

Advertisement

The team fed this brain activity to a neural network algorithm, training it to identify regularly occurring patterns that could belinked to repeated aspects ofspeech, such as vowels or consonants. These patterns werethen fed to a second neural network, which tried to turn them into words to form a sentence.

Each woman repeated the sentences at least twice, and the final repetition didnt formpart ofthe training data, allowingthe researchers to test the system.

Each time a person speaks the same sentence, the brain activity associated will be similar but not identical. Memorising the brain activity of these sentences wouldnt help, so the network instead has to learn whats similar about them so that it can generalise to this final example, says Makin. Across the four women, the AIs best performance was an average translation error rate of 3 per cent.

Makin says that using a small number of sentences made it easier for the AI to learn which words tend to follow others.

Forexample, the AI was able to decode that the word Turner wasalways likely to follow the word Tina in this set of sentences, from brain activity alone.

The team tried decoding the brain signal data into individual words at a time, rather than whole sentences, but this increased the error rate to 38 per cent even for the best performance. So the network clearly is learning facts about which words go together, and not just which neural activity maps to which words, says Makin.

This will make it hard toscale up the system to a larger vocabulary because each new word increases the number of possible sentences, reducing accuracy.

Makin says 250 words could still be useful for people who cant talk. We want to deploy this in a patient with an actual speech disability, he says, although it is possible their brain activity may be different from that of the women in this study, making this more difficult.

Sophie Scott at University College London says we are a long way from being able to translate brain signal data comprehensively. You probably know around 350,000 words, so its still an incredibly restricted set of speech that theyre using, she says.

Journal reference: Nature Neuroscience, DOI: 10.1038/s41593-020-0608-8

Go here to see the original:
Mind-reading AI turns thoughts into words using a brain implant - New Scientist News