Researchers at the University of Texas at Austin have developed a non-invasive device that can reconstruct continuous language from brain activity. Nature Neuroscience reports this. The new algorithm reconstructs continuous language from cortical semantic representations recorded via functional magnetic resonance imaging (fMRI).
Previously, non-invasive language decoders could only recognize a small number of words or phrases from a given list. The new decoder, however, produces intelligible word sequences that recover the meaning of heard speech, imagined speech, and even silent videos. This shows that a single decoder can be applied to different tasks.
The researchers tested the decoder in different regions of the cerebral cortex and found that continuous language can be decoded separately from several regions. Since brain-computer interfaces must respect mental privacy, scientists checked whether successful decoding requires the subject's cooperation. It turned out that cooperation is necessary both for training the decoder and for applying it.
During the experiments, participants listened to stories, imagined telling them, and watched silent videos. The decoder was able to recover 72-82% of the time points of heard speech, 41-74% of imagined speech, and 21-45% of watched films. When participants actively resisted by counting mentally or listing animal names, decoding efficiency dropped significantly, reaching 0-3%.
The study shows that the use of non-invasive language brain-computer interfaces is realistic, but they still require the active cooperation of the subject and cannot be used without the person's knowledge.
This research shows that mind-reading technology is gradually moving beyond the realm of science fiction. Although the current decoder requires an fMRI scanner, which is a large and expensive device, researchers are already thinking about more affordable and portable solutions. They note that with the permissions of current portable systems, it will be possible to decode about 50% of stimulus time points.
Particularly important is the fact that participants' resistance almost completely halts decoding. This means that the technology cannot yet be used independently of a person's will. However, over time, as devices become more sensitive and affordable, the issue of mental privacy will become more acute.
The authors of the study speak openly about these risks and note that their data have not been made publicly available, fearing that they could be used to find ways to bypass subjects' resistance. This shows that the scientific community is aware of the ethical consequences of such technologies.
