Decoding Speech from ECoG with Machine Translation Models
Loading...
Date
2023
Authors
Burakov, Roman
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This paper explores the use and improvement of brain-computer interface (BCI)- based speech neuroprostheses, devices designed to enhance communication for individuals with speech disorders. Focusing on the machine learning aspect, we address the existing challenges associated with these systems, such as the limited vocabulary and simple algorithms of previous research and the individual variances in electrode implantation sites. Our approach reframes the decoding of speech from BCI as a machine translation problem and employs existing language models for semantic knowledge transfer. This research provides an extensive analysis of current neural speech decoding and multilingual neural machine translation methods, adapts the pre-existing M2M100 neural machine translation model for decoding ECoG data into text, and introduces a state-of-the-art model for neural speech decoding that improves upon current methods in semantic text reconstructions.
Description
Keywords
Word Error Rate, BLEU score, BERTScore, decoding speech with machine translation models, acknowledgements, bachelor's thesis