Neural decoding of semantic concepts: a systematic literature review

• Published in: Journal of neural engineering 2022-04, Vol.19 (2), p.21002
• Main Authors: Rybář, Milan, Daly, Ian
• Format: Article
• Language: English
• Subjects: Brain - diagnostic imaging; conceptual decoding; electroencephalography (EEG); functional magnetic resonance imaging (fMRI); functional near infrared spectroscopy (fNIRS); intracranial electrodes; literature review; Machine Learning; Magnetic Resonance Imaging; Neuroimaging; semantic decoding; Semantics
• ISSN: 1741-2560; 1741-2552
• DOI: 10.1088/1741-2552/ac619a

Semantic concepts are coherent entities within our minds. They underpin our thought processes and are a part of the basis for our understanding of the world. Modern neuroscience research is increasingly exploring how individual semantic concepts are encoded within our brains and a number of studies are beginning to reveal key patterns of neural activity that underpin specific concepts. Building upon this basic understanding of the process of semantic neural encoding, neural engineers are beginning to explore tools and methods for semantic decoding: identifying which semantic concepts an individual is focused on at a given moment in time from recordings of their neural activity. In this paper we review the current literature on semantic neural decoding. We conducted this review according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. Specifically, we assess the eligibility of published peer-reviewed reports via a search of PubMed and Google Scholar. We identify a total of 74 studies in which semantic neural decoding is used to attempt to identify individual semantic concepts from neural activity. Our review reveals how modern neuroscientific tools have been developed to allow decoding of individual concepts from a range of neuroimaging modalities. We discuss specific neuroimaging methods, experimental designs, and machine learning pipelines that are employed to aid the decoding of semantic concepts. We quantify the efficacy of semantic decoders by measuring information transfer rates. We also discuss current challenges presented by this research area and present some possible solutions. Finally, we discuss some possible emerging and speculative future directions for this research area. Semantic decoding is a rapidly growing area of research. However, despite its increasingly widespread popularity and use in neuroscientific research this is the first literature review focusing on this topic across neuroimaging modalities and with a focus on quantifying the efficacy of semantic decoders.