Music can be reconstructed from human auditory cortex activity using nonlinear decoding models

• Published in: PLoS biology 2023-08, Vol.21 (8), p.e3002176-e3002176
• Main Authors: Bellier, Ludovic, Llorens, Anaïs, Marciano, Déborah, Gunduz, Aysegul, Schalk, Gerwin, Brunner, Peter, Knight, Robert T
• Format: Article
• Language: English
• Subjects: Ablation; Accuracy; Acoustics; Auditory cortex; Auditory perception; Biology and Life Sciences; Brain; Computer applications; Cortex (auditory); Cortex (temporal); Datasets; EEG; Electrodes; Harmony (Music); Hemispheric laterality; Human-computer interface; Implants; Medicine and Health Sciences; Music; Perception; Physical Sciences; Prediction models; Psychological aspects; Psychological research; Social Sciences; Song; Speech; Speech perception; Superior temporal gyrus; Temporal gyrus
• ISSN: 1545-7885; 1544-9173; 1545-7885
• DOI: 10.1371/journal.pbio.3002176

Music is core to human experience, yet the precise neural dynamics underlying music perception remain unknown. We analyzed a unique intracranial electroencephalography (iEEG) dataset of 29 patients who listened to a Pink Floyd song and applied a stimulus reconstruction approach previously used in the speech domain. We successfully reconstructed a recognizable song from direct neural recordings and quantified the impact of different factors on decoding accuracy. Combining encoding and decoding analyses, we found a right-hemisphere dominance for music perception with a primary role of the superior temporal gyrus (STG), evidenced a new STG subregion tuned to musical rhythm, and defined an anterior–posterior STG organization exhibiting sustained and onset responses to musical elements. Our findings show the feasibility of applying predictive modeling on short datasets acquired in single patients, paving the way for adding musical elements to brain–computer interface (BCI) applications.