A cross-modal adaptation approach for brain decoding

Brain decoding has become a hot topic in many recent brain studies. In a typical neuroimaging experiment, participants are presented with different categories of stimuli while their concurrent brain activity is recorded. Then a classifier is trained on the features extracted from the recorded brain...

Full description

Saved in:
Bibliographic Details
Main Authors: Ghaemmaghami, Pouya, Nabi, Moin, Yan Yan, Riccardi, Giuseppe, Sebe, Nicu
Format: Conference Proceeding
Language:English
Subjects:
brain decoding
Decoding
Dictionaries
domain adaptation
Electroencephalography
Feature extraction
genre classification
Motion pictures
Multimedia communication
multimedia information retrieval
signal processing
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Brain decoding has become a hot topic in many recent brain studies. In a typical neuroimaging experiment, participants are presented with different categories of stimuli while their concurrent brain activity is recorded. Then a classifier is trained on the features extracted from the recorded brain data to discriminate different target stimuli classes. It is a common practice to hypothesize that the stimulus-related information exists in the brain data if the decoder can accurately predict the target stimulus category. However, most of the neuroimaging studies suffer from few and noisy samples. These constraints affects the performance of such decoding systems. In order to cope with this limitation, a dictionary learning approach is used in this paper to transfer knowledge from the multimedia domain to the brain domain. We show that such cross-modal domain adaptation yields better performance of the learning algorithm in the brain domain. This is the first study in the direction of cross-modal adaptation by joint dictionary learning on multimedia and brain modality.
ISSN:2379-190X
DOI:10.1109/ICASSP.2017.7952300