Loading…
Classifying Chinese Medicine Constitution Using Multimodal Deep-Learning Model
Objective To develop a multimodal deep-learning model for classifying Chinese medicine constitution, i.e., the balanced and unbalanced constitutions, based on inspection of tongue and face images, pulse waves from palpation, and health information from a total of 540 subjects. Methods This study dat...
Saved in:
Published in: | Chinese journal of integrative medicine 2024-02, Vol.30 (2), p.163-170 |
---|---|
Main Authors: | , , |
Format: | Article |
Language: | English |
Subjects: | |
Citations: | Items that this one cites Items that cite this one |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Summary: | Objective
To develop a multimodal deep-learning model for classifying Chinese medicine constitution, i.e., the balanced and unbalanced constitutions, based on inspection of tongue and face images, pulse waves from palpation, and health information from a total of 540 subjects.
Methods
This study data consisted of tongue and face images, pulse waves obtained by palpation, and health information, including personal information, life habits, medical history, and current symptoms, from 540 subjects (202 males and 338 females). Convolutional neural networks, recurrent neural networks, and fully connected neural networks were used to extract deep features from the data. Feature fusion and decision fusion models were constructed for the multimodal data.
Results
The optimal models for tongue and face images, pulse waves and health information were ResNet18, Gate Recurrent Unit, and entity embedding, respectively. Feature fusion was superior to decision fusion. The multimodal analysis revealed that multimodal data compensated for the loss of information from a single mode, resulting in improved classification performance.
Conclusions
Multimodal data fusion can supplement single model information and improve classification performance. Our research underscores the effectiveness of multimodal deep learning technology to identify body constitution for modernizing and improving the intelligent application of Chinese medicine. |
---|---|
ISSN: | 1672-0415 1993-0402 |
DOI: | 10.1007/s11655-022-3541-8 |