Explaining Spectrograms in Machine Learning: A Study on Neural Networks for Speech Classification

This study investigates discriminative patterns learned by neural networks for accurate speech classification, with a specific focus on vowel classification tasks. By examining the activations and features of neural networks for vowel classification, we gain insights into what the networks "see...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-07
Main Authors: Jesin, James, Balamurali, B T, Abeysinghe, Binu, Liu, Junchen
Format: Article
Language:English
Subjects:
Classification
Knowledge representation
Linguistics
Machine learning
Neural networks
Spectrograms
Speech recognition
Vowels
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This study investigates discriminative patterns learned by neural networks for accurate speech classification, with a specific focus on vowel classification tasks. By examining the activations and features of neural networks for vowel classification, we gain insights into what the networks "see" in spectrograms. Through the use of class activation mapping, we identify the frequencies that contribute to vowel classification and compare these findings with linguistic knowledge. Experiments on a American English dataset of vowels showcases the explainability of neural networks and provides valuable insights into the causes of misclassifications and their characteristics when differentiating them from unvoiced speech. This study not only enhances our understanding of the underlying acoustic cues in vowel classification but also offers opportunities for improving speech recognition by bridging the gap between abstract representations in neural networks and established linguistic knowledge
ISSN:2331-8422