Classifying sex and strain from mouse ultrasonic vocalizations using deep learning

• Published in: PLoS computational biology 2020-06, Vol.16 (6), p.e1007918-e1007918
• Main Authors: Ivanenko, A, Watkins, P, van Gerven, M. A. J, Hammerschmidt, K, Englitz, B, Graham, Lyle J, Theunissen, Frédéric E
• Format: Article
• Language: English
• Subjects: Animal cognition; Animal vocalizations; Artificial neural networks; Biology and Life Sciences; Brain research; Classification; Computer and Information Sciences; Computer architecture; Deep learning; Feature extraction; Females; Gender aspects; House mouse; Identification and classification; Investigations; Line spectra; Machine learning; Neural networks; Neurophysiology; Physical Sciences; Physiological aspects; Principal components analysis; Regression analysis; Research and Analysis Methods; Sex; Sex differences; Sexes; Social factors; Social interaction; Social interactions; Social Sciences; Support vector machines; Vocalization behavior
• ISSN: 1553-7358; 1553-734X; 1553-7358
• DOI: 10.1371/journal.pcbi.1007918

Vocalizations are widely used for communication between animals. Mice use a large repertoire of ultrasonic vocalizations (USVs) in different social contexts. During social interaction recognizing the partner's sex is important, however, previous research remained inconclusive whether individual USVs contain this information. Using deep neural networks (DNNs) to classify the sex of the emitting mouse from the spectrogram we obtain unprecedented performance (77%, vs. SVM: 56%, Regression: 51%). Performance was even higher (85%) if the DNN could also use each mouse's individual properties during training, which may, however, be of limited practical value. Splitting estimation into two DNNs and using 24 extracted features per USV, spectrogram-to-features and features-to-sex (60%) failed to reach single-step performance. Extending the features by each USVs spectral line, frequency and time marginal in a semi-convolutional DNN resulted in a performance mid-way (64%). Analyzing the network structure suggests an increase in sparsity of activation and correlation with sex, specifically in the fully-connected layers. A detailed analysis of the USV structure, reveals a subset of male vocalizations characterized by a few acoustic features, while the majority of sex differences appear to rely on a complex combination of many features. The same network architecture was also able to achieve above-chance classification for cortexless mice, which were considered indistinguishable before. In summary, spectrotemporal differences between male and female USVs allow at least their partial classification, which enables sexual recognition between mice and automated attribution of USVs during analysis of social interactions.