Training a Singing Transcription Model Using Connectionist Temporal Classification Loss and Cross-Entropy Loss

• Published in: IEEE/ACM transactions on audio, speech, and language processing speech, and language processing, 2023, Vol.31, p.383-396
• Main Authors: Wang, Jun-You, Jang, Jyh-Shing Roger
• Format: Article
• Language: English
• Subjects: Classification; Computational modeling; connectionist temporal classification loss; Data models; Datasets; Entropy; Hidden Markov models; Instruments; music information retrieval; Performance evaluation; Predictive models; Singing; Singing transcription; Task analysis; Training
• ISSN: 2329-9290; 2329-9304
• DOI: 10.1109/TASLP.2022.3224297

In this paper, we propose a method that uses a combination of the Connectionist Temporal Classification (CTC) loss and the cross-entropy loss to train a note-level singing transcription model. By considering the task as predicting a note sequence of the input audio, we can compute the CTC loss between the prediction and the groundtruth note sequence, and further use it with the traditional cross-entropy loss to optimize the transcription model. By comparing the proposed method with a baseline that only utilizes the cross-entropy loss, the results show improved model performance on all the evaluation metrics. Furthermore, using the CTC loss allows the transcription model to learn from weakly labeled data, which is easier to annotate than traditional strongly labeled data. Moreover, we point out the issue of the intrinsic global time shift on the onset labels between datasets. By automatically estimating and calibrating the global time shift of the training dataset, the performance of the singing transcription model is then not affected by the global time shift in the cross-dataset evaluation scenario.