A Novel Machine Learning Algorithm: Music Arrangement and Timbre Transfer System

• Main Authors: Wang, Junyang, Sun, Wanzhen, Wu, Rubi, Fang, Yixuan, Liu, Ruibin, Li, Shaofei, Li, Zheng, Steven, Xin
• Format: Conference Proceeding
• Language: English
• Subjects: Accompaniment; Computational modeling; Instruments; Music; Music Composing; Music Generation; Neural networks; Rocks; Timbre Transfer; Training; Transformers
• ISSN: 2770-792X
• DOI: 10.1109/ICICSP55539.2022.10050687

Early neural network models have been used in image recognition and analysis. As time moved on, neural network usage expanded into music generation. Most generative models of music only focused on single-track generation. While much development had been produced, multi-track music composition and multi-instrumental band arrangements had been ignored. This paper introduced the Music Arrangement and Timbre Transfer System (MATTS) model, which further optimized existing models of Music Transformer and Differentiable Digital Signal Processing (DDSP) library to achieve computer music composition and instrumental band arrangements while maintaining realistic elements such as melodic sequences, repetitions, and imitations. Not only did it maintain realistic aspects, but our model also offered controls of pitch and amplitude. There were two forms of composition, where one with preconditioned user input and one without. MATTS could unconditionally compose and arrange a band to play the music it generated.