MuseCNN: Embedding-Guided Polyphonic Music Accompaniment Generation

Although various methods are proposed to generate music accompaniment tracks according to the main music track, they suffer from the problems of modeling music dependencies and representing music data, therefore, generating high-quality music accompaniment remains a challenging task. To address this issue, we propose a multi-track sequential convolutional neural network (MuseCNN) to generate accompaniment tracks corresponding to the main music track. Inspired by the similarity between pianoroll representation and pictures, we transform the pianoroll data into a two-channel music representation matrix and feed it into the convolutional neural network (CNN). Using a hierarchical loss function, we integrate all music tracks to maintain the coherence and harmony of the music. Based on the three-level loss design and multiple CNNs, the problem of modeling music dependencies can be solved. The experimental results indicate that our multi-CNN model can effectively learn complex music dependencies and generate harmonious long-sequence polyphonic music.