MRGAN: Multi-Criteria Relational GAN for Lyrics-Conditional Melody Generation

Music generation, as a creativity problem, attracts growing attention from artificial intelligence researchers. Among the challenging tasks, lyrics-conditional melody generation aims to leverage natural language processing (NLP) techniques to generate music from texts, for which Generative Adversarial Networks (GAN) has become a promising unsupervised solution. The adversarial training of two agents, i.e., generator and discriminator, allows GAN to achieve a better generation performance and has been proven effective in conditional generation tasks. In this paper, we propose the multi-criteria relational GAN (MRGAN), which includes a relation memory-based generator and two discriminators with a unique discrimination criterion each. The relational memory in the generator is adopted for long-time dependency modeling. Meanwhile, the two discriminators can judge both musical quality and conditional correspondence. Based on the bilingual evaluation understudy (BLEU) score, a new metric, named Music-BLEU, has also be designed to evaluate the lyrics-conditional melody generation. Experimental results verify that MRGAN outperforms existing approaches in related key metrics.