Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain Adaptation in Neural Machine Translation

Neural Machine Translation (NMT) models have achieved comparable results to human translation with a large number of parallel corpora available. However, their performance remains poor when translating on new domains with a limited number of data. Recent studies either only show the model's robustness to domain shift or the superiority in adapting to new domains with a limited number of data. A solution for addressing both the model's robustness and adaptability is underexplored. In this paper, we present a novel approach Epi-Curriculum to address low-resource domain adaptation (DA), which contains a new episodic training framework along with a denoised curriculum learning. Our episodic training framework enhances the model's robustness to domain shift by episodically exposing the encoder/decoder to an inexperienced decoder/encoder. The denoised curriculum learning filters the noised data and further improves the model's adaptability by gradually guiding the learning process from easy to more difficult tasks. Extensive experiments have been conducted on English-German (En-De), English-Romanian (En-Ro), and English-French (En-Fr) translation tasks. Our results show that: (i) Epi-Curriculum outperforms the baseline on unseen and seen domains by 2.28 and 3.64 BLEU score on En-De task, and 3.32 and 2.23 on En-Ro task; (ii) Our episodic training framework outperforms the recent popular meta-learning framework in terms of robustness to domain shift and achieves comparable adaptability to new domains.