Maximum Mean Discrepancy and Generative Moment Matching Networks: Tutorial and Survey

Maximum Mean Discrepancy (MMD), also called the kernel two-sample test, is a measurement for difference of two probability distributions. It measures this difference by pulling data to the reproducing kernel Hilbert space and calculating the difference of their first moments in that space. Generative Moment Matching Network (GMMN) uses MMD between the output of network and the training dataset as its loss function. Conditional GMMN makes it possible to generate data from a specific pattern or class label. Moreover, it is possible to find the best characteristic kernel function used in MMD loss function of GMMN. This tutorial and survey paper introduces MMD, GMMN, conditional GMMN, and MMD GAN for finding the best characteristic kernel in the MMD loss function.