R-D spatio-temporal adaptive quantization based on temporal distortion backpropagation in HEVC

Nowadays, Rate-Distortion Optimization (RDO) is commonly used in hybrid video coding to maximize coding efficiency. Usually, the rate distortion tradeoff is explicitly computed in offline encoder implementations whereas R(D) model are used in live encoders to select the best decisions at a lower computational cost. For sake of simplicity, this (mathematical) modelling is often performed for each coding unit (CU) individually and independently, obliterating the spatial or temporal dependency between CUs. In this paper, we provide a new spatio-temporal algorithm to compute local quantizers, based on a theoretical framework able to describe the temporal distortion propagation from an R-D standpoint. In particular, we model the temporal distortion propagation making possible the retro accumulations of any (spatial) psycho-visually weighted distortion onto reference images. Using the R(D) Shannon bound, its high bitrate approximation, and a Lagrange optimization, analytical solutions are obtained for the local quantizers and the Lagrange multiplier. The proposed algorithm shows -4.4% BD-BR SSIM gains in average over state-of-the art algorithm in HEVC, using the same SSIM-based psycho-visual function.