Perceptual quality assessment of JPEG, JPEG 2000, and JPEG XR

In this paper, we examine the rate-distortion performance in terms of perceptual quality of JPEG XR (ISO/IEC 29199-2 | ITU-T Rec. T.832)1 and earlier standardized image compression algorithms such as JPEG (ISO/IEC 10918-1 | ITU-T Rec. T.81)2 and JPEG 2000 (ISO/IEC 15444-1 | ITU-T Rec. T.800)3. Unfortunately, objective visual quality metrics (like MSE, PSNR, VQM, SSIM, etc.) do not always correlate well with the actual perceived image quality. In some specific cases, it is even possible that certain visible coding artifacts remain undetectable by these objective visual quality tests. As such, we conducted a series of subjective visual quality assessment tests to measure the visual performance of JPEG XR, JPEG 2000 and JPEG. This paper describes the design of the subjective visual quality assessment experiments, addressing the encountered difficulties and potential pitfalls. Our results indicate that for high bit-rates (i.e. more than 1 bpp) all three codecs more or less have an equal overall performance. However, as expected, at lower bit-rates JPEG performs significantly weaker for every tested image than JPEG 2000 and JPEG XR. On the other hand, both JPEG 2000 and JPEG XR appear to be very competitive at these low bit-rate ranges. Only for specific image content types (e.g. smooth gradient surfaces), JPEG XR appears to have some difficulties. Nevertheless, discarding the fact that JPEG 2000 offers more functionality features than JPEG XR, the latter performed very good for most images and almost in par with JPEG 2000. As a conclusion, the results of the subjective visual quality assessment tests show that JPEG XR successfully passed our verification experiments for low dynamic range imagery.