The 2018 PIRM Challenge on Perceptual Image Super-resolution

About

This paper reports on the 2018 PIRM challenge on perceptual super-resolution (SR), held in conjunction with the Perceptual Image Restoration and Manipulation (PIRM) workshop at ECCV 2018. In contrast to previous SR challenges, our evaluation methodology jointly quantifies accuracy and perceptual quality, therefore enabling perceptual-driven methods to compete alongside algorithms that target PSNR maximization. Twenty-one participating teams introduced algorithms which well-improved upon the existing state-of-the-art methods in perceptual SR, as confirmed by a human opinion study. We also analyze popular image quality measures and draw conclusions regarding which of them correlates best with human opinion scores. We conclude with an analysis of the current trends in perceptual SR, as reflected from the leading submissions.

Yochai Blau, Roey Mechrez, Radu Timofte, Tomer Michaeli, Lihi Zelnik-Manor · 2018

Related benchmarks

Task	Dataset	Result
No-Reference Image Quality Assessment	SPAQ	SROCC0.709	136
Image Quality Assessment	KADID-10k (test)	SRCC—	101
Image Quality Assessment	PIPAL NTIRE 2021 IQA Challenge (test)	PLCC0.166	32
Image Quality Assessment Consistency	SRIQA-Bench (test)	Mean SRCC0.616	11
Image Quality Assessment Consistency	BSD-SR (test)	SRCC (Mean)87.5	11
Image Quality Assessment Consistency	QADS (test)	Mean SRCC0.76	11
Image Quality Assessment Consistency	DIV2K (wild)	SSIM—	3
Image Quality Assessment Consistency	RealSR (test)	SSIM—	3
Image Quality Assessment Consistency	DRealSR (test)	SSIM—	3
Image Quality Assessment Consistency	Set5 & Set14	SSIM—	3