MFCT: Multi-Frequency Cascade Transformers for no-reference SR-IQA

• Published in: Computer vision and image understanding 2024-11, Vol.248, p.104104, Article 104104
• Main Authors: Fan, Dandan, Zhang, Kaibing, Li, Hui, Ren, Longgang, Shi, Guang
• Format: Article
• Language: English
• Subjects: Cascade transformers; Multi-frequency bands; Super-resolution image quality assessment
• ISSN: 1077-3142
• DOI: 10.1016/j.cviu.2024.104104

Super-resolution image reconstruction techniques have advanced quickly, leading to the generation of a sizable number of super-resolution images using different super-resolution techniques. Nevertheless, accurately assessing the quality of super-resolution images remains a formidable challenge. This paper introduces a novel Multi-Frequency Cascade Transformers (MFCT) for evaluating super-resolution image quality (SR-IQA). In the first step, we develop a unique Frequency-Divided Module (FDM) to transform the super-resolution images into three different frequency bands. Subsequently, the Cascade Transformer Blocks (CAF) incorporating hierarchical self-attention mechanisms are employed to capture cross-window features for quality perception. Ultimately, the image quality scores from different frequency bands are fused to derive the overall image quality score. The experimental results show that, on the chosen SR-IQA databases, the proposed MFCT-based SR-IQA method can consistently outperforms all the compared Image Quality Assessment (IQA) models. Furthermore, a collection of thorough ablation studies demonstrates that, when compared to other earlier rivals, the newly proposed approach exhibits impressive generalization ability. The code will be available at https://github.com/kbzhang0505/MFCT. [Display omitted] This paper introduces a novel approach for evaluating super-resolution image quality. In the first step, we develop a unique Frequency-divided module (FDM) to transform the super-resolution images into three different frequency bands. Subsequently, the cascade transformer blocks (CAF) incorporating hierarchical self-attention mechanisms are employed to capture cross-window features for quality perception. Ultimately, the image quality scores obtained from different frequency bands are fused to derive the overall image quality score. •Frequency-divided module extracts perceptual features to quantify the quality of SR images.•Cascade transformer blocks captures cross-window features for quality perception.•Different frequency bands jointly estimate the overall quality score of an SR image.