Chen 等人:Simple Baselines for Image Restoration(NAFNet,ECCV 2022)
選讀・DLCV 01B・網路的積木與組合:分類與影像復原網路 的論文導讀(第 1/4 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
挑戰
Chen 等人 [19] 問的是:影像修復真的需要愈來愈複雜的區塊嗎?他們保留單階段 U-Net,先以受控增量建立有競爭力的基準,再移除能被簡單閘門取代的元件。這是在區分整體架構與區塊設計的貢獻,不是宣稱所有非線性計算都不需要。
方法

圖片來源: [19], PDF 第 6 頁. 原始 PDF.
先建立可信的基準,再簡化。 從卷積網路出發,逐步加入層正規化、將 ReLU 換為 GELU,並加入通道注意力。穩定的基準才能用來評估簡化是否有效;正規化實驗同時改變學習率,歸因時必須考慮這點。
以乘法取代具名啟用函數。 逐點擴展與逐通道卷積後,把偶數通道的特徵切成相同大小的 ,SimpleGate 定義為:
表示通道串接, 為逐元素乘法;輸出通道數減半。這仍是非線性運算:兩條支路都放大 倍,乘積會放大 倍。「Nonlinear Activation Free」指移除 GELU 等傳統啟用函數,並非把網路變成線性濾波器。
通道注意力也能簡化。 對閘控後的特徵 ,簡化通道注意力可寫成:
對空間位置取平均, 是學得的通道投影,所得通道向量廣播到各空間位置。移除原本注意力的瓶頸與啟用函數後,仍保留依輸入而變的乘法閘門。
NAF 區塊的第一條殘差路徑包含正規化、擴展、逐通道濾波、SimpleGate、SCA 與投影;第二條正規化前饋路徑再使用 SimpleGate 與殘差相加。可學習的殘差尺度穩定初始映射。編碼器—解碼器仍負責跨解析度傳遞資訊,簡單區塊不能取代跳接與多尺度路徑。
讀圖與選用判斷。 沿著區塊追蹤特徵張量:正規化整理輸入,空間卷積混合鄰近資訊,SimpleGate 將兩組通道相乘,再進行通道加權與殘差相加。即使沒有具名的活化函數,乘法仍使映射具有非線性。評估替換區塊時,要固定修復任務、寬度、訓練流程與推論轉換;否則無法把進步歸因於移除活化函數。它適合有成對訓練資料時作為精簡的學習式基準,但不會估計物理模糊核。
結果
Table 6 的完整去雜訊模型在 SIDD 達 40.30 dB/0.962 SSIM,成本 65G MACs;較強基準顯示相同品質,但需要 84G MACs。這些完整模型結果須與較小的消融設定分開。
Table 4 的最終去模糊設定在 GoPro 達 33.69 dB/0.967 SSIM,其中包含測試時局部轉換;不能把全部進步都歸給區塊本身。
消融實驗
Table 1 累積建立基準:PlainNet 在學習率 時 GoPro PSNR 為 28.51,改成 的訓練不穩定;正規化讓較大學習率可行,達 31.90。這組比較同時改變訓練穩定性與最佳化設定。
Table 2 的 GoPro 基準為 32.35 dB,只換 SimpleGate 為 32.76、只換 SCA 為 32.54、兩者一起換為 32.85;SIDD 對應 39.85、39.93、39.95、39.96。不同工作對元件的受益幅度不同。
Table 5 重新在閘門內加入啟用函數,有些設定讓 SIDD 略升、GoPro 卻下降;支持簡單設計有用,未證明某種啟用函數在所有工作都最佳。Table 3 也顯示增加深度會增加延遲,品質並非全面單調上升。
結論
NAFNet 的論點是:有良好基準、乘法閘門與受控消融的樸素架構,可以匹敵更複雜的修復模型。必須區分「拿掉具名啟用函數」與「拿掉非線性特徵互動」。
限制與問題
簡化區塊不會消除訓練資料、記憶體與全域池化的需求。區塊訓練與完整影像推論的池化統計可能不同,因此論文評估局部轉換。成對去雜訊/去模糊結果不能直接外推到任意真實退化;相同運算量也不等於相同裝置延遲。
參考文獻
-
K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. CVF · arXiv
-
Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell and S. Xie, “A ConvNet for the 2020s,” in Proc. IEEE/CVF CVPR, 2022. CVF · arXiv
-
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in International Conference on Learning Representations (ICLR), 2021. arXiv
-
J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool and R. Timofte, “SwinIR: Image Restoration Using Swin Transformer,” in Proc. IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021. CVF · arXiv
-
T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, “Feature Pyramid Networks for Object Detection,” in Proc. IEEE CVPR, 2017. CVF · arXiv
-
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin and B. Guo, “Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2021. CVF · arXiv
-
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy et al., “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015. doi
-
W. Yu, M. Luo, P. Zhou, C. Si, Y. Zhou, X. Wang, J. Feng and S. Yan, “MetaFormer Is Actually What You Need for Vision,” in Proc. IEEE/CVF CVPR, 2022. CVF · arXiv
-
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” arXiv:1704.04861, 2017. arXiv
-
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in Proc. IEEE/CVF CVPR, 2018. CVF · arXiv
-
K. He, X. Zhang, S. Ren and J. Sun, “Identity Mappings in Deep Residual Networks,” in Proc. European Conference on Computer Vision (ECCV), LNCS, Springer, 2016. doi · arXiv
-
B. Lim, S. Son, H. Kim, S. Nah and K. M. Lee, “Enhanced Deep Residual Networks for Single Image Super-Resolution,” in Proc. IEEE CVPR Workshops (CVPRW), 2017. CVF · arXiv
-
H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve and H. Jégou, “Going Deeper With Image Transformers,” in Proc. IEEE/CVF ICCV, 2021. CVF
-
L. Chen, X. Chu, X. Zhang and J. Sun, NAFNet official implementation,
basicsr/models/archs/NAFNet_arch.py, GitHub (megvii-research/NAFNet), accessed 2026. code -
Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong and Y. Fu, “Image Super-Resolution Using Very Deep Residual Channel Attention Networks,” in Proc. ECCV, 2018. CVF · arXiv
-
I. Radosavovic, R. P. Kosaraju, R. Girshick, K. He and P. Dollár, “Designing Network Design Spaces,” in Proc. IEEE/CVF CVPR, 2020. CVF · arXiv
-
J. Hu, L. Shen and G. Sun, “Squeeze-and-Excitation Networks,” in Proc. IEEE/CVF CVPR, 2018. CVF · arXiv
-
S. Woo, J. Park, J.-Y. Lee and I. S. Kweon, “CBAM: Convolutional Block Attention Module,” in Proc. ECCV, 2018. CVF · arXiv
-
L. Chen, X. Chu, X. Zhang and J. Sun, “Simple Baselines for Image Restoration,” in Proc. ECCV, LNCS, Springer, 2022. doi · arXiv
-
M.-H. Guo, T.-X. Xu, J.-J. Liu, Z.-N. Liu, P.-T. Jiang, T.-J. Mu, S.-H. Zhang, R. R. Martin, M.-M. Cheng and S.-M. Hu, “Attention Mechanisms in Computer Vision: A Survey,” Computational Visual Media, vol. 8, no. 3, pp. 331–368, 2022. doi · arXiv
-
S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan and M.-H. Yang, “Restormer: Efficient Transformer for High-Resolution Image Restoration,” in Proc. IEEE/CVF CVPR, 2022. CVF · arXiv
-
X. Chen, Z. Li, Y. Pu, Y. Liu, J. Zhou, Y. Qiao and C. Dong, “A Comparative Study of Image Restoration Networks for General Backbone Network Design,” in Proc. ECCV 2024, LNCS, Springer, 2025. doi · arXiv
-
S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” in Proc. International Conference on Machine Learning (ICML), PMLR 37, 2015. PMLR
-
J. L. Ba, J. R. Kiros and G. E. Hinton, “Layer Normalization,” arXiv:1607.06450, 2016. arXiv
-
D. Ulyanov, A. Vedaldi and V. Lempitsky, “Instance Normalization: The Missing Ingredient for Fast Stylization,” arXiv:1607.08022, 2016. arXiv
-
Y. Wu and K. He, “Group Normalization,” in Proc. ECCV, 2018. CVF · arXiv
-
C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang and W. Shi, “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” in Proc. IEEE CVPR, 2017. CVF
-
X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao and C. C. Loy, “ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks,” in Proc. ECCV Workshops, 2018. CVF · arXiv
-
K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017. doi · arXiv
-
Q. Huynh-Thu and M. Ghanbari, “Scope of Validity of PSNR in Image/Video Quality Assessment,” Electronics Letters, vol. 44, no. 13, pp. 800–801, 2008. doi
-
A. Abdelhamed, S. Lin and M. S. Brown, “A High-Quality Denoising Dataset for Smartphone Cameras,” in Proc. IEEE/CVF CVPR, 2018. CVF
-
S. Nah, T. H. Kim and K. M. Lee, “Deep Multi-Scale Convolutional Neural Network for Dynamic Scene Deblurring,” in Proc. IEEE CVPR, 2017. CVF
-
N. Shazeer, “GLU Variants Improve Transformer,” arXiv:2002.05202, 2020. arXiv
-
Y. N. Dauphin, A. Fan, M. Auli and D. Grangier, “Language Modeling with Gated Convolutional Networks,” in Proc. ICML, PMLR 70, 2017. PMLR
-
D. Hendrycks and K. Gimpel, “Gaussian Error Linear Units (GELUs),” arXiv:1606.08415, 2016. arXiv
-
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert and Z. Wang, “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network,” in Proc. IEEE CVPR, 2016. CVF · arXiv
-
A. Odena, V. Dumoulin and C. Olah, “Deconvolution and Checkerboard Artifacts,” Distill, vol. 1, no. 10, 2016. Distill · doi
-
A. Aitken, C. Ledig, L. Theis, J. Caballero, Z. Wang and W. Shi, “Checkerboard Artifact Free Sub-Pixel Convolution: A Note on Sub-Pixel Convolution, Resize Convolution and Convolution Resize,” arXiv:1707.02937, 2017. arXiv
-
O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Proc. Medical Image Computing and Computer-Assisted Intervention (MICCAI), LNCS, Springer, pp. 234–241, 2015. doi · arXiv
-
G. Huang, Z. Liu, L. van der Maaten and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in Proc. IEEE CVPR, 2017. CVF
-
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. L. Yuille, “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834–848, 2018. doi · arXiv
-
W. Luo, Y. Li, R. Urtasun and R. Zemel, “Understanding the Effective Receptive Field in Deep Convolutional Neural Networks,” in Advances in Neural Information Processing Systems (NeurIPS), 2016. NeurIPS · arXiv
-
J. Kim, J. K. Lee and K. M. Lee, “Accurate Image Super-Resolution Using Very Deep Convolutional Networks,” in Proc. IEEE CVPR, 2016. CVF · arXiv
-
H. Guo, J. Li, T. Dai, Z. Ouyang, X. Ren and S.-T. Xia, “MambaIR: A Simple Baseline for Image Restoration with State-Space Model,” in Proc. ECCV 2024, LNCS, Springer, 2025. doi · arXiv
-
ITU-T, “Recommendation T.81: Information technology – Digital compression and coding of continuous-tone still images – Requirements and guidelines” (the JPEG standard), 1992. ITU
-
T. Elsken, J. H. Metzen and F. Hutter, “Neural Architecture Search: A Survey,” Journal of Machine Learning Research, vol. 20, 2019. JMLR · arXiv
-
A. Khan, A. Sohail, U. Zahoora and A. S. Qureshi, “A Survey of the Recent Architectures of Deep Convolutional Neural Networks,” Artificial Intelligence Review, vol. 53, no. 8, pp. 5455–5516, 2020. doi · arXiv
-
S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan and M. Shah, “Transformers in Vision: A Survey,” ACM Computing Surveys, vol. 54, no. 10s, 2022. doi · arXiv
-
S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon and S. Xie, “ConvNeXt V2: Co-Designing and Scaling ConvNets With Masked Autoencoders,” in Proc. IEEE/CVF CVPR, 2023. CVF · arXiv
-
M. Goldblum, H. Souri, R. Ni, M. Shu, V. Prabhu, G. Somepalli, P. Chattopadhyay, M. Ibrahim et al., “Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. arXiv
-
X. Chen, X. Wang, J. Zhou, Y. Qiao and C. Dong, “Activating More Pixels in Image Super-Resolution Transformer,” in Proc. IEEE/CVF CVPR, 2023. CVF · arXiv
-
H. Guo, Y. Guo, Y. Zha, Y. Zhang, W. Li, T. Dai, S.-T. Xia and Y. Li, “MambaIRv2: Attentive State Space Restoration,” in Proc. IEEE/CVF CVPR, 2025. CVF · arXiv
-
A. Araujo, W. Norris and J. Sim, “Computing Receptive Fields of Convolutional Neural Networks,” Distill, vol. 4, no. 11, 2019. Distill · doi
-
X. Ding, X. Zhang, N. Ma, J. Han, G. Ding and J. Sun, “RepVGG: Making VGG-style ConvNets Great Again,” in Proc. IEEE/CVF CVPR, pp. 13733–13742, 2021. CVF paper
-
S. Zagoruyko and N. Komodakis, “DiracNets: Training Very Deep Neural Networks Without Skip-Connections,” arXiv:1706.00388, 2017. arXiv
-
X. Ding, Y. Guo, G. Ding and J. Han, “ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks,” in Proc. IEEE/CVF ICCV, 2019. CVF
-
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia, “Pyramid Scene Parsing Network,” in Proc. IEEE CVPR, 2017. CVF
-
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth and B. Schiele, “The Cityscapes Dataset for Semantic Urban Scene Understanding,” in Proc. IEEE CVPR, 2016. CVF
-
E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan and Q. V. Le, “AutoAugment: Learning Augmentation Strategies From Data,” in Proc. IEEE/CVF CVPR, 2019. CVF
-
H. Zhang, M. Cisse, Y. N. Dauphin and D. Lopez-Paz, “mixup: Beyond Empirical Risk Minimization,” in Proc. ICLR, 2018. arXiv