Full search page →

LIIF’s three refinements

Optional · supplementary note 6 of 6 for DLCV 09A · Non-Blind Image Super-Resolution. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 6

  1. Example-based SR and sparse coding in detail
  2. Aliasing in the LR spectrum
  3. Adjustable gradient clipping
  4. Why a posterior sample costs twice the MMSE
  5. ESRGAN’s changes in detail
  6. LIIF’s three refinements

Deep Learning for Computer Vision

  1. 02 · Deep Image Classification
View the whole course tree →

This note explains three small additions that make LIIF’s feature lookup richer, smooth and aware of the output pixel size [38].

In the LIIF formula of the main text, feature unfolding concatenates the 3×33 \times 3 neighbourhood of each latent code into one vector, so each code describes a small region. Local ensemble evaluates the MLP with the four latent codes around xqx_q and blends the four predictions with area weights, so the output does not jump when the nearest code z∗z^{*} changes. Cell decoding also gives the MLP the height and width of the output pixel being rendered, so it knows whether it is predicting a fine or a coarse sample.

References

  1. S. C. Park, M. K. Park and M. G. Kang, “Super-Resolution Image Reconstruction: A Technical Overview,” IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21–36, 2003. doi
  2. Y. Blau and T. Michaeli, “The Perception-Distortion Tradeoff,” in Proc. IEEE/CVF CVPR, 2018, pp. 6228–6237. doi · arXiv
  3. MathWorks, “imresize: Resize image,” MATLAB documentation (default method bicubic; antialiasing on when shrinking). link
  4. Z. Wang, J. Chen and S. C. H. Hoi, “Deep Learning for Image Super-Resolution: A Survey,” IEEE TPAMI, vol. 43, no. 10, pp. 3365–3387, 2021. doi · arXiv
  5. R. Keys, “Cubic Convolution Interpolation for Digital Image Processing,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1153–1160, 1981. doi
  6. C. E. Duchon, “Lanczos Filtering in One and Two Dimensions,” Journal of Applied Meteorology, vol. 18, no. 8, pp. 1016–1022, 1979. doi
  7. W. T. Freeman, T. R. Jones and E. C. Pasztor, “Example-Based Super-Resolution,” IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56–65, 2002. doi
  8. D. Glasner, S. Bagon and M. Irani, “Super-Resolution from a Single Image,” in Proc. IEEE ICCV, 2009, pp. 349–356. doi
  9. J. Yang, J. Wright, T. S. Huang and Y. Ma, “Image Super-Resolution via Sparse Representation,” IEEE Transactions on Image Processing, vol. 19, no. 11, pp. 2861–2873, 2010. doi
  10. R. Timofte, V. De Smet and L. Van Gool, “Anchored Neighborhood Regression for Fast Example-Based Super-Resolution,” in Proc. IEEE ICCV, 2013, pp. 1920–1927. doi · CVF
  11. R. Timofte, V. De Smet and L. Van Gool, “A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution,” in Computer Vision – ACCV 2014, Lecture Notes in Computer Science, Springer, 2015, pp. 111–126. doi
  12. Y. Romano, J. Isidoro and P. Milanfar, “RAISR: Rapid and Accurate Image Super Resolution,” IEEE Transactions on Computational Imaging, vol. 3, no. 1, pp. 110–125, 2017. doi · arXiv
  13. N. Efrat, D. Glasner, A. Apartsin, B. Nadler and A. Levin, “Accurate Blur Models vs. Image Priors in Single Image Super-Resolution,” in Proc. IEEE ICCV, 2013, pp. 2832–2839. doi · CVF
  14. C. Dong, C. C. Loy, K. He and X. Tang, “Image Super-Resolution Using Deep Convolutional Networks,” IEEE TPAMI, vol. 38, no. 2, pp. 295–307, 2016. doi · arXiv
  15. C. Dong, C. C. Loy and X. Tang, “Accelerating the Super-Resolution Convolutional Neural Network,” in Proc. ECCV, 2016, pp. 391–407. doi · arXiv
  16. W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert and Z. Wang, “Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network,” in Proc. IEEE CVPR, 2016, pp. 1874–1883. doi · arXiv
  17. J. Kim, J. K. Lee and K. M. Lee, “Accurate Image Super-Resolution Using Very Deep Convolutional Networks,” in Proc. IEEE CVPR, 2016, pp. 1646–1654. doi · arXiv
  18. B. Lim, S. Son, H. Kim, S. Nah and K. M. Lee, “Enhanced Deep Residual Networks for Single Image Super-Resolution,” in Proc. IEEE CVPR Workshops, 2017, pp. 1132–1140. doi · arXiv
  19. Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong and Y. Fu, “Image Super-Resolution Using Very Deep Residual Channel Attention Networks,” in Proc. ECCV, 2018, pp. 294–310. doi · arXiv
  20. J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool and R. Timofte, “SwinIR: Image Restoration Using Swin Transformer,” in Proc. IEEE/CVF ICCV Workshops, 2021, pp. 1833–1844. doi · arXiv
  21. X. Chen, X. Wang, J. Zhou, Y. Qiao and C. Dong, “Activating More Pixels in Image Super-Resolution Transformer,” in Proc. IEEE/CVF CVPR, 2023, pp. 22367–22377. doi · arXiv
  22. PyTorch contributors, “PixelShuffle.cpp” (implementation of torch.nn.PixelShuffle and torch.nn.PixelUnshuffle), PyTorch source code. GitHub
  23. A. Aitken, C. Ledig, L. Theis, J. Caballero, Z. Wang and W. Shi, “Checkerboard Artifact Free Sub-Pixel Convolution: A Note on Sub-Pixel Convolution, Resize Convolution and Convolution Resize,” arXiv:1707.02937, 2017. arXiv
  24. C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang and W. Shi, “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” in Proc. IEEE CVPR, 2017, pp. 105–114. doi · arXiv
  25. R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang et al., “NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results,” in Proc. IEEE CVPR Workshops, 2017, pp. 1110–1121. doi · CVF
  26. X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao and C. C. Loy, “ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks,” in Proc. ECCV Workshops, 2018 (Lecture Notes in Computer Science, 2019), pp. 63–79. doi · arXiv
  27. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin and B. Guo, “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” in Proc. IEEE/CVF ICCV, 2021, pp. 9992–10002. doi · arXiv
  28. Z. Chen, Y. Zhang, J. Gu, L. Kong, X. Yang and F. Yu, “Dual Aggregation Transformer for Image Super-Resolution,” in Proc. IEEE/CVF ICCV, 2023, pp. 12278–12287. doi · arXiv
  29. H. Guo, J. Li, T. Dai, Z. Ouyang, X. Ren and S.-T. Xia, “MambaIR: A Simple Baseline for Image Restoration with State-Space Model,” in Proc. ECCV, 2024, pp. 222–241. doi · arXiv
  30. K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in Proc. ICLR, 2015. arXiv
  31. J. Johnson, A. Alahi and L. Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution,” in Proc. ECCV, 2016, pp. 694–711. doi · arXiv
  32. Y. Blau, R. Mechrez, R. Timofte, T. Michaeli and L. Zelnik-Manor, “The 2018 PIRM Challenge on Perceptual Image Super-Resolution,” in Proc. ECCV Workshops, 2018. arXiv
  33. A. Mittal, R. Soundararajan and A. C. Bovik, “Making a “Completely Blind” Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
  34. C. Ma, C.-Y. Yang, X. Yang and M.-H. Yang, “Learning a No-Reference Quality Metric for Single-Image Super-Resolution,” Computer Vision and Image Understanding, vol. 158, pp. 1–16, 2017. doi · arXiv
  35. C. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet and M. Norouzi, “Image Super-Resolution via Iterative Refinement,” IEEE TPAMI, 2022. doi · arXiv
  36. O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Proc. MICCAI, 2015, pp. 234–241. doi · arXiv
  37. X. Hu, H. Mu, X. Zhang, Z. Wang, T. Tan and J. Sun, “Meta-SR: A Magnification-Arbitrary Network for Super-Resolution,” in Proc. IEEE/CVF CVPR, 2019, pp. 1575–1584. doi · arXiv
  38. Y. Chen, S. Liu and X. Wang, “Learning Continuous Image Representation with Local Implicit Image Function,” in Proc. IEEE/CVF CVPR, 2021, pp. 8624–8634. doi · arXiv
  39. Z. Hui, X. Gao, Y. Yang and X. Wang, “Lightweight Image Super-Resolution with Information Multi-distillation Network,” in Proc. ACM International Conference on Multimedia, 2019, pp. 2024–2032. doi · arXiv
  40. B. Ren, Y. Li, N. Mehta, R. Timofte et al., “The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report,” in Proc. IEEE/CVF CVPR Workshops, 2024. CVF · arXiv
  41. B. Ren, H. Guo, L. Sun, Z. Wu, R. Timofte, Y. Li et al., “The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report,” in Proc. IEEE/CVF CVPR Workshops, 2025. CVF · arXiv
  42. ITU-R, “Recommendation BT.601: Studio Encoding Parameters of Digital Television for Standard 4:3 and Wide Screen 16:9 Aspect Ratios.” ITU
  43. MathWorks, “rgb2ycbcr: Convert RGB color values to YCbCr color space,” Image Processing Toolbox documentation. link
  44. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  45. Z. Chen, Z. Wu, E. Zamfir, K. Zhang, Y. Zhang, R. Timofte et al., “NTIRE 2024 Challenge on Image Super-Resolution (×4): Methods and Results,” in Proc. IEEE/CVF CVPR Workshops, 2024, pp. 6108–6132. doi · arXiv
  46. Z. Chen, K. Liu, J. Gong, J. Wang, L. Sun, Z. Wu, R. Timofte, Y. Zhang et al., “NTIRE 2025 Challenge on Image Super-Resolution (×4): Methods and Results,” in Proc. IEEE/CVF CVPR Workshops, 2025, pp. 1525–1535. doi · arXiv
  47. M. Bevilacqua, A. Roumy, C. Guillemot and M.-L. Alberi Morel, “Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding,” in Proc. BMVC, 2012, pp. 135.1–135.10. doi
  48. R. Zeyde, M. Elad and M. Protter, “On Single Image Scale-Up Using Sparse-Representations,” in Curves and Surfaces, Lecture Notes in Computer Science, Springer, 2012, pp. 711–730. doi
  49. D. Martin, C. Fowlkes, D. Tal and J. Malik, “A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics,” in Proc. IEEE ICCV, 2001, vol. 2, pp. 416–423. doi
  50. J.-B. Huang, A. Singh and N. Ahuja, “Single Image Super-Resolution from Transformed Self-Exemplars,” in Proc. IEEE CVPR, 2015, pp. 5197–5206. doi · CVF
  51. A. Fujimoto, T. Ogawa, K. Yamamoto, Y. Matsui, T. Yamasaki and K. Aizawa, “Manga109 Dataset and Creation of Metadata,” in Proc. 1st International Workshop on coMics ANalysis, Processing and Understanding (MANPU), 2016, pp. 1–5. doi
  52. Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, T. Ogawa, T. Yamasaki and K. Aizawa, “Sketch-Based Manga Retrieval Using Manga109 Dataset,” Multimedia Tools and Applications, vol. 76, no. 20, pp. 21811–21838, 2017. doi
  53. E. Agustsson and R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” in Proc. IEEE CVPR Workshops, 2017, pp. 1122–1131. doi · CVF
  54. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  55. W. Ren, R. Goyal, Z. Hu, T. T. Aumentado-Armstrong, I. Mohomed and A. Levinshtein, “Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution,” arXiv:2507.14367, 2025. arXiv
  56. B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
  57. A. Shocher, N. Cohen and M. Irani, “‘Zero-Shot’ Super-Resolution Using Deep Internal Learning,” in Proc. IEEE/CVF CVPR, 2018, pp. 3118–3126. doi · arXiv