Full search page →

S3Diff: degradation-guided low-rank adaptation

Optional · paper guide 4 of 8 for DLCV 09A.2 · One-Step Diffusion Super-Resolution. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 2

  1. Why a latent start is not a noising equation
  2. Score regularization and per-image measurement agreement

Paper guides · 8

  1. ResShift: a short restoration-conditioned chain (NeurIPS 2023)
  2. SinSR: distilling a deterministic image-conditioned map (CVPR 2024)
  3. OSEDiff: LR-latent start and variational score regularization (NeurIPS 2024)
  4. S3Diff: degradation-guided low-rank adaptation
  5. TSD-SR: target-aware score regularization (CVPR 2025)
  6. InvSR: a learned image-conditioned start (CVPR 2025)
  7. PiSA-SR: separate pixel and semantic adaptation (CVPR 2025)
  8. AddSR: prediction-refined adversarial restoration distillation

Deep Learning for Computer Vision

  1. 02 · Deep Image Classification
View the whole course tree →

Challenge

A pretrained one-step image generator is not automatically a reliable SR model. Blur and noise vary between photographs, and a fixed adaptation may under-correct one input while over-correcting another. S3Diff adapts the generator according to estimated damage, while avoiding an expensive semantic captioner for each input [4].

The design asks two separate questions: how should the network’s effective weights depend on degradation, and how should it learn positive/negative quality prompting? The first changes image-conditioned adaptation; the second teaches quality steering. Neither guarantees that an estimated blur/noise vector fully represents a real camera pipeline.

Method

Degradation estimates modulate low-rank adaptation, while positive and negative prompt paths support guidance; distinguish the training branches from the deployed one-step path.
Original paper figure 3 — Degradation estimates modulate low-rank adaptation, while positive and negative prompt paths support guidance; distinguish the training branches from the deployed one-step path.

Figure source: [4], arXiv:2409.17058v1, PDF page 5. Original PDF.

Starting prior and image path. The method uses an ADD-derived one-step latent image prior [14]. It feeds the LR-derived image representation with little or no added noise, adapts the VAE encoder and denoiser, and keeps the decoder fixed. The encoder can suppress input defects before the latent transformation. Freezing the decoder preserves the pretrained output representation, with the same general motivation tested by other latent-prior restorers.

Low-rank correction. For W∈Rm×nW\in\mathbb R^{m\times n}, write an adapted weight as W′=W+ABW'=W+AB, where A∈Rm×rA\in\mathbb R^{m\times r}, B∈Rr×nB\in\mathbb R^{r\times n} and rr is the rank. S3Diff inserts a learned correction C(d)∈Rr×rC(d)\in\mathbb R^{r\times r} between them: W′=W+AC(d)BW'=W+AC(d)B. This changes the effective low-rank directions according to the current degradation descriptor dd.

Degradation and block identity. A pretrained estimator supplies two normalized values representing blur and noise. A sinusoidal/Fourier embedding and learned mapping process that descriptor. Block-ID embeddings let the shared mapping generate different Ci(d)C_i(d) for different network blocks. A single shared matrix can be too restrictive, while an entirely separate large predictor for every block increases adaptation cost. The block embedding provides an intermediate design.

Dynamic computation. Unlike one globally merged static adapter, the effective correction can vary per input. The matrix factors still restrict its structure and parameter count, but estimation and modulation remain part of the executed system. If a deployment exposes manual degradation values, those are user-supplied estimates, not verified physical measurements.

Online negative targets. During training, a small fraction of targets is replaced by their degraded counterparts and associated with a negative-quality prompt. Positive targets use a generic positive-quality prompt. This teaches the model what negative appearance means using the same damage distribution, instead of relying only on offline generative negative examples. The reported negative-target probability is 0.05. Reconstruction/perceptual supervision applies to the selected target, while adversarial supervision applies to positive HR targets.

Losses and guidance. The generator combines L2, LPIPS and GAN terms, with reported weights 2, 5 and 0.5. The discriminator uses pretrained image features and trainable heads. At inference, positive/negative guidance can combine two conditioned predictions. This is still one state-update step, but the guided configuration must count both denoiser predictions. The source uses a small guidance strength because excessive guidance harms fidelity.

Training setup. The linked manuscript trains on LSDIR and a 10,000-image face subset from FFHQ [32], [33], with Real-ESRGAN-style synthetic damage [27]. Its selected adapter ranks are 16 for the encoder and 32 for the denoiser. Those ranks and the guidance setting belong to the reported version; they are not universal requirements for every low-rank SR model.

Results

In the opened manuscript’s synthetic DIV2K validation comparison, S3Diff gives PSNR 23.40, LPIPS 0.2571, DISTS 0.1730, FID 19.35 and MUSIQ 68.21. On RealSR it gives PSNR 25.03, LPIPS 0.2699 and FID 108.88. These figures describe that paper’s rerun of the compared checkpoints and processing protocol. Its OSEDiff rows differ from OSEDiff’s original tables; use a controlled rerun or keep the source table attached instead of merging them into one canonical score.

The reported improvement concerns the collection of measures and images, not every metric on every dataset. Some stronger-reference baselines have higher PSNR, while different no-reference metrics favor different appearance choices.

Ablations

For degradation-guided LoRA, using a shared correction matrix gives DIV2K LPIPS 0.2613 and MUSIQ 65.52; block-ID-conditioned unshared corrections give 0.2563 and 66.55. On RealSR, the same comparison changes LPIPS from 0.2626 to 0.2573 and MUSIQ from 64.23 to 65.78.

The negative-prompting study on RealSR gives PSNR 25.76 and MUSIQ 66.42 with online negative targets but no guidance; with guidance these become 25.03 and 67.88. Increasing preferred appearance therefore costs reference fidelity in that setting. Encoder/decoder adaptation and rank sweeps provide additional evidence that more trainable capacity or stronger guidance is not automatically better. The dynamic architecture and the guidance must be evaluated together with their cost.

Conclusion

Estimated degradation can modulate a low-rank generative restorer, and online negative targets can teach quality steering without a semantic captioner. These are distinct controls over adaptation and appearance.

Problems

The estimator can misread damage, and blur/noise descriptors omit many processing effects. Dynamic corrections and guided predictions add deployment work. Generic prompts avoid some semantic-extractor cost but do not solve missing-detail ambiguity. The pretrained prior can still invent texture, and parameter-efficient adaptation does not remove the large backbone. Compare the exact checkpoint, guidance and evaluation protocol, since experimental settings can differ between versions.

References

  1. R. Wu, L. Sun, Z. Ma, L. Zhang, “One-Step Effective Diffusion Network for Real-World Image Super-Resolution,” NeurIPS, 2024. source
  2. Z. Yue, J. Wang, C. C. Loy, “ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting,” NeurIPS, 2023. source
  3. Y. Wang, W. Yang, X. Chen, et al., “SinSR: Diffusion-Based Image Super-Resolution in a Single Step,” CVPR, 2024. source · supplementary ablations
  4. A. Zhang, Z. Yue, R. Pei, W. Ren, X. Cao, “Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors,” arXiv preprint, 2024. paper
  5. Y. Tai, R. Xie, C. Zhao, K. Zhang, Z. Zhang, J. Zhou, J. Yang, “AddSR: Accelerating diffusion-based blind super-resolution with adversarial diffusion distillation,” Pattern Recognition, 2026. published version · manuscript supplying the discussed tables
  6. L. Dong, Q. Fan, Y. Guo, Z. Wang, Q. Zhang, J. Chen, Y. Luo, C. Zou, “TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution,” CVPR, 2025. paper
  7. Z. Yue, K. Liao, C. C. Loy, “Arbitrary-steps Image Super-resolution via Diffusion Inversion,” CVPR, 2025. paper
  8. L. Sun, R. Wu, Z. Ma, S. Liu, Q. Yi, L. Zhang, “Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach,” CVPR, 2025. paper
  9. A. Liu, Y. Liu, J. Gu, Y. Qiao, C. Dong, “Blind Image Super-Resolution: A Survey and Beyond,” IEEE TPAMI (early access), 2022. source
  10. B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
  11. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” ICLR, 2022. source
  12. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  13. Z. Wang, C. Lu, Y. Wang, F. Bao, C. Li, H. Su, J. Zhu, “ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation,” NeurIPS, 2023. paper
  14. A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
  15. J. Cai, H. Zeng, H. Yong, Z. Cao, L. Zhang, “Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. source
  16. P. Wei, Z. Xie, H. Lu, Z. Zhan, Q. Ye, W. Zuo, L. Lin, “Component Divide-and-Conquer for Real-World Image Super-Resolution,” ECCV, 2020. source
  17. E. Agustsson and R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” in Proc. IEEE CVPR Workshops, 2017, pp. 1122–1131. doi · CVF
  18. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  19. K. Ding, K. Ma, S. Wang, E. P. Simoncelli, “Image Quality Assessment: Unifying Structure and Texture Similarity,” IEEE TPAMI, 2022. source
  20. A. Mittal, R. Soundararajan and A. C. Bovik, “Making a “Completely Blind” Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
  21. J. Ke, Q. Wang, Y. Wang, P. Milanfar, F. Yang, “MUSIQ: Multi-scale Image Quality Transformer,” ICCV, 2021. source
  22. S. Yang, T. Wu, S. Shi, S. Lao, Y. Gong, M. Cao, J. Wang, Y. Yang, “MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment,” CVPR Workshops, 2022. source
  23. J. Wang, K. C. K. Chan, C. C. Loy, “Exploring CLIP for Assessing the Look and Feel of Images,” AAAI, 2023. source
  24. X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
  25. Z. Wu, Y. Dong, X. Wei, “TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution,” CVPR, 2026. paper
  26. P. Esser, R. Rombach, B. Ommer, “Taming Transformers for High-Resolution Image Synthesis,” CVPR, 2021. paper
  27. X. Wang, L. Xie, C. Dong, Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution With Pure Synthetic Data,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. source
  28. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
  29. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
  30. R. Wu, T. Yang, L. Sun, Z. Zhang, S. Li, L. Zhang, “SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. source · supplementary experiments
  31. H. Liu, C. Li, Q. Wu, Y. J. Lee, “Visual Instruction Tuning,” NeurIPS, 2023. source
  32. Y. Li, K. Zhang, J. Liang, et al., “LSDIR: A Large Scale Dataset for Image Restoration,” CVPR Workshops, 2023. paper
  33. T. Karras, S. Laine, T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” CVPR, 2019. paper
  34. I. Loshchilov, F. Hutter, “Decoupled Weight Decay Regularization,” ICLR, 2019. paper
  35. P. Esser, S. Kulal, A. Blattmann, et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML, 2024. paper
  36. J. Ho, T. Salimans, “Classifier-Free Diffusion Guidance,” arXiv preprint, 2022. paper
  37. R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang et al., “NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results,” in Proc. IEEE CVPR Workshops, 2017, pp. 1110–1121. doi · CVF