Full search page →

InvSR: a learned image-conditioned start (CVPR 2025)

Optional · paper guide 6 of 8 for DLCV 09A.2 · One-Step Diffusion Super-Resolution. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 2

  1. Why a latent start is not a noising equation
  2. Score regularization and per-image measurement agreement

Paper guides · 8

  1. ResShift: a short restoration-conditioned chain (NeurIPS 2023)
  2. SinSR: distilling a deterministic image-conditioned map (CVPR 2024)
  3. OSEDiff: LR-latent start and variational score regularization (NeurIPS 2024)
  4. S3Diff: degradation-guided low-rank adaptation
  5. TSD-SR: target-aware score regularization (CVPR 2025)
  6. InvSR: a learned image-conditioned start (CVPR 2025)
  7. PiSA-SR: separate pixel and semantic adaptation (CVPR 2025)
  8. AddSR: prediction-refined adversarial restoration distillation

Deep Learning for Computer Vision

  1. 02 · Deep Image Classification
View the whole course tree →

Challenge

A pretrained reverse diffusion process may already contain a useful image prior, but a real LR observation does not supply the noisy state that will lead to its unknown HR original. InvSR learns that starting-state construction instead of adapting only the reverse denoiser [7]. Its central question is where to begin a pretrained reverse process so that a small selected number of steps produces a useful reconstruction.

Optimizing a separate inversion for every test image would be costly. Predicting an entire set of noise maps for all reverse steps would also be difficult. InvSR simplifies the problem by predicting only the starting perturbation at selected intermediate timesteps.

Method

A learned noise predictor creates an image-conditioned starting state at the selected diffusion time; later denoising can use different step counts.
Original paper figure 2 — A learned noise predictor creates an image-conditioned starting state at the selected diffusion time; later denoising can use different step counts.

Figure source: [7], arXiv:2412.09013v2, PDF page 5. Original PDF.

Partial prediction. Start the reverse process at an intermediate noise level with relatively high signal-to-noise ratio. The LR image already approximates much of the HR structure, so there is no need to begin at the original pure-noise endpoint. Fast-sampler skipping then reduces the number of visited states. The paper considers one through five reverse steps in its main experiments.

Learned starting state. A timestep-conditioned predictor fwf_w receives the enlarged LR image or its latent representation and supplies a perturbation. The state is

xτ=αˉτ y+1−αˉτ fw(y,τ).x_{\tau}=\sqrt{\bar\alpha_{\tau}}\,y+ \sqrt{1-\bar\alpha_{\tau}}\,f_w(y,\tau).

Here yy is the LR-derived representation, τ\tau a selected starting time, and αˉτ\bar\alpha_{\tau} the pretrained diffusion schedule coefficient. The predictor parameterization can generate a perturbation through predicted mean and variance. Importantly, the starting perturbation is allowed to be correlated with the input: it compensates for using yy instead of the unknown HR representation. It is not assumed to be independent, zero-mean standard Gaussian noise simply because the paper calls it a noise map.

Frozen reverse prior. Run the pretrained image-generation reverse process from this predicted state. The selected model is an ADD-derived fast prior [14], and the paper compares it with a conventional pretrained latent-diffusion alternative [29]. Training the start predictor can backpropagate through the fixed reverse model without updating all its weights. A frozen network still incurs computation during both this training path and deployment.

Intermediate randomness. The method does not predict a separate perturbation for every visited state. At intermediate states it uses the pretrained process and the prescribed sampling noise. An additional intermediate predictor is tested and yields little improvement in the chosen high-SNR regime. This empirical simplification is conditional on that regime; it is not a general statement that all diffusion inversions need only one predicted state.

Training losses. Evaluate the reconstructed output after the selected reverse process using L2, learned perceptual and adversarial terms. The implementation computes losses in the latent domain and adapts its perceptual comparator for that domain. The losses train the starting-state predictor to select useful inputs for the fixed prior. There is no HR image or per-image optimization required at deployment.

Budget selection. Train the predictor for several starting timesteps through a time embedding. During inference, choose a start and a skipping sequence. The starting time and number of visited reverse states are separate controls. A later start can retain more observation structure; additional updates can add detail or, on noisy inputs, amplify artifacts. The model’s name “arbitrary steps” should be read within the trained/validated time range, not as evidence for every possible off-grid schedule.

Results

On ImageNet-Test, the main one-step row reports PSNR 24.14, LPIPS 0.2517, NIQE 4.3815 and MUSIQ 72.2900. On RealSR it reports 24.50, 0.2872, 4.2189 and 67.4586. These use the paper’s specific starting time and test preprocessing. Its parameter table lists a small learned component relative to a large pretrained prior; do not report the predictor’s parameter count as the complete model footprint.

The flexibility study is especially informative. At one reverse step, starting at timestep 250 gives PSNR 23.84 and MUSIQ 72.5773; starting at 100 gives 24.66 and 70.8251. The choice changes fidelity and appearance even when the step count is unchanged. A step count alone therefore does not fully identify the operating point.

Ablations

The prior-backbone comparison at one step and starting time 200 gives LPIPS 0.2517 with the fast prior and 0.2647 with the conventional alternative. This supports the selected backbone in that experiment, not a general claim that distillation always improves all inversion methods.

Adding intermediate noise prediction changes the five-step LPIPS from 0.2844 to 0.2844 at displayed precision, and MUSIQ from 69.8427 to 69.8466. The extra module has negligible measured benefit there. For loss terms, L2-only training gives PSNR 26.71 and LPIPS 0.2850; the combined perceptual/adversarial configuration gives 24.14 and 0.2517. Better perceptual reconstruction comes with lower pixel fidelity. The paper also shows cases where more steps amplify input noise, so budget selection depends on the damage.

Conclusion

Learning a useful input state can make a fixed generative reverse process an effective short-budget restorer. Starting time, predictor and sampler are all parts of the system, with different effects on fidelity and appearance.

Problems

The predictor can construct an unsuitable state for unfamiliar damage. The pretrained prior and decoder remain large. One reverse step still includes the starting-state predictor and representation processing, so it is not one total network call. Flexible schedules can change content or amplify artifacts. A plausible inverted trajectory is not evidence that the recovered HR texture belonged to the original photograph.

References

  1. R. Wu, L. Sun, Z. Ma, L. Zhang, “One-Step Effective Diffusion Network for Real-World Image Super-Resolution,” NeurIPS, 2024. source
  2. Z. Yue, J. Wang, C. C. Loy, “ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting,” NeurIPS, 2023. source
  3. Y. Wang, W. Yang, X. Chen, et al., “SinSR: Diffusion-Based Image Super-Resolution in a Single Step,” CVPR, 2024. source · supplementary ablations
  4. A. Zhang, Z. Yue, R. Pei, W. Ren, X. Cao, “Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors,” arXiv preprint, 2024. paper
  5. Y. Tai, R. Xie, C. Zhao, K. Zhang, Z. Zhang, J. Zhou, J. Yang, “AddSR: Accelerating diffusion-based blind super-resolution with adversarial diffusion distillation,” Pattern Recognition, 2026. published version · manuscript supplying the discussed tables
  6. L. Dong, Q. Fan, Y. Guo, Z. Wang, Q. Zhang, J. Chen, Y. Luo, C. Zou, “TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution,” CVPR, 2025. paper
  7. Z. Yue, K. Liao, C. C. Loy, “Arbitrary-steps Image Super-resolution via Diffusion Inversion,” CVPR, 2025. paper
  8. L. Sun, R. Wu, Z. Ma, S. Liu, Q. Yi, L. Zhang, “Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach,” CVPR, 2025. paper
  9. A. Liu, Y. Liu, J. Gu, Y. Qiao, C. Dong, “Blind Image Super-Resolution: A Survey and Beyond,” IEEE TPAMI (early access), 2022. source
  10. B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
  11. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” ICLR, 2022. source
  12. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  13. Z. Wang, C. Lu, Y. Wang, F. Bao, C. Li, H. Su, J. Zhu, “ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation,” NeurIPS, 2023. paper
  14. A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
  15. J. Cai, H. Zeng, H. Yong, Z. Cao, L. Zhang, “Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. source
  16. P. Wei, Z. Xie, H. Lu, Z. Zhan, Q. Ye, W. Zuo, L. Lin, “Component Divide-and-Conquer for Real-World Image Super-Resolution,” ECCV, 2020. source
  17. E. Agustsson and R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” in Proc. IEEE CVPR Workshops, 2017, pp. 1122–1131. doi · CVF
  18. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  19. K. Ding, K. Ma, S. Wang, E. P. Simoncelli, “Image Quality Assessment: Unifying Structure and Texture Similarity,” IEEE TPAMI, 2022. source
  20. A. Mittal, R. Soundararajan and A. C. Bovik, “Making a “Completely Blind” Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
  21. J. Ke, Q. Wang, Y. Wang, P. Milanfar, F. Yang, “MUSIQ: Multi-scale Image Quality Transformer,” ICCV, 2021. source
  22. S. Yang, T. Wu, S. Shi, S. Lao, Y. Gong, M. Cao, J. Wang, Y. Yang, “MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment,” CVPR Workshops, 2022. source
  23. J. Wang, K. C. K. Chan, C. C. Loy, “Exploring CLIP for Assessing the Look and Feel of Images,” AAAI, 2023. source
  24. X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
  25. Z. Wu, Y. Dong, X. Wei, “TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution,” CVPR, 2026. paper
  26. P. Esser, R. Rombach, B. Ommer, “Taming Transformers for High-Resolution Image Synthesis,” CVPR, 2021. paper
  27. X. Wang, L. Xie, C. Dong, Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution With Pure Synthetic Data,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. source
  28. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
  29. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
  30. R. Wu, T. Yang, L. Sun, Z. Zhang, S. Li, L. Zhang, “SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. source · supplementary experiments
  31. H. Liu, C. Li, Q. Wu, Y. J. Lee, “Visual Instruction Tuning,” NeurIPS, 2023. source
  32. Y. Li, K. Zhang, J. Liang, et al., “LSDIR: A Large Scale Dataset for Image Restoration,” CVPR Workshops, 2023. paper
  33. T. Karras, S. Laine, T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” CVPR, 2019. paper
  34. I. Loshchilov, F. Hutter, “Decoupled Weight Decay Regularization,” ICLR, 2019. paper
  35. P. Esser, S. Kulal, A. Blattmann, et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML, 2024. paper
  36. J. Ho, T. Salimans, “Classifier-Free Diffusion Guidance,” arXiv preprint, 2022. paper
  37. R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang et al., “NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results,” in Proc. IEEE CVPR Workshops, 2017, pp. 1110–1121. doi · CVF