Full search page →

Improved MeanFlow — prediction inputs and flexible guidance

Optional · paper guide 4 of 6 for ML 08 · Diffusion Models and Flow Matching. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

Question and thesis

Can a one-step model retain MeanFlow’s interval prediction while making training and guidance more flexible? Geng et al. identify a subtle dependency in the original objective and develop improved MeanFlow, abbreviated iMF [49]. Read MeanFlow’s total derivative and stop-gradient first. The paper separates three changes: the regression formulation, inference-time guidance conditioning and an architecture for handling those conditions.

Method and figure

The original compound predictor also receives the sampled conditional velocity through its JVP. The improved predictor obtains that direction from the network, leaving the sample velocity as a regression target.
Source: Geng et al., CVPR 2026 proceedings. Figure 1, PDF p. 1. The original compound predictor also receives the sampled conditional velocity through its JVP. The improved predictor obtains that direction from the network, leaving the sample velocity as a regression target. [49]

Keep MeanFlow’s convention: zt=(1−t)x+tϵz_t=(1-t)x+t\epsilon, with data xx at zero and noise ϵ\epsilon at one. Write the improved compound prediction as

Vθ=uθ(zt,r,t)+(t−r)sg⁡[Jzuθ vθ(zt,t)+∂tuθ],L=E∥Vθ−(ϵ−x)∥2.V_\theta=u_\theta(z_t,r,t)+(t-r)\operatorname{sg}[J_zu_\theta\,v_\theta(z_t,t)+\partial_tu_\theta],\qquad L=\mathbb E\|V_\theta-(\epsilon-x)\|^2.

Here uθu_\theta predicts interval-average velocity between rr and tt, vθv_\theta predicts instantaneous velocity, JzJ_z is the state Jacobian, and sg⁡\operatorname{sg} blocks gradients through the derivative result. VθV_\theta combines these terms into an instantaneous-velocity prediction; the expectation averages sampled times, data and noise. One implementation reuses the boundary vθ(zt,t)=uθ(zt,t,t)v_\theta(z_t,t)=u_\theta(z_t,t,t); another adds an auxiliary velocity head used during training.

Figure 1 isolates the key distinction. Original MeanFlow’s equivalent compound predictor uses the sampled ϵ−x\epsilon-x inside its JVP as well as in the loss target. iMF supplies a predicted direction to the JVP, so the prediction depends on the observable noisy input and conditions. The regression label still uses ϵ−x\epsilon-x. Stop-gradient remains in the implemented prediction; this is not a claim that the loss became ordinary full differentiation through every derivative.

Evidence

Table 1 and Figure 6 start from MeanFlow-B/2, trained for 240 epochs on ImageNet 256×256 with 1-NFE evaluation. In the guided ablation, changing the objective with an auxiliary head moves FID from 6.17 to 5.68; guidance conditioning including an interval reaches 4.57; in-context conditioning reaches 4.09. The final 1.72 result belongs to a larger, further-developed system. Table 2 explicitly warns that model sizes and compute are not fully matched across the original and improved families.

Design analysis

Guidance scale and its active interval become network inputs rather than fixed training choices. Multiple condition tokens carry time, class and guidance information alongside image-latent tokens. This can preserve one-call inference while allowing trained guidance settings to vary. It differs from evaluating separate conditional and unconditional predictions at sampling time. The objective change and the conditioning change solve related but distinct problems; a single best score cannot isolate both.

Reading checkpoint

In the displayed objective, mark every place where the sampled clean image enters. Which term is a training label, and which inputs are available when generating? Then explain why changing guidance at inference requires a model trained to accept that condition, rather than simply relabeling an ordinary MeanFlow checkpoint.

Limits and next step

Training still uses derivative computation, approximation and stop-gradient; the improvement is not a universal convergence proof. ImageNet results still use a pretrained VAE representation, and generator NFE excludes decoder work. Continue to JiT and Pixel MeanFlow to separate the choice of interval objective from the choice of pixel-space network output.

Continue reading: JiT — clean-image prediction in pixel space

References

  1. L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui and M.-H. Yang, “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. arXiv
  2. Y. Lipman, M. Havasi, P. Holderrieth, N. Shaul, M. Le, B. Karrer, R. T. Q. Chen, D. Lopez-Paz, H. Ben-Hamu and I. Gat, “Flow Matching Guide and Code,” arXiv:2412.06264, 2024. arXiv
  3. J. Ho, A. Jain and P. Abbeel, “Denoising Diffusion Probabilistic Models,” NeurIPS, 2020. arXiv
  4. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR, 2021. arXiv
  5. J. Song, C. Meng and S. Ermon, “Denoising Diffusion Implicit Models,” ICLR, 2021. arXiv
  6. J. Ho and T. Salimans, “Classifier-Free Diffusion Guidance,” NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications; arXiv:2207.12598, 2022. arXiv
  7. R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. arXiv
  8. W. Peebles and S. Xie, “Scalable Diffusion Models with Transformers,” ICCV, 2023. arXiv
  9. P. Esser, S. Kulal, A. Blattmann et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML, PMLR 235, 2024. PMLR
  10. Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel and M. Le, “Flow Matching for Generative Modeling,” ICLR, 2023. arXiv
  11. X. Liu, C. Gong and Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” ICLR, 2023. arXiv
  12. Y. Song, P. Dhariwal, M. Chen and I. Sutskever, “Consistency Models,” ICML, 2023. arXiv
  13. Z. Geng, M. Deng, X. Bai, J. Z. Kolter and K. He, “Mean Flows for One-step Generative Modeling,” NeurIPS, 2025. arXiv
  14. S. J. D. Prince, Understanding Deep Learning, MIT Press, 2023. book site
  15. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan and S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics,” ICML, PMLR 37, 2015. PMLR
  16. P. Vincent, “A Connection Between Score Matching and Denoising Autoencoders,” Neural Computation, vol. 23, no. 7, pp. 1661–1674, 2011. doi
  17. Y. Song and S. Ermon, “Generative Modeling by Estimating Gradients of the Data Distribution,” NeurIPS, 2019. arXiv
  18. P. Dhariwal and A. Nichol, “Diffusion Models Beat GANs on Image Synthesis,” NeurIPS, 2021. NeurIPS
  19. A. Q. Nichol and P. Dhariwal, “Improved Denoising Diffusion Probabilistic Models,” ICML, PMLR 139, 2021. PMLR
  20. T. Karras, M. Aittala, T. Aila and S. Laine, “Elucidating the Design Space of Diffusion-Based Generative Models,” NeurIPS, 2022. arXiv
  21. C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li and J. Zhu, “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps,” NeurIPS, 2022. arXiv
  22. T. Salimans and J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” ICLR, 2022. arXiv
  23. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler and S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS (NIPS), 2017. arXiv
  24. T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen and T. Aila, “Improved Precision and Recall Metric for Assessing Generative Models,” NeurIPS, 2019. arXiv
  25. G. Stein, J. C. Cresswell, R. Hosseinzadeh et al., “Exposing Flaws of Generative Model Evaluation Metrics and Their Unfair Treatment of Diffusion Models,” NeurIPS, 2023. arXiv
  26. S. Jayasumana, S. Ramalingam, A. Veit, D. Glasner, A. Chakrabarti and S. Kumar, “Rethinking FID: Towards a Better Evaluation Metric for Image Generation,” CVPR, 2024. CVF
  27. F.-A. Croitoru, V. Hondru, R. T. Ionescu and M. Shah, “Diffusion Models in Vision: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. doi
  28. H. Cao, C. Tan, Z. Gao, Y. Xu, G. Chen, P.-A. Heng and S. Z. Li, “A Survey on Generative Diffusion Models,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 7, pp. 2814–2830, 2024. doi
  29. C.-H. Lai, Y. Song, D. Kim, Y. Mitsufuji and S. Ermon, “The Principles of Diffusion Models,” monograph, arXiv:2510.21890, 2025. arXiv
  30. H. Shen, J. Zhang, B. Xiong et al., “Efficient Diffusion Models: A Survey,” Transactions on Machine Learning Research, 2025. arXiv
  31. A. Melnik, M. Ljubljanac, C. Lu, Q. Yan, W. Ren and H. Ritter, “Video Diffusion Models: A Survey,” Transactions on Machine Learning Research, 2024. arXiv
  32. C. Lu and Y. Song, “Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models,” ICLR, 2025. arXiv
  33. A. Sauer, D. Lorenz, A. Blattmann and R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. arXiv
  34. M. S. Albergo and E. Vanden-Eijnden, “Building Normalizing Flows with Stochastic Interpolants,” ICLR, 2023. arXiv
  35. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” CVPR, 2009, pp. 248–255. doi
  36. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” CVPR, 2016, pp. 2818–2826. arXiv
  37. O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” MICCAI, LNCS 9351, pp. 234–241, 2015. arXiv
  38. A. Radford, J. W. Kim, C. Hallacy et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. arXiv
  39. M. Oquab, T. Darcet, T. Moutakanni et al., “DINOv2: Learning Robust Visual Features without Supervision,” Transactions on Machine Learning Research, 2024. arXiv
  40. B. Efron, “Tweedie’s Formula and Selection Bias,” Journal of the American Statistical Association, vol. 106, no. 496, pp. 1602–1614, 2011. doi
  41. A. Dosovitskiy, L. Beyer, A. Kolesnikov et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” ICLR, 2021. arXiv
  42. T. Li and K. He, “Back to Basics: Let Denoising Generative Models Denoise,” CVPR, 2026 (preprint 2025). paper, publication record, official JiT code.
  43. B. Zheng, N. Ma, S. Tong and S. Xie, “Diffusion Transformers with Representation Autoencoders,” ICLR, 2026 (preprint 2025). proceedings, manuscript.
  44. S. Tong, B. Zheng, Z. Wang, B. Tang, N. Ma, E. Brown, J. Yang, R. Fergus, Y. LeCun and S. Xie, “Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders,” CVPR Findings Track, 2026. manuscript, author publication record.
  45. D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna and R. Rombach, “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis,” ICLR, 2024. proceedings, official model card.
  46. Black Forest Labs, “FLUX.1 [dev],” official pretrained-model card, 2024. model card.
  47. W. Kong, Q. Tian, Z. Zhang et al., “HunyuanVideo: A Systematic Framework For Large Video Generative Models,” technical report, arXiv:2412.03603, 2024. report, official release.
  48. Team Wan, A. Wang, B. Ai, B. Wen et al., “Wan: Open and Advanced Large-Scale Video Generative Models,” technical report, arXiv:2503.20314, 2025. report, official Wan2.1 release.
  49. Z. Geng, Y. Lu, Z. Wu, E. Shechtman, J. Z. Kolter and K. He, “Improved Mean Flows: On the Challenges of Fastforward Generative Models,” CVPR, pp. 30467–30476, 2026. CVF
  50. Y. Lu, S. Lu, Q. Sun, H. Zhao, Z. Jiang, X. Wang, T. Li, Z. Geng and K. He, “One-step Latent-free Image Generation with Pixel Mean Flows,” ICML, PMLR 306, pp. 81265–81277, 2026. PMLR