完整搜尋頁面 →

Pixel MeanFlow:透過去雜訊影像場一步生成像素

選讀・ML 08・擴散模型與流匹配 的論文導讀(第 6/6 篇)。引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

問題與論點

MeanFlow 的區間學習,能否和 JiT 的影像輸出設計結合,做到一次呼叫的像素生成?Lu 等人藉由分開網路輸出空間及訓練損失空間,提出 pixel MeanFlow,簡稱 pMF [50]。先讀 MeanFlow、iMF 與 JiT。這不是讓一般 JiT checkpoint 只走一次 Euler;雙時間場及訓練目標都改變了。

方法與圖解

原圖沿模擬 ODE 軌跡,比較平均速度場與誘發的去雜訊影像場;中間場呈影像狀是一種建模假設,不是所有時間對的保證。
來源:Lu et al., ICML 2026, PMLR 306 proceedings; official PMLR GitHub-hosted PDF. 圖 1,PDF 第 1 頁。原圖沿模擬 ODE 軌跡,比較平均速度場與誘發的去雜訊影像場;中間場呈影像狀是一種建模假設,不是所有時間對的保證。 [50]

沿用 MeanFlow,t=0t=0 是資料,t=1t=1 是雜訊。對狀態 ztz_t,定義影像狀場及其網路參數化:

x⋆(zt,r,t)=zt−t u(zt,r,t),uθ(zt,r,t)=zt−xθ(zt,r,t)t.x^\star(z_t,r,t)=z_t-t\,u(z_t,r,t),\qquad u_\theta(z_t,r,t)=\frac{z_t-x_\theta(z_t,r,t)}{t}.

其中 rr、tt 為區間端點,uu 為底層平均速度場,x⋆x^\star 為誘發的去雜訊影像場,xθx_\theta 為網路直接預測。轉換使用 t>0t\gt0,靠近零時需實作上的數值保護。將導出的 uθu_\theta 放進 iMF 複合瞬時速度預測器,再對條件速度 ϵ−x\epsilon-x 訓練。其中 xx 為抽樣乾淨影像,ϵ\epsilon 為先驗雜訊。因此流程是影像輸出 → 平均速度 → 瞬時速度損失。

當 r=tr=t,理想影像狀場對應瞬時速度的去雜訊量;當 r=0r=0,它等於 ODE 終點。中間時間對則是由圖 1 支持的近似流形假設,不是所有輸出必在自然影像流形上的定理。使用完整區間時,取樣式簡化成直接由雜訊預測影像。

證據

表 2 在固定 token 序列長度下,比較影像輸出與平均速度輸出。256×256 ImageNet、B/16 模型、160 個 epoch、指定最佳化器及 MSE 目標下,FID 為 9.56 對 164.89;64×64、B/4 設定則是 3.80 對 3.82。這些匹配比較呈現高維預測問題。圖 3 另外研究最佳化與感知損失加入的效果,不能把所有進步都歸給場的轉換。

設計分析

直接輸出影像,使生成器訓練可加入影像空間感知損失。較強配方使用預訓練特徵提取器計算該損失,雖然生成時沒有潛在 tokenizer 或解碼器。因此「無潛在空間」不代表訓練涉及的所有元件都沒有預訓練。同樣地,省去解碼延遲與減少網路呼叫,是不同效率增益;實際時間仍取決於模型大小及硬體。

閱讀檢查點

把 r=0r=0、t=1t=1 代入 MeanFlow 更新及影像到速度的轉換,說明為何最終影像等於網路輸出。再解釋為何內部場 x⋆(zt,r,t)x^\star(z_t,r,t) 通常不等於與這筆帶雜訊輸入配對的乾淨訓練影像。

限制與下一步

證據來自所研究的類別條件影像設定,並未涵蓋任意編修、文字生成影像或影片。流形論證是近似的,感知目標也可能以精確忠實度換取特徵相似性。回到本章評估段落,在相同解析度與條件下,比較完整流程的 NFE、預訓練揭露、品質及實測延遲。

接著閱讀: ML 08・評估生成模型

參考文獻

  1. L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui and M.-H. Yang, “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. arXiv
  2. Y. Lipman, M. Havasi, P. Holderrieth, N. Shaul, M. Le, B. Karrer, R. T. Q. Chen, D. Lopez-Paz, H. Ben-Hamu and I. Gat, “Flow Matching Guide and Code,” arXiv:2412.06264, 2024. arXiv
  3. J. Ho, A. Jain and P. Abbeel, “Denoising Diffusion Probabilistic Models,” NeurIPS, 2020. arXiv
  4. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR, 2021. arXiv
  5. J. Song, C. Meng and S. Ermon, “Denoising Diffusion Implicit Models,” ICLR, 2021. arXiv
  6. J. Ho and T. Salimans, “Classifier-Free Diffusion Guidance,” NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications; arXiv:2207.12598, 2022. arXiv
  7. R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. arXiv
  8. W. Peebles and S. Xie, “Scalable Diffusion Models with Transformers,” ICCV, 2023. arXiv
  9. P. Esser, S. Kulal, A. Blattmann et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML, PMLR 235, 2024. PMLR
  10. Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel and M. Le, “Flow Matching for Generative Modeling,” ICLR, 2023. arXiv
  11. X. Liu, C. Gong and Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” ICLR, 2023. arXiv
  12. Y. Song, P. Dhariwal, M. Chen and I. Sutskever, “Consistency Models,” ICML, 2023. arXiv
  13. Z. Geng, M. Deng, X. Bai, J. Z. Kolter and K. He, “Mean Flows for One-step Generative Modeling,” NeurIPS, 2025. arXiv
  14. S. J. D. Prince, Understanding Deep Learning, MIT Press, 2023. book site
  15. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan and S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics,” ICML, PMLR 37, 2015. PMLR
  16. P. Vincent, “A Connection Between Score Matching and Denoising Autoencoders,” Neural Computation, vol. 23, no. 7, pp. 1661–1674, 2011. doi
  17. Y. Song and S. Ermon, “Generative Modeling by Estimating Gradients of the Data Distribution,” NeurIPS, 2019. arXiv
  18. P. Dhariwal and A. Nichol, “Diffusion Models Beat GANs on Image Synthesis,” NeurIPS, 2021. NeurIPS
  19. A. Q. Nichol and P. Dhariwal, “Improved Denoising Diffusion Probabilistic Models,” ICML, PMLR 139, 2021. PMLR
  20. T. Karras, M. Aittala, T. Aila and S. Laine, “Elucidating the Design Space of Diffusion-Based Generative Models,” NeurIPS, 2022. arXiv
  21. C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li and J. Zhu, “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps,” NeurIPS, 2022. arXiv
  22. T. Salimans and J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” ICLR, 2022. arXiv
  23. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler and S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS (NIPS), 2017. arXiv
  24. T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen and T. Aila, “Improved Precision and Recall Metric for Assessing Generative Models,” NeurIPS, 2019. arXiv
  25. G. Stein, J. C. Cresswell, R. Hosseinzadeh et al., “Exposing Flaws of Generative Model Evaluation Metrics and Their Unfair Treatment of Diffusion Models,” NeurIPS, 2023. arXiv
  26. S. Jayasumana, S. Ramalingam, A. Veit, D. Glasner, A. Chakrabarti and S. Kumar, “Rethinking FID: Towards a Better Evaluation Metric for Image Generation,” CVPR, 2024. CVF
  27. F.-A. Croitoru, V. Hondru, R. T. Ionescu and M. Shah, “Diffusion Models in Vision: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. doi
  28. H. Cao, C. Tan, Z. Gao, Y. Xu, G. Chen, P.-A. Heng and S. Z. Li, “A Survey on Generative Diffusion Models,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 7, pp. 2814–2830, 2024. doi
  29. C.-H. Lai, Y. Song, D. Kim, Y. Mitsufuji and S. Ermon, “The Principles of Diffusion Models,” monograph, arXiv:2510.21890, 2025. arXiv
  30. H. Shen, J. Zhang, B. Xiong et al., “Efficient Diffusion Models: A Survey,” Transactions on Machine Learning Research, 2025. arXiv
  31. A. Melnik, M. Ljubljanac, C. Lu, Q. Yan, W. Ren and H. Ritter, “Video Diffusion Models: A Survey,” Transactions on Machine Learning Research, 2024. arXiv
  32. C. Lu and Y. Song, “Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models,” ICLR, 2025. arXiv
  33. A. Sauer, D. Lorenz, A. Blattmann and R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. arXiv
  34. M. S. Albergo and E. Vanden-Eijnden, “Building Normalizing Flows with Stochastic Interpolants,” ICLR, 2023. arXiv
  35. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” CVPR, 2009, pp. 248–255. doi
  36. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” CVPR, 2016, pp. 2818–2826. arXiv
  37. O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” MICCAI, LNCS 9351, pp. 234–241, 2015. arXiv
  38. A. Radford, J. W. Kim, C. Hallacy et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. arXiv
  39. M. Oquab, T. Darcet, T. Moutakanni et al., “DINOv2: Learning Robust Visual Features without Supervision,” Transactions on Machine Learning Research, 2024. arXiv
  40. B. Efron, “Tweedie’s Formula and Selection Bias,” Journal of the American Statistical Association, vol. 106, no. 496, pp. 1602–1614, 2011. doi
  41. A. Dosovitskiy, L. Beyer, A. Kolesnikov et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” ICLR, 2021. arXiv
  42. T. Li and K. He, “Back to Basics: Let Denoising Generative Models Denoise,” CVPR, 2026 (preprint 2025). paper, publication record, official JiT code.
  43. B. Zheng, N. Ma, S. Tong and S. Xie, “Diffusion Transformers with Representation Autoencoders,” ICLR, 2026 (preprint 2025). proceedings, manuscript.
  44. S. Tong, B. Zheng, Z. Wang, B. Tang, N. Ma, E. Brown, J. Yang, R. Fergus, Y. LeCun and S. Xie, “Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders,” CVPR Findings Track, 2026. manuscript, author publication record.
  45. D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna and R. Rombach, “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis,” ICLR, 2024. proceedings, official model card.
  46. Black Forest Labs, “FLUX.1 [dev],” official pretrained-model card, 2024. model card.
  47. W. Kong, Q. Tian, Z. Zhang et al., “HunyuanVideo: A Systematic Framework For Large Video Generative Models,” technical report, arXiv:2412.03603, 2024. report, official release.
  48. Team Wan, A. Wang, B. Ai, B. Wen et al., “Wan: Open and Advanced Large-Scale Video Generative Models,” technical report, arXiv:2503.20314, 2025. report, official Wan2.1 release.
  49. Z. Geng, Y. Lu, Z. Wu, E. Shechtman, J. Z. Kolter and K. He, “Improved Mean Flows: On the Challenges of Fastforward Generative Models,” CVPR, pp. 30467–30476, 2026. CVF
  50. Y. Lu, S. Lu, Q. Sun, H. Zhao, Z. Jiang, X. Wang, T. Li, Z. Geng and K. He, “One-step Latent-free Image Generation with Pixel Mean Flows,” ICML, PMLR 306, pp. 81265–81277, 2026. PMLR