VAE:透過可計算的下界學習
選讀・ML 07・生成模型:自編碼器、VAE、GAN 與正規化流 的論文導讀(第 1/2 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
問題與論點
非線性潛在變數模型的真實後驗難以計算時,如何訓練?Kingma 與 Welling 利用變分下界及重參數化梯度估計,同時學習生成模型與近似推論模型 [6]。先讀條件機率、KL 散度與反向傳播。編碼器是推論工具,不是保證找回唯一隱藏原因的解碼器反函數。
方法與圖解

圖 1 區分生成方向與辨識方向。對單筆觀測,
其中 為觀測, 為潛在變數, 為先驗, 為解碼器概似, 為近似後驗。、 分別屬於解碼器與編碼器; 是證據下界。它與對數證據的落差,等於近似後驗到模型真實後驗的 KL 散度。
對角高斯編碼器使用
其中 、 為預測平均與標準差, 表示逐元素乘法, 是與參數無關的雜訊。隨機性進入可微轉換後,梯度便能通過樣本建構流程。生成則從先驗取樣再執行解碼器,不需要輸入影像或編碼器。
證據
圖 2 在 MNIST 與 Frey faces、不同潛在維度下,比較變分下界最佳化與 wake-sleep。橫軸是處理過的訓練樣本數,不是經過時間。圖 3 另外檢查訓練資料量改變時的估計邊際概似。下界曲線、邊際概似估計與好看的生成影像,是不同證據,不能互換名稱。
設計分析
第一項鼓勵解釋觀測,KL 項把推得的編碼連到生成時使用的先驗。解碼器有彈性,不代表近似後驗就精確。重參數化技巧降低這類連續分布的梯度估計難度,但不能直接用同一公式解決任意離散取樣。
閱讀檢查點
指出訓練與無條件生成時,各由哪個分布提供潛在樣本。再解釋:編碼訓練影像後能良好重建,為何不能證明獨立先驗樣本也會產生同樣好的觀測?
限制與下一步
結果取決於概似家族與近似落差。平滑的二維潛在圖,不代表已證明解耦或不確定性校準。接著讀 GAN,對照明確的變分密度學習與對抗式樣本匹配;再讀 VQ-VAE,理解離散表示及另行學習的先驗。
接著閱讀: GAN:分布賽局與理想條件下的保證
參考文獻
- C. M. Bishop and H. Bishop, Deep Learning: Foundations and Concepts, Springer, 2024. Book
- S. J. D. Prince, Understanding Deep Learning, MIT Press, 2023. Book and notebooks
- S. Bond-Taylor, A. Leach, Y. Long and C. G. Willcocks, “Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022. Paper
- G. Papamakarios et al., “Normalizing Flows for Probabilistic Modeling and Inference,” JMLR, 2021. Review
- L. Theis, A. van den Oord and M. Bethge, “A note on the evaluation of generative models,” ICLR, 2016. Paper
- D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” ICLR, 2014. Paper
- S. R. Bowman et al., “Generating Sentences from a Continuous Space,” CoNLL, 2016. Paper
- I. Higgins et al., “β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework,” ICLR, 2017. Paper
- D. P. Kingma and M. Welling, “An Introduction to Variational Autoencoders,” Foundations and Trends in Machine Learning, 2019. Review
- A. Vahdat and J. Kautz, “NVAE: A Deep Hierarchical Variational Autoencoder,” NeurIPS, 2020. Paper
- I. Goodfellow et al., “Generative Adversarial Nets,” NeurIPS, 2014. Paper
- T. Salimans et al., “Improved Techniques for Training GANs,” NeurIPS, 2016. Paper
- M. Arjovsky, S. Chintala and L. Bottou, “Wasserstein GAN,” ICML, 2017. Paper
- I. Gulrajani et al., “Improved Training of Wasserstein GANs,” NeurIPS, 2017. Paper
- T. Miyato et al., “Spectral Normalization for Generative Adversarial Networks,” ICLR, 2018. Paper
- L. Mescheder, A. Geiger and S. Nowozin, “Which Training Methods for GANs do actually Converge?” ICML, 2018. Paper
- T. Karras, S. Laine and T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” CVPR, 2019. Paper
- L. Dinh, J. Sohl-Dickstein and S. Bengio, “Density estimation using Real NVP,” ICLR, 2017. Paper
- D. P. Kingma and P. Dhariwal, “Glow: Generative Flow with Invertible 1x1 Convolutions,” NeurIPS, 2018. Paper
- D. J. Rezende and S. Mohamed, “Variational Inference with Normalizing Flows,” ICML, 2015. Paper
- R. T. Q. Chen, Y. Rubanova, J. Bettencourt and D. Duvenaud, “Neural Ordinary Differential Equations,” NeurIPS, 2018. Paper
- Y. Lipman et al., “Flow Matching for Generative Modeling,” ICLR, 2023. Paper
- I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016, ch. 18–20. Book
- Y. Song and D. P. Kingma, “How to Train Your Energy-Based Models,” arXiv, 2021. Paper
- A. van den Oord, N. Kalchbrenner and K. Kavukcuoglu, “Pixel Recurrent Neural Networks,” ICML, 2016. Paper
- J. Ho, A. Jain and P. Abbeel, “Denoising Diffusion Probabilistic Models,” NeurIPS, 2020. Paper
- L. Yang et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. Survey
- E. Nalisnick et al., “Do Deep Generative Models Know What They Don’t Know?” ICLR, 2019. Paper
- M. Heusel et al., “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS, 2017. Paper
- M. S. M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet and S. Gelly, “Assessing Generative Models via Precision and Recall,” NeurIPS, 2018. Paper
- T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen and T. Aila, “Improved Precision and Recall Metric for Assessing Generative Models,” NeurIPS, 2019. Paper
- G. Stein et al., “Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models,” NeurIPS, 2023. Paper
- S. Jayasumana et al., “Rethinking FID: Towards a Better Evaluation Metric for Image Generation,” CVPR, 2024. Paper
- A. van den Oord, O. Vinyals and K. Kavukcuoglu, “Neural Discrete Representation Learning,” NeurIPS, 2017. Paper
- R. Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. Paper
- M. Kang et al., “Scaling up GANs for Text-to-Image Synthesis,” CVPR, 2023. Paper
- Y. Huang, A. Gokaslan, V. Kuleshov and J. Tompkin, “The GAN is dead; long live the GAN! A Modern GAN Baseline,” NeurIPS, 2024. Paper