SimCLR:正樣本對如何塑造表示
選讀・ML 06・表示學習與自監督 的論文導讀(第 1/1 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
問題與論點
不使用人工類別標籤,能否透過配對同一影像的變換視圖,學到有用視覺表示?Chen 等人把增強流程、非線性投影頭與對比最佳化列為核心設計 [7]。先讀向量相似度與交叉熵。「自監督」說明目標從哪裡來,不代表系統沒有「哪些內容應保持可辨識」的假設。
方法與圖解

圖 2 對同一影像獨立抽取兩種增強。共享編碼器產生表示,投影頭再映到計算對比損失的空間。由 張來源影像得到 個視圖時,單向正樣本對損失為
其中 、 是同一來源影像的投影向量, 遍歷批次中其他視圖, 為餘弦相似度, 為正溫度。分母排除錨點自己,但包含正樣本。訓練會平均每對正樣本的兩個方向;下游評估使用投影前的編碼器表示。
證據
論文以凍結表示搭配可訓練線性分類器評估,也研究少標籤微調;兩種協定回答不同問題。增強實驗顯示,裁切與色彩變換的組合很重要。圖 8 比較無投影、線性頭與非線性頭,非線性頭改善了被評估的編碼器表示。這比把所有進步都歸給單一「對比學習」標籤更具體。
設計分析
投影空間可以捨去對比任務不需要的細節,同時讓前一層表示保留其他任務可能需要的資訊。因此評估時移除投影頭符合論文設計,並非與訓練矛盾。批次大小、訓練長度與編碼器寬度也影響結果;應比較相同設定,不要把最高的摘要準確度套到每種編碼器。
閱讀檢查點
算出單一損失分母有多少候選,說明為何包含正樣本卻排除錨點。再提出一種會改變下游標籤的增強;若仍把兩個視圖視為正樣本對,會鼓勵哪種不正確的不變性?
限制與下一步
不同來源影像可能共享語意,卻被當成負樣本;強增強也可能刪除任務所需細節。線性探測量測受限分類器能否取用表示,不代表表示的所有可能用途。接著讀 ML 10 與 CLIP:把第二張影像視圖換成配對文字,會如何改變正樣本定義與下游預測介面?
接著閱讀: CLIP:從影像文字配對到零樣本分類器
參考文獻
- Y. Bengio, A. Courville and P. Vincent, “Representation Learning: A Review and New Perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, 2013. Paper · arXiv
- J. Yosinski, J. Clune, Y. Bengio and H. Lipson, “How transferable are features in deep neural networks?” NeurIPS, 2014. Paper
- X. Liu et al., “Self-supervised Learning: Generative or Contrastive,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, 2023 (online 2021). Paper · arXiv
- J. Gui et al., “A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. Paper · arXiv
- R. Balestriero et al., “A Cookbook of Self-Supervised Learning,” arXiv:2304.12210, 2023. Paper
- A. van den Oord, Y. Li and O. Vinyals, “Representation Learning with Contrastive Predictive Coding,” arXiv:1807.03748, 2018. Paper
- T. Chen, S. Kornblith, M. Norouzi and G. Hinton, “A Simple Framework for Contrastive Learning of Visual Representations,” ICML, 2020. Paper
- K. He, H. Fan, Y. Wu, S. Xie and R. Girshick, “Momentum Contrast for Unsupervised Visual Representation Learning,” CVPR, 2020. Paper
- T. Wang and P. Isola, “Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere,” ICML, 2020. Paper
- J.-B. Grill et al., “Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning,” NeurIPS, 2020. Paper
- X. Chen and K. He, “Exploring Simple Siamese Representation Learning,” CVPR, 2021. Paper
- M. Caron et al., “Emerging Properties in Self-Supervised Vision Transformers,” ICCV, 2021. Paper
- J. Zbontar, L. Jing, I. Misra, Y. LeCun and S. Deny, “Barlow Twins: Self-Supervised Learning via Redundancy Reduction,” ICML, 2021. Paper
- A. Bardes, J. Ponce and Y. LeCun, “VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning,” ICLR, 2022. Paper
- H. Bao, L. Dong, S. Piao and F. Wei, “BEiT: BERT Pre-Training of Image Transformers,” ICLR, 2022. Paper
- K. He et al., “Masked Autoencoders Are Scalable Vision Learners,” CVPR, 2022. Paper
- M. Assran et al., “Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture,” CVPR, 2023. Paper
- M. Assran et al., “V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning,” technical report, 2025. Report
- A. Radford et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. Paper
- X. Zhai, B. Mustafa, A. Kolesnikov and L. Beyer, “Sigmoid Loss for Language Image Pre-Training,” ICCV, 2023. Paper
- M. Oquab et al., “DINOv2: Learning Robust Visual Features without Supervision,” Transactions on Machine Learning Research, 2024. Paper
- O. Siméoni et al., “DINOv3,” technical report, 2025. Report
- A. Kumar, A. Raghunathan, R. Jones, T. Ma and P. Liang, “Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution,” ICLR, 2022. Paper
- S. J. D. Prince, Understanding Deep Learning, MIT Press, 2023. Book and notebooks