完整搜尋頁面 →

論文導讀

選讀:ML 10・基礎模型與視覺語言學習 所依據論文的導讀,每篇一頁:說明它要解決的問題、核心想法、結果,以及閱讀時該注意什麼。

導覽:本頁與本系列: 導覽:本頁與本系列

以下為選讀導讀。先讀問題與原圖,再逐層展開證據、設計分析及閱讀檢查點。

  1. CLIP:從影像文字配對到零樣本分類器

    自然語言監督能否讓影像編碼器適用於新分類任務,而不必重新擬合任務專用分類器?

參考文獻

  1. R. Bommasani et al., “On the Opportunities and Risks of Foundation Models,” Stanford CRFM report, arXiv:2108.07258, 2021; revised 2022. arXiv
  2. D. Caffagni et al., “The Revolution of Multimodal Large Language Models: A Survey,” Findings of ACL, 2024. arXiv
  3. A. Radford et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. arXiv
  4. X. Zhai, B. Mustafa, A. Kolesnikov and L. Beyer, “Sigmoid Loss for Language Image Pre-Training,” ICCV, 2023. arXiv
  5. A. Kirillov et al., “Segment Anything,” ICCV, 2023. arXiv
  6. N. Ravi et al., “SAM 2: Segment Anything in Images and Videos,” ICLR, 2025. arXiv
  7. N. Carion et al., “SAM 3: Segment Anything with Concepts,” arXiv:2511.16719, 2025; revised March 2026. arXiv
  8. H. Liu, C. Li, Q. Wu and Y. J. Lee, “Visual Instruction Tuning,” NeurIPS, 2023. arXiv
  9. S. Bai et al., “Qwen3-VL Technical Report,” arXiv:2511.21631, 2025. arXiv
  10. E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” ICLR, 2022. arXiv
  11. M. Assran et al., “V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning,” arXiv:2506.09985, 2025. arXiv
  12. J. Wang et al., “VGGT: Visual Geometry Grounded Transformer,” CVPR, 2025. arXiv
  13. L. Han et al., “A Survey of Generative Categories and Techniques in Multimodal Generative Models,” arXiv:2506.10016, 2025. arXiv
  14. C. M. Bishop and H. Bishop, Deep Learning: Foundations and Concepts, Springer, 2024. Book
  15. M. Oquab et al., “DINOv2: Learning Robust Visual Features without Supervision,” Transactions on Machine Learning Research, 2024. arXiv
  16. O. Siméoni et al., “DINOv3,” arXiv:2508.10104, 2025. arXiv
  17. J. Kaplan et al., “Scaling Laws for Neural Language Models,” arXiv:2001.08361, 2020. arXiv
  18. J. Hoffmann et al., “Training Compute-Optimal Large Language Models,” arXiv:2203.15556, 2022; published as “An empirical analysis of compute-optimal large language model training,” NeurIPS, 2022. arXiv
  19. X. Zhai, A. Kolesnikov, N. Houlsby and L. Beyer, “Scaling Vision Transformers,” CVPR, 2022. arXiv
  20. M. Cherti et al., “Reproducible Scaling Laws for Contrastive Language-Image Learning,” CVPR, 2023. arXiv
  21. A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” ICLR, 2021. arXiv
  22. K. He, X. Chen, S. Xie, Y. Li, P. Dollár and R. Girshick, “Masked Autoencoders Are Scalable Vision Learners,” CVPR, 2022. arXiv
  23. H. Bao, L. Dong, S. Piao and F. Wei, “BEiT: BERT Pre-Training of Image Transformers,” ICLR, 2022. arXiv
  24. M. Caron et al., “Emerging Properties in Self-Supervised Vision Transformers,” ICCV, 2021. arXiv
  25. C. Jia et al., “Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision,” ICML, 2021. arXiv
  26. J. Li, D. Li, S. Savarese and S. Hoi, “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models,” ICML, 2023. PMLR
  27. J.-B. Alayrac et al., “Flamingo: a Visual Language Model for Few-Shot Learning,” NeurIPS, 2022. arXiv
  28. H. Liu, C. Li, Y. Li and Y. J. Lee, “Improved Baselines with Visual Instruction Tuning,” CVPR, 2024. arXiv
  29. N. Houlsby et al., “Parameter-Efficient Transfer Learning for NLP,” ICML, 2019. PMLR
  30. M. Jia et al., “Visual Prompt Tuning,” ECCV, 2022. arXiv
  31. T. Dettmers, A. Pagnoni, A. Holtzman and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” NeurIPS, 2023. arXiv
  32. A. Kumar, A. Raghunathan, R. Jones, T. Ma and P. Liang, “Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution,” ICLR, 2022. arXiv
  33. M. Wortsman et al., “Robust Fine-Tuning of Zero-Shot Models,” CVPR, 2022. arXiv
  34. C. Schuhmann et al., “LAION-5B: An Open Large-Scale Dataset for Training Next Generation Image-Text Models,” NeurIPS Datasets and Benchmarks Track, 2022. arXiv
  35. A. Birhane, V. U. Prabhu and E. Kahembwe, “Multimodal Datasets: Misogyny, Pornography, and Malignant Stereotypes,” arXiv:2110.01963, 2021. arXiv
  36. P. Mayilvahanan, T. Wiedemer, E. Rusak, M. Bethge and W. Brendel, “Does CLIP’s Generalization Performance Mainly Stem from High Train-Test Similarity?,” ICLR, 2024. arXiv
  37. Y. Li et al., “Evaluating Object Hallucination in Large Vision-Language Models,” EMNLP, 2023. arXiv
  38. X. Yue et al., “MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI,” CVPR, 2024. arXiv
  39. M. Awais et al., “Foundational Models Defining a New Era in Vision: A Survey and Outlook,” arXiv:2307.13721, 2023. arXiv
  40. S. Yin et al., “A Survey on Multimodal Large Language Models,” National Science Review, 2024. arXiv
  41. J. Zhang, J. Huang, S. Jin and S. Lu, “Vision-Language Models for Vision Tasks: A Survey,” IEEE TPAMI, vol. 46, no. 8, 2024. arXiv
  42. L. Xu et al., “Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment,” arXiv:2312.12148, 2023. arXiv
  43. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” CVPR, 2009, pp. 248–255. doi
  44. J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” NAACL-HLT, 2019, pp. 4171–4186. doi
  45. OpenAI, “GPT-4 Technical Report,” arXiv:2303.08774, 2023. arXiv