完整搜尋頁面 →

Domingos:機器學習實務中的關鍵觀念

選讀・ML 01・機器學習入門 的論文導讀(第 1/1 篇)。引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

問題與論點

為什麼技術上正確的學習器,仍可能產生不好用的系統?Domingos 主張,實務成敗取決於演算法周圍的選擇:表示、評估、最佳化、資料,以及泛化所需的假設 [26]。這是一篇整理經驗的觀點文章,不是提出新架構並爭取排行榜名次。先讀本章訓練/驗證/測試的區分,再用這篇檢查一個實際能執行的實驗。

方法與圖解

飛鏢圖區分系統性偏差與不同學得模型之間的變動;這是比喻,不是量測基準。
來源:Domingos, Communications of the ACM 55(10), 2012; author-hosted published article. 圖 1,PDF 第 4 頁。飛鏢圖區分系統性偏差與不同學得模型之間的變動;這是比喻,不是量測基準。 [26]

表 1 把學習器拆成三部分:表示決定能表達哪些預測器,評估決定什麼算好預測,最佳化則搜尋預測器。改進其中一部分,不會自動改善其他部分。可用下式作為閱讀記號:

R^(f)=1n∑i=1nℓ(f(xi),yi).\widehat R(f)=\frac{1}{n}\sum_{i=1}^{n}\ell(f(x_i),y_i).

ff 是候選預測器,xix_i、yiy_i 是訓練輸入與目標,nn 是訓練樣本數,ℓ\ell 衡量單一預測的誤差。這只是整理概念的記號,不是文章提出的新演算法。最佳化這個平均值,與估計未來資料表現,是不同工作。

圖 1 中,集中但偏離中心的落點表示系統性偏差,分散落點表示變動。圖中比較的是重複學習的結果,不是同一模型輸出的不同信心分數。較有表達力的模型可能降低表示誤差,也可能使選擇更依賴手上的樣本。

證據

圖 2 反駁「表示符合資料產生規則就一定較好」:即使真實分類器是一組規則,資料較少時,較簡單的機率學習器仍可能勝過規則學習器。該實驗使用 64 個布林特徵,曲線是 100 次隨機規則產生實驗的平均,誤差棒為兩個標準差。這支持特定學習條件下的結論,不是所有演算法家族的通用排名。

設計分析

文章提醒,反覆調參也會污染測試結果,即使沒有直接拿測試標籤訓練。交叉驗證也可能因過度用於選擇而被過擬合。應把預測器的訓練目標,與選擇特徵、超參數及停止規則的協定分開。特徵建構與更多相關資料,有時比擴大學習器搜尋清單更重要。

閱讀檢查點

選一個分類任務,分別寫下表示、指標與最佳化方法,再指出哪些資料影響了各項決策。若最終測試分數曾用來選特徵,該分數就不再是完全獨立的最終評估。用圖 1 解釋:只知道訓練誤差降低,為何仍無法判斷剩下哪種誤差。

限制與下一步

這些建議依情境而定,不是保證成功的定理。偏差與變異比喻不能取代任務特定的誤差分解,歷史案例也無法決定現代模型的排名。接著讀 ML 11 的評估章,把建議轉成分組切分、不確定性估計與部署量測。把文章當作實驗設計檢查表,仍須實際檢查資料。

接著閱讀: Temperature Scaling:不改類別的信心校準

參考文獻

  1. Shyandram, “Introduction to Machine Learning,” Medium post, 2024; updated 2026. link
  2. A. L. Samuel, “Some studies in machine learning using the game of checkers,” IBM Journal of Research and Development, vol. 3, no. 3, pp. 210–229, 1959. doi
  3. I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016. book site
  4. A. Ng, E. Shyu, A. Bagul and G. Ladwig, “Machine Learning Specialization,” DeepLearning.AI and Stanford Online, Coursera. course page
  5. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. link
  6. E. Alpaydin and C. Kaynak, “Optical Recognition of Handwritten Digits,” UCI Machine Learning Repository, 1998. doi
  7. M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives, and prospects,” Science, vol. 349, no. 6245, pp. 255–260, 2015. doi
  8. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018. book page
  9. Y. LeCun and I. Misra, “Self-supervised learning: The dark matter of intelligence,” Meta AI blog, 2021. link
  10. R. Bommasani, D. A. Hudson, E. Adeli et al., “On the opportunities and risks of foundation models,” arXiv:2108.07258, 2021. arXiv
  11. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan et al., “Language models are few-shot learners,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020. link
  12. K. He, X. Chen, S. Xie, Y. Li, P. Dollár and R. Girshick, “Masked autoencoders are scalable vision learners,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15979–15988, 2022. arXiv
  13. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright et al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022. link
  14. DeepSeek-AI, “DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,” Nature, vol. 645, pp. 633–638, 2025. arXiv
  15. A. K. Jain, “Data clustering: 50 years beyond K-means,” Pattern Recognition Letters, vol. 31, no. 8, pp. 651–666, 2010. doi
  16. I. T. Jolliffe and J. Cadima, “Principal component analysis: A review and recent developments,” Philosophical Transactions of the Royal Society A, vol. 374, no. 2065, 20150202, 2016. doi
  17. V. Chandola, A. Banerjee and V. Kumar, “Anomaly detection: A survey,” ACM Computing Surveys, vol. 41, no. 3, 2009. doi
  18. L. Bottou, F. E. Curtis and J. Nocedal, “Optimization methods for large-scale machine learning,” SIAM Review, vol. 60, no. 2, pp. 223–311, 2018. doi
  19. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. International Conference on Learning Representations (ICLR), 2015. arXiv
  20. A. E. Hoerl and R. W. Kennard, “Ridge regression: Biased estimation for nonorthogonal problems,” Technometrics, vol. 12, no. 1, pp. 55–67, 1970. doi
  21. Y. Bengio, A. Courville and P. Vincent, “Representation learning: A review and new perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, 2013. doi
  22. L. Jing and Y. Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 11, pp. 4037–4058, 2021. doi
  23. J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo and D. Tao, “A survey on self-supervised learning: Algorithms, applications, and future trends,” arXiv:2301.05712, 2023. arXiv
  24. R. Balestriero, M. Ibrahim, V. Sobal, A. Morcos, S. Shekhar et al., “A cookbook of self-supervised learning,” arXiv:2304.12210, 2023. arXiv
  25. S. Casper, X. Davies, C. Shi et al., “Open problems and fundamental limitations of reinforcement learning from human feedback,” Transactions on Machine Learning Research, 2023. TMLR · arXiv
  26. P. Domingos, “A Few Useful Things to Know About Machine Learning,” Communications of the ACM, 55(10), pp. 78–87, 2012. doi
  27. NumPy developers, “numpy.polyfit,” NumPy 文件,查閱日期 2026-11-05。最小平方求解與奇異值