Domingos:機器學習實務中的關鍵觀念
選讀・ML 01・機器學習入門 的論文導讀(第 1/1 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
問題與論點
為什麼技術上正確的學習器,仍可能產生不好用的系統?Domingos 主張,實務成敗取決於演算法周圍的選擇:表示、評估、最佳化、資料,以及泛化所需的假設 [26]。這是一篇整理經驗的觀點文章,不是提出新架構並爭取排行榜名次。先讀本章訓練/驗證/測試的區分,再用這篇檢查一個實際能執行的實驗。
方法與圖解

表 1 把學習器拆成三部分:表示決定能表達哪些預測器,評估決定什麼算好預測,最佳化則搜尋預測器。改進其中一部分,不會自動改善其他部分。可用下式作為閱讀記號:
是候選預測器,、 是訓練輸入與目標, 是訓練樣本數, 衡量單一預測的誤差。這只是整理概念的記號,不是文章提出的新演算法。最佳化這個平均值,與估計未來資料表現,是不同工作。
圖 1 中,集中但偏離中心的落點表示系統性偏差,分散落點表示變動。圖中比較的是重複學習的結果,不是同一模型輸出的不同信心分數。較有表達力的模型可能降低表示誤差,也可能使選擇更依賴手上的樣本。
證據
圖 2 反駁「表示符合資料產生規則就一定較好」:即使真實分類器是一組規則,資料較少時,較簡單的機率學習器仍可能勝過規則學習器。該實驗使用 64 個布林特徵,曲線是 100 次隨機規則產生實驗的平均,誤差棒為兩個標準差。這支持特定學習條件下的結論,不是所有演算法家族的通用排名。
設計分析
文章提醒,反覆調參也會污染測試結果,即使沒有直接拿測試標籤訓練。交叉驗證也可能因過度用於選擇而被過擬合。應把預測器的訓練目標,與選擇特徵、超參數及停止規則的協定分開。特徵建構與更多相關資料,有時比擴大學習器搜尋清單更重要。
閱讀檢查點
選一個分類任務,分別寫下表示、指標與最佳化方法,再指出哪些資料影響了各項決策。若最終測試分數曾用來選特徵,該分數就不再是完全獨立的最終評估。用圖 1 解釋:只知道訓練誤差降低,為何仍無法判斷剩下哪種誤差。
限制與下一步
這些建議依情境而定,不是保證成功的定理。偏差與變異比喻不能取代任務特定的誤差分解,歷史案例也無法決定現代模型的排名。接著讀 ML 11 的評估章,把建議轉成分組切分、不確定性估計與部署量測。把文章當作實驗設計檢查表,仍須實際檢查資料。
接著閱讀: Temperature Scaling:不改類別的信心校準
參考文獻
- Shyandram, “Introduction to Machine Learning,” Medium post, 2024; updated 2026. link
- A. L. Samuel, “Some studies in machine learning using the game of checkers,” IBM Journal of Research and Development, vol. 3, no. 3, pp. 210–229, 1959. doi
- I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016. book site
- A. Ng, E. Shyu, A. Bagul and G. Ladwig, “Machine Learning Specialization,” DeepLearning.AI and Stanford Online, Coursera. course page
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. link
- E. Alpaydin and C. Kaynak, “Optical Recognition of Handwritten Digits,” UCI Machine Learning Repository, 1998. doi
- M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives, and prospects,” Science, vol. 349, no. 6245, pp. 255–260, 2015. doi
- R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018. book page
- Y. LeCun and I. Misra, “Self-supervised learning: The dark matter of intelligence,” Meta AI blog, 2021. link
- R. Bommasani, D. A. Hudson, E. Adeli et al., “On the opportunities and risks of foundation models,” arXiv:2108.07258, 2021. arXiv
- T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan et al., “Language models are few-shot learners,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020. link
- K. He, X. Chen, S. Xie, Y. Li, P. Dollár and R. Girshick, “Masked autoencoders are scalable vision learners,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15979–15988, 2022. arXiv
- L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright et al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022. link
- DeepSeek-AI, “DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,” Nature, vol. 645, pp. 633–638, 2025. arXiv
- A. K. Jain, “Data clustering: 50 years beyond K-means,” Pattern Recognition Letters, vol. 31, no. 8, pp. 651–666, 2010. doi
- I. T. Jolliffe and J. Cadima, “Principal component analysis: A review and recent developments,” Philosophical Transactions of the Royal Society A, vol. 374, no. 2065, 20150202, 2016. doi
- V. Chandola, A. Banerjee and V. Kumar, “Anomaly detection: A survey,” ACM Computing Surveys, vol. 41, no. 3, 2009. doi
- L. Bottou, F. E. Curtis and J. Nocedal, “Optimization methods for large-scale machine learning,” SIAM Review, vol. 60, no. 2, pp. 223–311, 2018. doi
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. International Conference on Learning Representations (ICLR), 2015. arXiv
- A. E. Hoerl and R. W. Kennard, “Ridge regression: Biased estimation for nonorthogonal problems,” Technometrics, vol. 12, no. 1, pp. 55–67, 1970. doi
- Y. Bengio, A. Courville and P. Vincent, “Representation learning: A review and new perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, 2013. doi
- L. Jing and Y. Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 11, pp. 4037–4058, 2021. doi
- J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo and D. Tao, “A survey on self-supervised learning: Algorithms, applications, and future trends,” arXiv:2301.05712, 2023. arXiv
- R. Balestriero, M. Ibrahim, V. Sobal, A. Morcos, S. Shekhar et al., “A cookbook of self-supervised learning,” arXiv:2304.12210, 2023. arXiv
- S. Casper, X. Davies, C. Shi et al., “Open problems and fundamental limitations of reinforcement learning from human feedback,” Transactions on Machine Learning Research, 2023. TMLR · arXiv
- P. Domingos, “A Few Useful Things to Know About Machine Learning,” Communications of the ACM, 55(10), pp. 78–87, 2012. doi
- NumPy developers, “numpy.polyfit,” NumPy 文件,查閱日期 2026-11-05。最小平方求解與奇異值