Full search page →

Paper guides

Optional reading guides to the papers behind ML 01A · Linear Algebra for Learning, one page each: the problem, the idea, what it showed and what to check when you read it.

Navigate: this page and this track: Navigate: this page and this track

These guides are optional. Start with the question and original figure, then unfold the evidence, design analysis and reading checkpoint.

  1. PCA — Jolliffe and Cadima’s Review

    How can many correlated measurements be summarized without silently changing the meaning of the data?

References

  1. I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016, ch. 2 “Linear Algebra”. Book
  2. M. P. Deisenroth, A. A. Faisal and C. S. Ong, Mathematics for Machine Learning, Cambridge University Press, 2020. Book
  3. G. Strang, Introduction to Linear Algebra, 6th ed., Wellesley-Cambridge Press, 2023. Book site
  4. G. H. Golub and C. F. Van Loan, Matrix Computations, 4th ed., Johns Hopkins University Press, 2013. DOI
  5. L. N. Trefethen and D. Bau III, Numerical Linear Algebra, SIAM, 1997. DOI
  6. C. Eckart and G. Young, “The Approximation of One Matrix by Another of Lower Rank,” Psychometrika, vol. 1, pp. 211–218, 1936. DOI
  7. R. Penrose, “A Generalized Inverse for Matrices,” Mathematical Proceedings of the Cambridge Philosophical Society, vol. 51, pp. 406–413, 1955. DOI
  8. A. E. Hoerl and R. W. Kennard, “Ridge Regression: Biased Estimation for Nonorthogonal Problems,” Technometrics, vol. 12, pp. 55–67, 1970. DOI
  9. K. Pearson, “On Lines and Planes of Closest Fit to Systems of Points in Space,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, vol. 2, pp. 559–572, 1901. DOI
  10. H. Hotelling, “Analysis of a Complex of Statistical Variables into Principal Components,” Journal of Educational Psychology, vol. 24, pp. 417–441, 1933. DOI
  11. I. T. Jolliffe and J. Cadima, “Principal Component Analysis: A Review and Recent Developments,” Philosophical Transactions of the Royal Society A, vol. 374, 20150202, 2016. DOI
  12. N. Halko, P.-G. Martinsson and J. A. Tropp, “Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions,” SIAM Review, vol. 53, no. 2, pp. 217–288, 2011. DOI · arXiv
  13. P.-G. Martinsson and J. A. Tropp, “Randomized Numerical Linear Algebra: Foundations and Algorithms,” Acta Numerica, vol. 29, pp. 403–572, 2020. DOI · arXiv
  14. R. Murray, J. Demmel, M. W. Mahoney, N. B. Erichson, M. Melnichenko, O. A. Malik, L. Grigori, P. Luszczek, M. Dereziński, M. E. Lopes, T. Liang, H. Luo and J. Dongarra, “Randomized Numerical Linear Algebra: A Perspective on the Field With an Eye to Software,” LAPACK Working Note 299, 2023. arXiv
  15. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” in Proc. International Conference on Learning Representations (ICLR), 2022. arXiv
  16. J. Zhao, Z. Zhang, B. Chen, Z. Wang, A. Anandkumar and Y. Tian, “GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection,” ICML, 2024. arXiv
  17. T. Miyato, T. Kataoka, M. Koyama and Y. Yoshida, “Spectral Normalization for Generative Adversarial Networks,” ICLR, 2018. arXiv
  18. T. G. Kolda and B. W. Bader, “Tensor Decompositions and Applications,” SIAM Review, vol. 51, no. 3, pp. 455–500, 2009. DOI
  19. C. M. Bishop and H. Bishop, Deep Learning: Foundations and Concepts, Springer, 2024. Book