Supplementary notes
Optional notes for IT 07 · Statistical Evidence, Maximum Entropy and Growth, one page each. Nothing in the main text depends on them.
Navigate: this page and this track: Navigate: this page and this track
These notes are optional examples of why a theorem’s assumptions matter.
- When the support moves with the parameter
A derivative taken inside a density’s support can miss information carried by its moving boundary.
References
-
T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006, Chapters 6, 11–12 and 16. Publisher book and contents.
-
T. Weissman, “Method of Types,” Stanford EE276 Lecture 15 notes, 2024. Primary lecture notes.
-
F. Hellström, G. Durisi, B. Guedj and M. Raginsky, “Generalization Bounds: Perspectives from Information Theory and PAC-Bayes,” Foundations and Trends in Machine Learning, 18(1), 1–223, 2025. Publisher metadata, author version.
-
A. El Gamal, “Information Theory and Statistical Learning,” proposed textbook chapter, arXiv preprint, 2026. Draft and publication status.
-
MIT OpenCourseWare, “Lecture 6: Efficient Estimators, Rao-Cramer Bound,” 14.381 Statistical Method in Economics, Fall 2018, pp. 3–6. Primary lecture notes.