PR 03 · Complexity versus shrinkage
IntermediatePR 03 · Linear Models for Regression and ClassificationInteractive plot
Description
Fit polynomial basis functions with ridge regression and separate training error from recovery of the underlying function.
Try this: At degree seven, increase regularization. Does the lowest training error also give the lowest underlying-function error?
Hints
The chapter section that uses this lab:
Explanation
Answer the prediction first, or open the explanation when you are ready.
Show the explanation
It rises from about 0.0005 to about 0.0063. The extra flexibility fits the noise in the 12 training points, which lowers the training error and moves the curve away from the true function.
Lab · PR 03 · Complexity versus shrinkage
- Training mean squared error
- 0.0538
- True-function grid error
- 0.029
- Squared coefficient norm
- 5.403
Changing λ alters the fitted coefficients, not the observations. Large penalties can remove noise-fitting but also suppress the curvature needed to represent the signal.
Try this: At degree seven, increase regularization. Does the lowest training error also give the lowest underlying-function error?
The blue fit solves sum of squared residuals + λ times the squared non-intercept coefficients. Twelve fixed synthetic points have deterministic perturbations. The dashed curve is sin(πx); its grid error is available only because this is a known synthetic problem. Polynomial features are linear in their coefficients.