PR 02 · How wide should a density kernel be?
IntermediatePR 02 · Density Estimation and Bayesian LearningInteractive plot
Description
Change the bandwidth and inspect individual kernels, the estimated density and leave-one-out prediction.
Try this: Find a bandwidth that improves the leave-one-out log score. Why can a spiky fit score worse on a held-out point?
Explanation
Answer the prediction first, or open the explanation when you are ready.
Show the explanation
A small h leaves each held-out point almost alone (score about −5.9), a large h blurs the two clusters together (about −2.2), and a moderate h scores best (about −1.8).
Lab · PR 02 · How wide should a density kernel be?
- Mean leave-one-out log density
- -1.827
- Peak density
- 0.283
Small bandwidth preserves local bumps; large bandwidth merges them. Leave-one-out evaluation prevents each sample from predicting itself.
Try this: Find a bandwidth that improves the leave-one-out log score. Why can a spiky fit score worse on a held-out point?
Ten fixed synthetic observations; Gaussian kernels integrate to one. The score averages log density at each point after excluding that point. It is a density score in nats, not classification accuracy. The plot shows individual contributions divided by ten.