Full search page →

PR 02 · How wide should a density kernel be?

IntermediatePR 02 · Density Estimation and Bayesian LearningInteractive plot

Description

Change the bandwidth and inspect individual kernels, the estimated density and leave-one-out prediction.

Try this: Find a bandwidth that improves the leave-one-out log score. Why can a spiky fit score worse on a held-out point?

Predict first

Which bandwidth gives the best held-out log-likelihood: h = 0.1, 0.5 or 2?

Hints

The chapter section that uses this lab:

Explanation

Answer the prediction first, or open the explanation when you are ready.

Show the explanation

A small h leaves each held-out point almost alone (score about −5.9), a large h blurs the two clusters together (about −2.2), and a moderate h scores best (about −1.8).

Lab · PR 02 · How wide should a density kernel be?

Predict first

Which bandwidth gives the best held-out log-likelihood: h = 0.1, 0.5 or 2?

Mean leave-one-out log density
-1.827
Peak density
0.283

Small bandwidth preserves local bumps; large bandwidth merges them. Leave-one-out evaluation prevents each sample from predicting itself.

Try this: Find a bandwidth that improves the leave-one-out log score. Why can a spiky fit score worse on a held-out point?

Ten fixed synthetic observations; Gaussian kernels integrate to one. The score averages log density at each point after excluding that point. It is a density score in nats, not classification accuracy. The plot shows individual contributions divided by ten.

Method sources

Open in the Playground →