Full search page →

A VAE trades reconstruction against prior agreement

IntermediateML 06A · Linear Factor Models and AutoencodersInteractive plot

Description

Move and widen a Gaussian posterior, then compute expected decoder loss and the exact KL penalty.

Try this: Set q(z|x) equal to the prior. Why is KL zero but reconstruction still poor for x=1? Can shrinking σ improve one term while worsening the other?

Lab · A VAE trades reconstruction against prior agreement

Expected decoder NLL
0.806
KL(q || prior), nats
0.638
NLL + β KL
1.444

Reconstruction favors latent values near x=1 and a narrow posterior. KL penalizes both moving away from zero and changing variance away from one. Prior agreement alone does not guarantee useful reconstruction.

Try this: Set q(z|x) equal to the prior. Why is KL zero but reconstruction still poor for x=1? Can shrinking σ improve one term while worsening the other?

One observation x=1, q=N(μ,σ²), prior=N(0,1), fixed decoder p(x|z)=N(z,0.5²). Expected negative log likelihood is analytic; no samples or network training. β=1 gives the negative evidence lower bound for this model; other β values are weighted objectives.

Method sources

Open in the Playground →