Full search page →

PR 10 · Does confidence survive a shifted population?

AdvancedPR 10 · Recognition Capstone: Rejection, Shift and Learned FeaturesInteractive plot

Description

Set a rejection threshold, shift both class distributions and compare accepted coverage with selective error.

Try this: Hold the rejection threshold fixed while shifting the population. Can high reported confidence coexist with more accepted mistakes?

Lab · PR 10 · Does confidence survive a shifted population?

Accepted coverage
0.666
Error among accepted predictions
0.068
Deployment-optimal boundary
0

The classifier still places its decision boundary at zero, while the deployment-optimal boundary moves with δ. Rejecting uncertain predictions does not repair confidence under shift.

Try this: Hold the rejection threshold fixed while shifting the population. Can high reported confidence coexist with more accepted mistakes?

The fixed classifier uses training posterior sigmoid(2x), learned for equal-prior N(−1,1) and N(1,1). Deployment shifts both means by δ without updating the classifier. Coverage and accepted error are numerical integrals over the synthetic deployment population, not estimates from a benchmark.

Method sources

Open in the Playground →