PR 10 · Does confidence survive a shifted population?
AdvancedPR 10 · Recognition Capstone: Rejection, Shift and Learned FeaturesInteractive plot
Description
Set a rejection threshold, shift both class distributions and compare accepted coverage with selective error.
Try this: Hold the rejection threshold fixed while shifting the population. Can high reported confidence coexist with more accepted mistakes?
Hints
The chapter section that uses this lab:
Lab · PR 10 · Does confidence survive a shifted population?
- Accepted coverage
- 0.666
- Error among accepted predictions
- 0.068
- Deployment-optimal boundary
- 0
The classifier still places its decision boundary at zero, while the deployment-optimal boundary moves with δ. Rejecting uncertain predictions does not repair confidence under shift.
Try this: Hold the rejection threshold fixed while shifting the population. Can high reported confidence coexist with more accepted mistakes?
The fixed classifier uses training posterior sigmoid(2x), learned for equal-prior N(−1,1) and N(1,1). Deployment shifts both means by δ without updating the classifier. Coverage and accepted error are numerical integrals over the synthetic deployment population, not estimates from a benchmark.