DLCV 02 · A decision threshold changes which mistakes you make
AdvancedDLCV 02 · Deep Image ClassificationInteractive plot
Description
Move the decision threshold on fixed classifier scores. The confusion matrix changes, and the operating point slides along the ROC and precision-recall curves built from the same scores.
Try this: Find a threshold with no false positives and note where the dot sits on the ROC curve. How many positives are missed? Then raise the false-positive cost c: the curves do not move, but which point has the lowest mean error cost does.
Explanation
Answer the prediction first, or open the explanation when you are ready.
Show the explanation
A lower threshold admits more samples as positive. At 0.5 the point is (FPR, TPR) = (0.4, 0.8); at 0.3 it is (0.6, 1.0). Recall rose from 4 of 5 positives to 5 of 5, but false positives rose from 2 to 3. No threshold can move up-left on this curve: it only slides along it.
Lab · DLCV 02 · A decision threshold changes which mistakes you make
- Precision
- 0.66667
- Recall
- 0.8
- ROC AUC (all thresholds)
- 0.84
- F1
- 0.72727
- Accuracy
- 0.7
- Mean error cost
- 0.3
True positives (TP): 4; false positives (FP): 2; false negatives (FN): 1; true negatives (TN): 3. Lowering the threshold can recover missed positives while admitting false positives. Precision and recall use different denominators; accuracy alone cannot express unequal error costs.
Try this: Find a threshold with no false positives and note where the dot sits on the ROC curve. How many positives are missed? Then raise the false-positive cost c: the curves do not move, but which point has the lowest mean error cost does.
Ten fixed synthetic scores with five positive labels. Predict class 1 when score ≥ threshold; no model is trained. Each vertex of the curves is one threshold (one of the ten scores); lines between vertices are only drawn to guide the eye. ROC: x is FP/(FP+TN), y is recall TP/(TP+FN); AUC is the trapezoid area. Precision-recall: precision is undefined if nothing is predicted positive, so no dot is drawn. Mean error cost = (c×FP+FN)/10, with false-negative cost fixed at 1. Adjusting c changes evaluation, not scores, decisions or curves.