Full search page →

DLCV 02 · A decision threshold changes which mistakes you make

AdvancedDLCV 02 · Deep Image ClassificationInteractive plot

Description

Move the decision threshold on fixed classifier scores. The confusion matrix changes, and the operating point slides along the ROC and precision-recall curves built from the same scores.

Try this: Find a threshold with no false positives and note where the dot sits on the ROC curve. How many positives are missed? Then raise the false-positive cost c: the curves do not move, but which point has the lowest mean error cost does.

Predict first

The threshold is 0.5. You lower it to 0.3. Where does the operating point move on the ROC curve?

Hints

The chapter section that uses this lab:

Explanation

Answer the prediction first, or open the explanation when you are ready.

Show the explanation

A lower threshold admits more samples as positive. At 0.5 the point is (FPR, TPR) = (0.4, 0.8); at 0.3 it is (0.6, 1.0). Recall rose from 4 of 5 positives to 5 of 5, but false positives rose from 2 to 3. No threshold can move up-left on this curve: it only slides along it.

Lab · DLCV 02 · A decision threshold changes which mistakes you make

Predict first

The threshold is 0.5. You lower it to 0.3. Where does the operating point move on the ROC curve?

Precision
0.66667
Recall
0.8
ROC AUC (all thresholds)
0.84
F1
0.72727
Accuracy
0.7
Mean error cost
0.3

True positives (TP): 4; false positives (FP): 2; false negatives (FN): 1; true negatives (TN): 3. Lowering the threshold can recover missed positives while admitting false positives. Precision and recall use different denominators; accuracy alone cannot express unequal error costs.

Try this: Find a threshold with no false positives and note where the dot sits on the ROC curve. How many positives are missed? Then raise the false-positive cost c: the curves do not move, but which point has the lowest mean error cost does.

Ten fixed synthetic scores with five positive labels. Predict class 1 when score ≥ threshold; no model is trained. Each vertex of the curves is one threshold (one of the ten scores); lines between vertices are only drawn to guide the eye. ROC: x is FP/(FP+TN), y is recall TP/(TP+FN); AUC is the trapezoid area. Precision-recall: precision is undefined if nothing is predicted positive, so no dot is drawn. Mean error cost = (c×FP+FN)/10, with false-negative cost fixed at 1. Adjusting c changes evaluation, not scores, decisions or curves.

Method sources

Open in the Playground →