Full search page →

Same predictions, different probability quality

BeginnerNLP 03 · Neural Networks for NLPInteractive plot

Description

Apply one positive temperature to fixed validation logits and compare accuracy, log loss and squared probability error.

Try this: Find a temperature with lower log loss than T=1. Why does accuracy stay fixed even while confidence changes?

Predict first

Raise the temperature T from 1 to 4. Does the classifier's accuracy change?

Explanation

Answer the prediction first, or open the explanation when you are ready.

Show the explanation

Dividing every logit by T never changes its sign, so every predicted class is unchanged (accuracy stays 0.75 here). Only the confidence moves, and with it the log loss.

Lab · Same predictions, different probability quality

Predict first

Raise the temperature T from 1 to 4. Does the classifier's accuracy change?

Accuracy
75%
Mean log loss, nats
0.498
Brier score
0.177

Positive temperature preserves logit ordering and binary decisions. It can adjust overconfidence or underconfidence, but cannot correct which examples the classifier gets wrong.

Try this: Find a temperature with lower log loss than T=1. Why does accuracy stay fixed even while confidence changes?

Eight synthetic validation observations, not a benchmark. Binary probabilities use σ(logit/T), with a fixed 0.5 decision threshold. Choosing T here illustrates validation tuning; it does not establish performance on an unseen test set.

Method sources

Open in the Playground →