Full search page →

One outlier, three meanings of best fit

AdvancedOPT 05 · Approximation, Statistical Estimation and Experiment DesignInteractive plot

Description

Move one observation and compare squared, absolute and Huber losses for fitting a constant.

Try this: Increase the outlier from 2 to 12. Compare the movement of the squared-loss and Huber estimates.

Predict first

The sixth observation moves from 2 up to 12 while the other five stay put. Which fit is dragged furthest?

Explanation

Answer the prediction first, or open the explanation when you are ready.

Show the explanation

The squared loss pulls with a force that grows with distance, so its fit moves from 0.33 to 2. The absolute and Huber losses cap each point's pull, so their fits stay at 0.25.

Lab · One outlier, three meanings of best fit

Predict first

The sixth observation moves from 2 up to 12 while the other five stay put. Which fit is dragged furthest?

Fitted constant
1.333
Total loss
27.917

The loss defines what an error costs. Robust residual loss and parameter regularization solve different problems.

Try this: Increase the outlier from 2 to 12. Compare the movement of the squared-loss and Huber estimates.

Five fixed observations are −1, −0.5, 0, 0.5, 1. The sixth is adjustable. Squared loss includes ½; Huber uses δ=1. For absolute loss, any point in [0,0.5] minimizes the loss; the plot chooses 0.25.

Method sources

Open in the Playground →