Full search page →

Observe · Predict · Test

Playground

Start with your chapter and one important question. Predict, change a parameter, and inspect the result. Find the experiments that matter for each chapter, with a reading-only choice where appropriate.

73 experiments97 chapter guides8 tracksRuns in your browser
Choose experiments chapter by chapter 97

Each chapter has a suggested starting point. A shared experiment can support the same concept in several chapters.

Foundations
  1. Start Here · Learning Paths and Recommended Textbooks

    This chapter is a reading and setup guide. Choose a learning route and run its first example; a separate numerical playground would not test that objective.

  2. Images as Arrays

    Add the same number to every pixel of a real photo and compare 8-bit wraparound with saturating and floating-point arithmetic.

Image Processing
  1. IP 01 · Introduction

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  2. IP 02 · Digital Image Fundamentals

    Keep every s-th pixel and fewer gray levels of a real photo and of a synthetic zone plate, and see false patterns (aliasing) appear unless you filter first.

  3. IP 03 · Intensity Transformations and Spatial Filtering

    Predict the response of flat regions and edges as you change the kernel weights.

  4. IP 04 · Filtering in the Frequency Domain

    Filter a real photo in the frequency domain and compare an ideal, a Butterworth and a Gaussian low-pass at the same cutoff.

  5. IP 05 · Image Restoration and Reconstruction

    Blur a real photo with a Gaussian PSF, add a little noise, then divide by the blur in the frequency domain. See the plain inverse filter fail, and find the Wiener constant K that rescues it.

  6. IP 06 · Color Image Processing

    Multiply a real color photo by a gain and compare three ways of handling the channels that overflow: clip each one, rescale the pixel, or tone-map it.

  7. IP 07 · Wavelet and Other Image Transforms

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  8. IP 08 · Image Compression and Watermarking

    Transform a real photo with an 8 × 8 block DCT or a Haar wavelet, round every coefficient to a coarse step, and see where the lost information shows up.

  9. IP 09 · Morphological Image Processing

    Clean a thresholded photo of coins that carries specks, scratches and holes. Change the operation, the shape and the size of the structuring element, and see what each choice removes, fills or keeps.

  10. IP 10 · Image Segmentation I: Edge Detection, Thresholding, and Region Detection

    Dim one side of a page and watch Otsu’s single threshold lose the text, while a threshold that follows the local light keeps it.

  11. IP 11 · Image Segmentation II: Active Contours — Snakes and Level Sets

    Run a real snake, the Kass–Witkin–Terzopoulos contour of IP 11, on a photograph of coins: start a circle around one coin and see how far from its rim the circle may begin before the snake settles in the wrong place.

  12. IP 12 · Feature Extraction

    Slide a small window over a real photograph and watch the window’s gradient vectors, its structure-tensor ellipse and the Harris response decide whether the spot is flat, an edge or a corner.

  13. IP 13 · Image Pattern Classification

    Run leave-one-out k-nearest-neighbour classification on the real iris measurements, then change the unit of one feature: no flower moves, yet the neighbours and the accuracy do.

Natural Language Processing
  1. NLP 01 · Introduction to Natural Language Processing

    Build an n-gram distribution from a tiny editable corpus. Inspect counts, sentence boundaries and smoothing.

  2. NLP 02 · Linguistic Essentials

    Change a small reading relation and evaluate two quantifier orders exactly.

  3. NLP 03 · Neural Networks for NLP

    Compare a fixed two-unit network with and without nonlinear activations on the XOR corners.

  4. NLP 04 · Lexical Semantics and Word Embeddings

    Pad a three-word sentence with zero vectors and compare a correct masked mean with a mean that counts the padding in its denominator.

  5. NLP 05 · Recurrent Neural Networks for Language

    Compare a fixed recurrent decay with input-dependent write decisions while distractors pass through a sequence.

  6. NLP 06 · Sequence Labeling with RNNs (POS Tagging, Information Extraction)

    Combine emission scores, transition scores and BIO span constraints to choose a complete tag path.

  7. NLP 07 · Attention Mechanism and Transformers

    Compare bidirectional and causal attention using the same query, keys and values.

  8. NLP 08 · Sequence-to-Sequence Models with Transformers (Machine Translation, Chatbots)

    Compare bidirectional and causal attention using the same query, keys and values.

  9. NLP 09 · Language Modeling with Transformers (Natural Language Generation)

    Follow fixed logits through temperature scaling, top-k truncation, nucleus truncation and renormalization.

  10. NLP 10 · Pretrained Models: BERT

    Change a right-context clue, hide it causally, then attach different output heads to the same small contextual vector.

  11. NLP 11 · Classification with BERT (Sentiment Analysis, Natural Language Inference)

    Change a right-context clue, hide it causally, then attach different output heads to the same small contextual vector.

  12. NLP 12 · Syntactic and Semantic Tasks

    Change a small reading relation and evaluate two quantifier orders exactly.

Pattern Recognition
  1. PR 01 · Patterns, Probability and Bayes Decisions

    Move a decision boundary by changing class frequency and the cost of a false alarm.

  2. PR 02 · Density Estimation and Bayesian Learning

    Change the bandwidth and inspect individual kernels, the estimated density and leave-one-out prediction.

  3. PR 03 · Linear Models for Regression and Classification

    Fit polynomial basis functions with ridge regression and separate training error from recovery of the underlying function.

  4. PR 04 · Neighbors, Kernels and Nonlinear Classifiers

    Run leave-one-out k-nearest-neighbour classification on the real iris measurements, then change the unit of one feature: no flower moves, yet the neighbours and the accuracy do.

  5. PR 05 · Feature Selection, Transforms and Dimensionality

    Rotate a one-dimensional projection and compare reconstruction error with class separation.

  6. PR 06 · Template Matching and Structured Recognition

    Warp a signal in time and inspect a dynamic time warping cost matrix and its optimal alignment path.

  7. PR 07 · Clustering, Mixtures and EM

    Step through Gaussian-mixture EM and observe responsibilities, likelihood and the symmetric-initialization trap.

  8. PR 08 · Approximate Inference and Bayesian Model Complexity

    Move and widen a Gaussian approximation to a two-mode target; compare penalties for missing mass and filling the valley.

  9. PR 09 · Ensembles, Validation and Cluster Validity

    Grow real decision trees on 300 noisy two-moons points, bootstrap them, let them vote, and watch the test error fall and then flatten as trees are added.

  10. PR 10 · Recognition Capstone: Rejection, Shift and Learned Features

    Set a rejection threshold, shift both class distributions and compare accepted coverage with selective error.

Machine Learning
  1. ML 01 · Introduction to Machine Learning

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  2. ML 01A · Linear Algebra for Learning

    Move the images of the two basis vectors and watch a real photograph follow. The determinant is the signed area scale, the singular values are the stretch along two perpendicular directions, and rank 1 flattens the picture onto a line.

  3. ML 01B · Probability and Information Theory

    Change the true probability and the model probability; measure the extra coding cost caused by a wrong model.

  4. ML 01C · Numerical Computation and Gradient Checks

    Check the derivative of exp(x), separating truncation error from floating-point cancellation.

  5. ML 02 · Neural Networks from Perceptron to Deep Networks

    Train a network with 2 inputs, m tanh hidden units and 1 sigmoid output on the two moons and the two circles, one gradient-descent step at a time. Watch the decision map and the training and test loss: too few hidden units, too small a learning rate and too large a learning rate each fail in their own way.

  6. ML 03 · Convolution, Image Filtering and CNNs

    Predict the response of flat regions and edges as you change the kernel weights.

  7. ML 04 · Recurrent and Recursive Networks

    Compare a fixed recurrent decay with input-dependent write decisions while distractors pass through a sequence.

  8. ML 05 · Attention and Vision Transformers

    Compare bidirectional and causal attention using the same query, keys and values.

  9. ML 06 · Representation Learning and Self-Supervision

    Move paired and unpaired unit vectors, then inspect the normalized matching loss used by contrastive learning.

  10. ML 06A · Linear Factor Models and Autoencoders

    Transform a real photo with an 8 × 8 block DCT or a Haar wavelet, round every coefficient to a coarse step, and see where the lost information shows up.

  11. ML 07 · Generative Models: Autoencoders, VAEs, GANs and Flows

    Move and widen a Gaussian posterior, then compute expected decoder loss and the exact KL penalty.

  12. ML 08 · Diffusion Models and Flow Matching

    Mix the astronaut photograph with fixed noise, look at the noise, clean-image and velocity targets side by side, then give the network a small error and see which conversion back to the clean image survives.

  13. ML 09 · Autoregressive Vision: Pixels, Tokens and Scales

    Enumerate all sixteen binary 2×2 images under a first-order pixel model, then change the generation order.

  14. ML 10 · Foundation Models and Vision–Language Learning

    Move paired and unpaired unit vectors, then inspect the normalized matching loss used by contrastive learning.

  15. ML 11 · Evaluation, Efficiency and Deployment

    Apply one positive temperature to fixed validation logits and compare accuracy, log loss and squared probability error.

Deep Learning for Computer Vision
  1. DLCV 01 · Deep Learning for Image Processing and Computer Vision

    Pick a stack of convolution layers and a spot in a real photograph: the exact box of input pixels that one output unit depends on is drawn on the image, with its size, jump and the share of the image it covers.

  2. DLCV 01A · Training Deep Vision Models in Practice

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  3. DLCV 01B · Building Blocks and How to Compose Them: Classification and Restoration Networks

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  4. DLCV 01C · Model Compression and Efficient Inference

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  5. DLCV 02 · Deep Image Classification

    Move the decision threshold on fixed classifier scores. The confusion matrix changes, and the operating point slides along the ROC and precision-recall curves built from the same scores.

  6. DLCV 02A · Face Recognition and Metric Learning

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  7. DLCV 03 · Object Detection: From R-CNN to DETR

    When do overlapping detections represent one object? Inspect how greedy suppression makes that decision.

  8. DLCV 03A · One-Stage Detection, Read Through YOLOv4

    When do overlapping detections represent one object? Inspect how greedy suppression makes that decision.

  9. DLCV 03B · 3D Object Detection: LiDAR, Cameras and Bird's-Eye View

    Turn a predicted box a few degrees away from the ground truth in the bird’s-eye view and watch the exact rotated IoU fall while the centre distance stays at zero.

  10. DLCV 04 · Image Segmentation: Semantic, Instance and Panoptic

    Simulate a segmentation network that predicts at a coarse output stride and compare pixel accuracy, mean IoU and the IoU of the smallest class.

  11. DLCV 05 · Human Pose and Keypoint Estimation

    Render a Gaussian heatmap at output stride 1, 2, 4 or 8 for 500 random sub-pixel keypoints and compare three ways to read the position back: the brightest cell, a quarter-cell offset and a Taylor step.

  12. DLCV 06 · Text in Images: OCR and Document Understanding

    Edit the blank probability in the frames between two equal letters of a toy recognizer output, and watch greedy best-path decoding and prefix beam search read it.

  13. DLCV 07 · Object Tracking: From Single-Object to Multi-Object

    Track three coloured blocks through 30 synthetic frames with a Kalman filter, a cost matrix and the Hungarian algorithm, and watch two identities swap when only box overlap is used.

  14. DLCV 08 · Video Understanding and Action Recognition

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  15. DLCV 09 · Deep Image Restoration: Low-Level Vision

    A blurry observation fits two sharp clean signals. The blurry average has the lowest expected squared error; a sharp guess looks better but is usually farther from the truth.

  16. DLCV 09A · Non-Blind Image Super-Resolution

    A blurry observation fits two sharp clean signals. The blurry average has the lowest expected squared error; a sharp guess looks better but is usually farther from the truth.

  17. DLCV 09A.1 · Blind and Real-World Image Super-Resolution

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  18. DLCV 09A.2 · One-Step Diffusion Super-Resolution

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  19. DLCV 09A.3 · Video Super-Resolution: Using Time to Recover Detail

    Keep every s-th pixel and fewer gray levels of a real photo and of a synthetic zone plate, and see false patterns (aliasing) appear unless you filter first.

  20. DLCV 09B · Generative Image Restoration: Sampling a Plausible Answer

    A blurry observation fits two sharp clean signals. The blurry average has the lowest expected squared error; a sharp guess looks better but is usually farther from the truth.

  21. DLCV 09C · Low-Light Image Enhancement

    Simulate a dim photo with photon and read noise, then compare more light, more digital gain and a denoiser.

  22. DLCV 10 · Image Fusion: Complementary Images, Useful Outputs

    Two shots of one scene are each sharp in a different region. Average them, or pick every pixel from the shot with more local detail, and score both against the all-in-focus truth.

  23. DLCV 11 · 3D Reconstruction: From Multi-View Geometry to NeRF and Gaussian Splatting

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  24. DLCV 11A · Depth Estimation and Optical Flow

    Run block matching on a synthetic stereo pair with known disparity, and see small windows turn noisy while large windows fatten the foreground.

  25. DLCV 12 · Image Generation: From GANs to Text-to-Image Systems

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  26. DLCV 12A · Video Generation

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  27. DLCV 12B · One-Step Generation

    This chapter’s own figures, code and exercises carry its main idea; a separate interactive demo would not add to them.

  28. DLCV 13 · World Models: Video, Latent Dynamics and Planning

    Compare one-step predictions, open-loop rollout and periodic observation refresh under the same actions.

  29. DLCV 14 · Robustness: Distribution Shift, Adversarial Examples and Anomalies

    Train a linear classifier on noisy stripe images of 64, 256 or 1,024 pixels, then change every pixel by the same few levels, once along the loss gradient (FGSM) and once with random signs.

Optimization
  1. OPT 01 · Modeling Decisions and Convex Sets

    Place two feasible points on a circle, slide along the segment between them, and compare a disk with an annulus.

  2. OPT 02 · Convex Functions, Composition and Conjugates

    Measure the Jensen gap while moving between two inputs. A negative gap supplies a concrete counterexample.

  3. OPT 03 · Linear, Quadratic, Conic and Geometric Programs

    Compare coordinate clipping with the exact solution of a coupled quadratic under nonnegative constraints.

  4. OPT 04 · Duality, KKT Conditions and Certificates

    Change a candidate point and multiplier. A small gap is meaningful only with feasibility.

  5. OPT 05 · Approximation, Statistical Estimation and Experiment Design

    Move one observation and compare squared, absolute and Huber losses for fitting a constant.

  6. OPT 06 · Gradient Descent, Newton Steps and Numerical Reliability

    See why a step size that works on a round bowl can diverge on a narrow one.

  7. OPT 07 · Equality Constraints and Interior-Point Methods

    Raise the barrier weight t on a two-variable linear program and watch the central point, its dual bound and the gap m/t.

  8. OPT 08 · Geometric Applications, Splitting and Modern Methods

    Compare soft thresholding with ordinary shrinkage and inspect its nonsmooth objective.

Information Theory
  1. IT 01 · Entropy, Relative Entropy and Mutual Information

    Separate source uncertainty, transmitted information and channel capacity.

  2. IT 02 · Typical Sequences, AEP and Entropy Rates

    Compare the probability mass of a typical set with its fraction of all possible binary blocks.

  3. IT 03 · Lossless Coding and Universal Compression

    Build a binary prefix code and compare its average length with source entropy.

  4. IT 04 · Channel Capacity and Reliable Communication

    Separate source uncertainty, transmitted information and channel capacity.

  5. IT 05 · Differential Entropy and Gaussian Channels

    Compare equal allocation with water-filling across two parallel Gaussian channels.

  6. IT 06 · Rate–Distortion and Lossy Representations

    Inspect the Bernoulli rate–distortion curve and an attaining joint distribution under Hamming error.

  7. IT 07 · Statistical Evidence, Maximum Entropy and Growth

    Slide a decision threshold across two exact binomial distributions and compute both error probabilities.

  8. IT 08 · Networks, Information Inequalities and Learning

    Compute information about a noisy XOR label from each independent bit and from their pair.

Showing 73 experiments

Foundations

Images as Arrays · Why does brightening make bright pixels dark?

Add the same number to every pixel of a real photo and compare 8-bit wraparound with saturating and floating-point arithmetic.

Open experimentClose experiment
Predict first

Add 60 to every pixel of an 8-bit photo: which pixels change the most?

Loading image…

Original photo (8-bit)
Your arithmetic (see Show)
Correct result: clip(pixel + amount, 0, 255)
Output value against input value

Try this: With the amount at 60 and the view on overflow, see where the red pixels sit. Now switch the arithmetic to saturate: the same pixels turn white instead of black, and the PSNR against the correct result becomes infinite. Last, set the amount to −100 with wraparound: which pixels flip, and in which direction?

Camera test image reduced to 256 × 256 by 2 × 2 averaging, so the fractions differ slightly from the chapter, which counts on the 512 × 512 original. The pixels are real bytes: wraparound is done in a Uint8Array, so the sum is reduced modulo 256 exactly as NumPy does for gray + np.uint8(60). Saturate clamps the integer sum to 0..255 (what cv2.add does); float adds in floating point, then clips and rounds, which gives the same bytes for whole-number amounts. A negative amount corresponds to subtracting. PSNR compares the result with the correct clipped result, with peak 255. Brightness here is the mean pixel value, 0 to 255; this shows arithmetic on stored values, not a physical exposure model.

Method and image sources

ML 01A · What does a 2 × 2 matrix do to a picture?

Move the images of the two basis vectors and watch a real photograph follow. The determinant is the signed area scale, the singular values are the stretch along two perpendicular directions, and rank 1 flattens the picture onto a line.

Open experimentClose experiment
Predict first

The matrix A = [[0, 1], [1, 0]] sends e1 to (0, 1) and e2 to (1, 0): it swaps x and y. What does it do to the photograph?

Loading image…

Original photograph with the unit square, e1 (red), e2 (yellow) and the unit circle (dashed)
The photograph after A: the unit square, Ae1 (red), Ae2 (yellow) and the ellipse. Drag the round tips, or click, or focus and use keys 1, 2 and the arrow keys

Try this: The default is a shear: det A = 1, so areas are kept, yet σ1 = 1.62 and σ2 = 0.62 (κ = 2.62), and the unit circle becomes an ellipse. Keeping the area does not keep the shape. Now choose Stretch and slide k from 1 down through 0 to −1: at k = 0 the picture collapses onto a line (rank 1, κ = ∞), and for negative k it returns as a mirror image (det A < 0). Last, choose Projection and decide which points of the picture land on the same spot, and whether any matrix could undo that.

Real photograph: skimage.data.camera at 256 × 256 (credit below). The plane origin is the centre of the photo and 64 pixels are one unit, so the photo spans −2 to 2 in x and y, with y pointing up as in a maths figure. The right picture is made by inverse mapping: each output pixel looks up where it came from (the inverse of A applied to its position) with bilinear interpolation, and squeezed regions average several samples to avoid jagged edges; checkerboard marks places the picture does not reach. A rank-1 matrix has no inverse, so it is drawn as a line whose value at each spot is the mean of the photo along the whole line of points that collapsed onto it. Dashed cyan is the image of the unit circle, an ellipse whose semi-axes (solid cyan) have lengths σ1 and σ2; they are left out when σ1 = σ2. Controls: with u = (cos θ, sin θ), Shear is I + k·u·vT (v is u turned by 90°), Stretch is I + (k − 1)·u·uT (k = 1 is the identity), Rotation is k times a rotation by θ, Reflection is k times a mirror across the line along u, Projection is k·u·uT. Because they act along the direction θ, the two sliders move all four entries a, b, c, d. "Your own" has no restriction: drag the red and yellow tips on the right picture, or click it, or focus it and press 1 or 2 to choose an arrow and the arrow keys to move it (Shift for bigger steps). Any matrix with entries between −1.95 and 1.95 in steps of 0.01 is reachable; the sliders are switched off in that mode. Singular values use the closed-form 2 × 2 formula (Blinn), checked against numpy.linalg.svd; a matrix counts as singular when σ2 is below one billionth of σ1. Numbers are rounded to two decimals.

Method and image sources

Image Processing

IP 02 · What do you lose with fewer pixels, and what with fewer gray levels?

Keep every s-th pixel and fewer gray levels of a real photo and of a synthetic zone plate, and see false patterns (aliasing) appear unless you filter first.

Open experimentClose experiment
Predict first

You keep every 4th pixel in each direction. Does a blur before sampling help or hurt?

Loading image…

Photo: original
Photo: kept samples, enlarged
Zone plate (synthetic): original
Zone plate: kept samples, enlarged

Try this: Set the stride to 1 and the bits to 3 (8 gray levels): the photo shows bands, yet its PSNR is higher than with stride 4 at 8 bits, and even higher than with stride 2. Which loss does PSNR punish more? Then set the bits back to 8 and compare the zone plate at strides 2, 4 and 8: where do the false rings begin, and how does that radius shrink with the stride? Finally turn the Gaussian prefilter on.

Photo: the camera test image, 256 × 256. Zone plate: synthetic, 256 × 256, I(r) = 0.5 + 0.5 cos(k r²) with r the distance from the centre and k = π / (2 r_max), so its local frequency rises linearly to 0.5 cycles per pixel at the corner. Sampling keeps pixel (0, 0) of every stride × stride block (a[::s, ::s]); gray levels use the chapter’s rule of rounding to the centre of one of 2^bits bins; the result is enlarged by pixel replication (nearest neighbour) so single samples stay visible. The optional prefilter is a Gaussian with σ = stride / 2 applied before sampling. PSNR compares the enlarged result with the original 8-bit image (peak 255). False-pattern contrast is the standard deviation of the kept zone-plate samples that lie beyond radius π / (2 · stride · k), where the true pattern is finer than the coarse grid can show; a faithful result is flat gray there, so a large value means false rings. PSNR cannot tell a false pattern from honest blur, which is why both numbers are shown.

Method and image sources

IP 04 · Does the sharpest low-pass filter give the best image?

Filter a real photo in the frequency domain and compare an ideal, a Butterworth and a Gaussian low-pass at the same cutoff.

Open experimentClose experiment
Predict first

At the same cutoff, which of the three filters pushes the most pixels outside the original image’s brightness range (ringing)?

Loading image…

Original crop
Spectrum kept by the filter (log scale)
Filtered image
Pixels outside the original range: brighter (red), darker (blue)
One image row through a strong edge

Try this: Choose the ideal filter with cutoff 12 and look at the red and blue pixels around the edges. Switch to Gaussian at the same cutoff. Which filter rings, and which has the higher PSNR?

A real 128 × 128 crop of the camera test image. The transfer function multiplies its 2-D FFT (no padding, so the image is treated as periodic) and the inverse transform is not clipped before ripples are counted. Cutoff D0 is in cycles per image; Gaussian H = exp(−D²/2D0²); Butterworth H = 1/(1 + (D/D0)^2n). Overshoot counts pixels more than 0.02 outside the original image’s own gray range. PSNR compares the filtered image, clipped to [0, 1], with the original.

Method and image sources

IP 05 · Why does undoing a blur amplify the noise?

Blur a real photo with a Gaussian PSF, add a little noise, then divide by the blur in the frequency domain. See the plain inverse filter fail, and find the Wiener constant K that rescues it.

Open experimentClose experiment
Predict first

The camera image is blurred (σ = 2) and a barely visible amount of noise is added (0.005, about one gray level in 255). The plain inverse filter (K = 0) divides every frequency by the blur’s transfer function. What does the result look like?

Loading image…

Original (known)
Blurred, plus noise
Restored with H*/(|H|² + K)
PSNR of the restored image against K (the same blur and noise)

Try this: Leave the noise at 0.005 and raise log10 K from −7 in small steps. The restored PSNR climbs, peaks, then falls: where is the peak, and what does the image look like on each side of it? Then set the noise to 0: is K = 0 still a disaster? Finally raise the noise to 0.02 and see where the best K moves.

Real 256 × 256 camera image. The degradation follows the chapter, G = H·F + N, in the frequency domain. H is the transform of a Gaussian PSF of standard deviation σ pixels, H = exp(−2π²σ²(u² + v²)) with u and v in cycles per pixel; the blur is periodic (circular), as in the chapter’s code, so the filter matches the blur exactly and no padding is used. N is a fixed seeded white Gaussian field scaled by the noise level (a standard deviation as a fraction of the 0 to 1 range, so 0.005 is about 1.3 gray levels of 255); it is added to the blurred image and not rounded to 8 bits. The restoration is the constant-K Wiener filter of the chapter, F̂ = H*/(|H|² + K) · G. Here K is a tuning constant that you choose, not an estimate of the noise-to-signal power ratio. K = 0 is the plain inverse 1/H, where frequencies with |H| below 10^−6 are set to 0 (the convention of the chapter’s code); that is why its peak gain stops at about a million and why no-noise restoration is not perfect for large σ. PSNR compares each image, clipped to 0 to 1, with the original. The gain is the largest value of |H|/(|H|² + K) over all frequencies; for K greater than 0 it never exceeds 1/(2√K).

Method and image sources

IP 06 · Why does brightening turn reds and oranges yellow, then white?

Multiply a real color photo by a gain and compare three ways of handling the channels that overflow: clip each one, rescale the pixel, or tone-map it.

Open experimentClose experiment
Predict first

Multiply an orange pixel (R = 200, G = 100, B = 40) by 2 with per-channel clipping: what happens to its hue?

Loading image…

Original photo
Result, or hue map (red: hue moved, blue: color lost, gray: unchanged)
Hue of the overflowing colored pixels, before and after (10° bins)

Try this: Set the gain to 2 with per-channel clipping and look at the hue map: the wooden table and the coffee are red (hue moved) and the bright cup walls are blue (colour lost). Raise the gain to 4: why do the orange bars of the hue histogram pile up in one bin at 60° (yellow), and where do the other pixels go? Now switch to rescaling: what does not change, and what does the picture lose instead?

Photo: the coffee test image reduced to 256 × 171 (CC0, Rachel Michetti). The gain multiplies the stored 8-bit sRGB values directly, like np.clip(g * x, 0, 1) in the Images as Arrays chapter; a real exposure change acts on linear light, so the numbers would differ but the story would not. Overflow means some gained channel exceeds 255. Clip: each channel is cut at 255. Rescale: an overflowing pixel is divided by its largest channel, so the channel ratios, hence the hue, survive, but its largest channel always ends at 255. Tone map: a Reinhard-style operator L(1 + L/g²)/(1 + L) on the luma L of the gained pixel (white point g, the gain), applied as one common factor to R, G and B, followed by clipping of any channel still above 255. Hue is the HSV hue angle in degrees (red 0°, green 120°, blue 240°), close to but not identical with the chapter’s HSI hue. Pixels with HSV saturation or value below 0.1 have no usable hue and are left out of the hue statistics; a pixel that was coloured but ends below 0.1 saturation counts as colour lost. Saturation is HSV saturation, averaged over coloured pixels that overflow.

Method and image sources

IP 08 · How many coefficients can you throw away?

Transform a real photo with an 8 × 8 block DCT or a Haar wavelet, round every coefficient to a coarse step, and see where the lost information shows up.

Open experimentClose experiment
Predict first

You keep the default setting, where most coefficients are rounded to zero. Where in the picture will the reconstruction error be largest?

Loading image…

Original (yellow box = zoom)
Reconstruction from the quantized coefficients
Surviving coefficients, log scale, zeros black (DCT: one 8 × 8 tile per block, block average top-left; Haar: coarsest average top-left, finer details outward)
Error: reconstruction − original, × 10, gray = 0
Zoom (32 × 32 pixels, 6 ×): original
Zoom: reconstruction (DCT: the ruler marks the 8-pixel blocks)

Try this: Set the DCT step to 0.6 and look at the sky in the zoom box. Then switch to Haar at the same step: does the staircase go away, and which transform keeps fewer nonzero coefficients at about the same PSNR? Finally move the zoom box to the grass: how much of the texture is left?

A real 256 × 256 version of the camera test image (a 2 × 2 average of skimage.data.camera), gray values in [0, 1]. Both transforms are orthonormal: an 8 × 8 block DCT (each block coded on its own) and a 5-level 2-D Haar wavelet transform. The quantizer is uniform: every coefficient c becomes round(c / step) × step, with the same step for all coefficients including the block averages, so a coefficient survives only if |c| is at least step / 2. This is simpler than the frequency-dependent table of JPEG in IP 08, so the pictures show what the transform and the rounding do on their own. "Nonzero" counts surviving coefficients. It is not a bit rate: no entropy coder is implemented, and the real cost also depends on which coefficients survive and how they are coded. PSNR has peak 1 and uses the decoder output before clipping. Because both transforms are orthonormal, the squared error in pixels equals the squared error in the coefficients: what the quantizer removes is exactly the error, and the decoder cannot bring it back, because many different images give the same quantized coefficients. A bottleneck autoencoder (ML 06A) loses detail in the same way, except that its basis is learned from data instead of fixed. The error image shows (reconstruction − original) × 10 around mid-gray. "Smoothest" and "busiest" halves are the pixels with the smaller and larger Sobel gradient of the original. Haar is the simplest wavelet and is piecewise constant, so it also produces blocks, of several sizes; smoother wavelets such as the one in IP 08’s wavelet comparison blur instead.

Method and image sources

IP 09 · Which shapes does a structuring element keep?

Clean a thresholded photo of coins that carries specks, scratches and holes. Change the operation, the shape and the size of the structuring element, and see what each choice removes, fills or keeps.

Open experimentClose experiment
Predict first

Open the mask with a disk of radius 4. Every speck and scratch is thinner than the disk (at most 7 pixels wide), and every coin is far wider. What happens?

Loading image…

Input mask: coins with specks, scratches and holes
Output (black stays, red removed, blue added)
The structuring element (each square is one pixel; the orange dot is the origin)

Try this: With opening and a disk of size 4, all clutter goes. Now switch the element to a horizontal line of size 4: which pieces survive, and why? Then raise the disk size with opening until a coin disappears. Finally switch to closing: what does it fix that opening cannot?

The photograph is skimage’s coins image (303 × 384). The clean mask smooths it (Gaussian, σ = 2), subtracts a wide Gaussian background (σ = 50) so the bright corner does not count, and thresholds with Otsu’s method; that gives exactly 24 solid coins. Clutter is then added with a fixed seed so the answer is known: 40 specks (blobs of radius 0 to 3 pixels), 10 thin scratches (5 horizontal, 5 vertical) and 18 round holes of radius 1 to 3 pixels inside the coins (scripts/figures/playground_restore_assets.py). Notation follows the chapter: A is the foreground, B the structuring element, opening = erosion then dilation, closing = dilation then erosion, gradient = dilation minus erosion. Size r means a disk of radius r (all offsets with dx² + dy² ≤ r²), a square of (2r + 1) × (2r + 1) pixels, or a line of 2r + 1 pixels. Pixels outside the image count as background. Components are 8-connected; a hole is a background region that does not touch the border; clutter means a connected piece under 200 pixels (the smallest coin has more than a thousand). In the output picture, black pixels stay foreground, red were foreground and are gone, blue are new.

Method and image sources

IP 10 · Where does one global threshold fail?

Dim one side of a page and watch Otsu’s single threshold lose the text, while a threshold that follows the local light keeps it.

Open experimentClose experiment
Predict first

The lamp is much stronger on the left than on the right (ramp 0.5). Can one global threshold still separate ink from paper across the whole page?

Loading image…

The page under uneven light
True text mask (known exactly)
One global Otsu threshold
Local threshold
Histogram of the lit page and the Otsu threshold

Try this: Start with the lighting ramp at 0, then raise it slowly. At what ramp does the global mask start to lose text on the dim side? Then shrink the local window to 11: why does the local mask get worse?

The page is synthetic (scripts/figures/playground_images.py), so the true text mask is known exactly and IoU is a real score, not a proxy. Lighting multiplies the page by 1 − ramp · x/(w − 1). Global: Otsu’s threshold on the 256-bin histogram. Local: a pixel is ink if it is darker than (1 − c) times the Gaussian-weighted average of its window (σ = (window − 1)/6). IoU is the overlap of the ink mask with the true mask divided by their union.

Method and image sources

IP 11 · Does the snake find the edge, or get stuck?

Run a real snake, the Kass–Witkin–Terzopoulos contour of IP 11, on a photograph of coins: start a circle around one coin and see how far from its rim the circle may begin before the snake settles in the wrong place.

Open experimentClose experiment
Predict first

The start circle has radius 40 pixels; the coin has radius about 23. With the default settings, will the snake still find the coin?

Loading image…

Coins with the start circle (white dashed), snapshots at 10, 30, 100 and 300 iterations (orange), the final snake (yellow) and the true coin boundary (cyan dashed). Click or drag to set the start radius
Edge map e = |∇(G_σ ∗ image)| (dark blue: strong edge) with the start circle (dashed) and the final snake (orange)
Snake energy against iteration
Mean distance to the true coin boundary against iteration

Try this: The default circle has radius 40 and the coin about 23: the snake ends 17.7 px from the coin boundary, caught by the neighbouring coins. (1) Lower the start radius to 28: it finds the rim (0.5 px). (2) Back at radius 40, raise the smoothness to 1.0: now it finds the coin too (0.4 px). Why does a stiffer, more contracting snake reach the coin from farther away? (3) Keep smoothness 2.0 and start inside the coin at radius 12: the snake shrinks to a small curve (17.4 px). (4) With smoothness 0.05, a start at radius 12 reaches the rim at σ = 3 (1.1 px) but stays inside at σ = 2 (10.2 px). What does the blur remove?

Real image: a 144 × 144 crop of skimage.data.coins, centred on one coin and showing parts of the eight coins around it (credit below). The snake follows IP 11 and its figure code: a closed contour of 100 points, the semi-implicit step x(t) = (A + γI)⁻¹(γ x(t−1) + f) with A built from [β, −α−4β, 2α+6β, −α−4β, β], γ = 1, α = β = the smoothness slider, and the force f = κ∇e with κ = 4, where the edge map e = |∇(G_σ ∗ image)| is scaled to a maximum of 1 and sampled by bilinear interpolation. There is no resampling, no balloon force and no step cap; points stay inside the image. It stops when no point moves more than 0.05 px in one step, or after 1500 iterations. This is not scikit-image’s active_contour: that function builds the same matrix A, but takes a Sobel gradient of the image you pass in, a spline for the force and caps every step at one pixel. The true boundary is the circle fitted by least squares to the boundary pixels of the coin’s Otsu mask (holes filled); the coin is not a perfect circle, so a snake sitting on the rim is still about half a pixel from it. Distance is the mean over the 100 points of |distance from the circle’s centre − its radius|; “found” means at most 2 px. Energy is E = ½ Σ(α|v(i+1) − v(i)|² + β|v(i+1) − 2v(i) + v(i−1)|²) − κ Σ e(v(i)), in arbitrary units; the orange dashed line is E for the fitted circle itself. A click or drag on an image sets the start radius so that the circle passes through the pointer; the circle always stays centred on the coin. Click positions are rounded to whole pixels.

Method and image sources

IP 12 · Probe a pixel: flat, edge or corner?

Slide a small window over a real photograph and watch the window’s gradient vectors, its structure-tensor ellipse and the Harris response decide whether the spot is flat, an edge or a corner.

Open experimentClose experiment
Predict first

Three places are marked on the photograph: A, B and C. Which one is a corner, a point where shifting the window in any direction changes what it sees?

Loading image…

Photograph with the probe window (yellow) and marks A, B, C. Click or drag to move the probe
The window magnified, with a gradient arrow in each pixel (orange)
Harris response R over the whole image (red: corner, blue: edge, white: flat)
One dot per window pixel: its gradient (Ix, Iy). The orange ellipse is the structure tensor
The two eigenvalues of this window (dot) among all windows of the image (grey)

Try this: Place the probe on the three marked points with the sliders (A: x 200, y 20; B: x 80, y 52; C: x 30, y 80) and compare the dots: a tiny huddle at the origin, a cloud in two directions and a thin line. Then try the left tripod leg (x 122, y 232) and the right leg (x 183, y 220): the line turns, yet λ2/λ1 stays below 0.0025 for both, because both are edges. Finally raise the window from 9 to 25 pixels: the share of the image labelled corner grows from 1.4% to 8.9%. Why does a bigger window find more corners?

Real photograph: skimage.data.camera at 256 × 256 (see the credit below). Gradients are Gaussian derivatives with σ = 1 pixel, as in the IP 12 code. The window is W × W pixels with Gaussian weights of σ = W/6, so W = 9 gives the chapter’s integration scale σ = 1.5 (the chapter code truncates the Gaussian at 4σ instead of at the box). M is the weighted sum of the outer products of the gradient vectors, R = det M − 0.05 · tr(M)². Gradients are in gray levels per pixel (0 to 255) and eigenvalues in their squares. The flat, edge and corner labels use a cutoff chosen for this lab, not part of Harris and Stephens: an eigenvalue counts as large when it is at least 5% of the largest λ1 anywhere in the image at that window size. The Harris detector proper keeps local maxima of R above a threshold. The heat map shows R divided by its image maximum, compressed by a square root: red positive, blue negative, white near zero.

Method and image sources

Edit a filter kernel

Edit 3 × 3 weights and inspect smoothing, sharpening and edge responses.

Open experimentClose experiment
Kernel playgroundEdit any cell or pick a preset. The output updates as you type.
Input I
Output G = K ⋆ I

Try this: start with the identity kernel, then make the weights sum to zero. What happens in a flat region?

This computes cross-correlation with an unflipped kernel and replicated boundaries. Signed zero-sum responses use mid-gray as zero; values outside the display range are clipped.

IP 03 · Intensity Transformations and Spatial Filtering →ML 03 · Convolution, Image Filtering and CNNs →

Method source: Torralba, Isola & Freeman, Foundations of Computer Vision, MIT Press, 2024

Natural Language Processing

Count the next word

Build an n-gram distribution from a tiny editable corpus. Inspect counts, sentence boundaries and smoothing.

Open experimentClose experiment
Context count
3
Vocabulary size
8
Probability sum (all tokens)
1

Context: robots. P(w | h) = (count(h,w) + α) / (count(h) + α|V|).

Try this: Enter an unseen context and set α to zero. Why is the estimate undefined? Increase α and inspect where the probability goes.

Each nonempty line is one sentence. Tokens are lowercased and split on whitespace; punctuation stays attached. The first 8,000 characters are used. Vocabulary comes only from the corpus, plus [EOS] and [UNK]. [BOS] pads the start but is never predicted. The last n − 1 query tokens form the context; an empty query means sentence start. This is a counting demonstration, not a Chinese word segmenter.

Method sources

Build subwords one merge at a time

Train byte-pair encoding on a tiny weighted vocabulary and apply the learned merge order to a new word.

Open experimentClose experiment
Learned merges
4
Training pieces, weighted
31
Input pieces
5

Merge counts come only from the fixed training vocabulary. Encoding applies the learned rules in order; it does not refit them to your input.

Try this: Compare “lower”, “slowest” and “glow”. Which pieces transfer to unseen words, and which merges only compress the training vocabulary?

Character-level BPE with an explicit end marker ▁; lexicographic tie breaking. This is not a byte tokenizer or a pretrained vocabulary. New characters remain individual pieces; spaces inside the input are treated as characters.

Method sources

Does “everyone read a book” mean the same book?

Change a small reading relation and evaluate two quantifier orders exactly.

Open experimentClose experiment
Each person: some book
True
One book: everyone
False

∀ person ∃ book allows a different book per person. ∃ book ∀ person requires one shared book; changing quantifier order changes the claim.

Try this: Make “each person read some book” true while “one book was read by everyone” is false. Can the reverse happen?

This is a finite truth-condition model, not an automatic interpretation of an ambiguous sentence. A shared book satisfies both readings.

Method sources

Does padding change a mean-pooled sentence?

Pad a three-word sentence with zero vectors and compare a correct masked mean with a mean that counts the padding in its denominator.

Open experimentClose experiment
Predict first

Padding bug selected, with 9 zero-padding tokens added to the 3 real words. Compared with the correct masked mean, the pooled value h becomes

Pooled h
0.233
Masked-mean h (reference)
0.7
P(class 1)
0.763
Reference P(class 1)
0.971
Real / counted length
3 / 9

Zero padding must not change a masked mean. Counting the padding in the denominator shrinks h by 3 divided by the counted length and pulls P(class 1) toward 0.5, although the sentence is the same. An unweighted mean also ignores word order; only the position-weighted diagnostic tells the two sentences apart.

Try this: Keep the padding bug selected and drag the padding count from 0 to 9: predict h and P(class 1) before each step, and explain why the classifier drifts toward 0.5 although the sentence never changed. Then switch to the masked mean and confirm that padding cannot matter. Last, try position weights on the reversed sentence: that is the only mode where word order changes the answer.

Fixed scalar embeddings: dogs=1, chase=0.4, cats=0.7; classifier p=σ(5h). The masked mean divides by the 3 real tokens, so zero padding adds nothing. The bug divides by 3 + padding. The reference bar always shows the masked mean of the same sentence. A plain mean ignores word order by construction, so swapping the sentence changes nothing there; position weights 1,2,3 are a diagnostic, not a trained syntax model.

Method sources

Local tag choices versus a coherent sequence

Combine emission scores, transition scores and BIO span constraints to choose a complete tag path.

Open experimentClose experiment
Best total score
7.3
Local path score
6.7
Local path legal?
Yes

The highlighted cells maximize the total path score, not each token separately. An I tag can only continue a B or I when constraints are enforced.

Try this: Raise the score for I at “York”. Can a transition preference change the best path even when the local winner stays fixed?

O=outside, B=begin, I=inside one entity type. Scores are synthetic, not probabilities. All 81 paths are enumerated exactly; dynamic programming would find the same maximum more efficiently on long sequences.

Method sources

Temperature, top-k and top-p change the distribution

Follow fixed logits through temperature scaling, top-k truncation, nucleus truncation and renormalization.

Open experimentClose experiment
Predict first

Hold the temperature at 1 and lower top-p slowly from 1.0. How does the output entropy change?

Candidates retained
3
Entropy, bits
1.201
Probability sum
1

Temperature rescales relative logits. Truncation sets excluded candidates to zero and renormalizes the survivors; it cannot improve the information available to the model.

Try this: Keep temperature fixed and lower top-p. Why can the output entropy change abruptly even though the slider moves continuously?

Five synthetic candidate tokens with logits [3,2,1,0,−1]. This applies top-k first, then top-p to its renormalized distribution. No model is trained and no quality score is inferred from entropy.

Method sources

Why does this document rank first?

Inspect BM25 term saturation and document-length normalization on a fully visible local corpus.

Open experimentClose experiment
Top document
D2
Top score
1.86
Matching documents
4

IDF rewards terms found in fewer documents. k₁ controls how repeated matches saturate; b controls the penalty for length relative to this corpus mean.

Try this: Search “red”, then “robots objects”. Change length normalization and saturation; explain each rank swap from document terms.

Six synthetic documents, lowercase English word tokenization, distinct query terms, positive smoothed IDF. A lexical relevance score is not evidence that a document answers a question.

Method sources

What may a masked predictor see?

Change a right-context clue, hide it causally, then attach different output heads to the same small contextual vector.

Open experimentClose experiment
Context positions used
3
First output probability
0.931
Output probabilities sum
1

The target content is always absent. A bidirectional objective can use right context; a causal next-token objective cannot. An output head maps representations to the labels required by a task.

Try this: Change the right-hand clue under each mask. When must the masked-position prediction remain identical, and why?

A hand-built uniform context encoder and two linear heads illustrate BERT-style visibility and task interfaces. These are not BERT embeddings or pretrained predictions. The sequence head reads this toy pooled context, not an actual [CLS] representation.

Method sources

Pattern Recognition

PR 04 · Does changing a feature’s unit change its neighbours?

Run leave-one-out k-nearest-neighbour classification on the real iris measurements, then change the unit of one feature: no flower moves, yet the neighbours and the accuracy do.

Open experimentClose experiment
Predict first

Every number in one feature column is multiplied by 10, for example decimetres to centimetres or centimetres to millimetres. The flowers do not move and the column can be divided back by 10. What happens to a k-nearest-neighbour classifier?

Loading image…

k-NN decision map on the iris flowers (axes in the original centimetres; ring = misclassified in leave-one-out)
Leave-one-out accuracy as the unit of the second feature changes (the data are the same in every column)

Try this: Start with sepal width and petal width, 5 neighbours, petal width in decimetres (×0.1). Step the unit to centimetres (×1) and millimetres (×10): how do the accuracy and the number of changed nearest neighbours move? Then switch to petal length and petal width. Is the effect as large? Finally compare with the standardized setting.

Data: the 150 iris flowers bundled with scikit-learn (three species, 50 each), two of the four measurements at a time, in centimetres. The classifier is the plain k-NN rule with Euclidean distance. Only the second feature is rescaled (the first stays in centimetres); "standardized" divides both features by their standard deviation. Accuracy is leave-one-out: each flower is classified by the other 149, and the map is trained on all 150. The measurements are rounded to 0.1 cm, so many distances tie exactly, and the fifth and sixth neighbour of a flower are often equally far away. Ties are settled in a fixed way: equal distances are ordered by a fixed random ranking of the flowers (the file lists the species one after another, so ordering by position would favour setosa), and a tie between classes goes to the tied class with the nearest member. The last metric counts the answers that rest on such a tie. scikit-learn orders equal distances in its own way: on the 108 settings checked in the tests (3 feature pairs, 6 units, 6 values of k) it agrees with this lab on every answer that does not rest on a tie, and its leave-one-out count differs from this one by at most 7 flowers. Rescaling is reversible, so it adds no information; the accuracy is a single 150-flower sample, not a general result.

Method and image sources

When should a classifier change its mind?

Move a decision boundary by changing class frequency and the cost of a false alarm.

Open experimentClose experiment
Decision boundary x
0
Posterior P(1 | x)
0.731
Required posterior for class 1
0.5

At x = 0.5, choose class 1. Compare conditional risks: choosing 0 costs 0.731; choosing 1 costs 0.269. The boundary obeys 2x + log(prior odds) = log(false-positive cost).

Try this: Keep the observation fixed. Can increasing the false-positive cost switch the decision without changing the posterior?

A synthetic population: X|class 0 ~ N(−1,1), X|class 1 ~ N(1,1). The slider sets the true class prior. A false negative costs 1; correct decisions cost 0. Curves show class-conditional densities, not posterior probabilities.

Method sources

PR 02 · How wide should a density kernel be?

Change the bandwidth and inspect individual kernels, the estimated density and leave-one-out prediction.

Open experimentClose experiment
Predict first

Which bandwidth gives the best held-out log-likelihood: h = 0.1, 0.5 or 2?

Mean leave-one-out log density
-1.827
Peak density
0.283

Small bandwidth preserves local bumps; large bandwidth merges them. Leave-one-out evaluation prevents each sample from predicting itself.

Try this: Find a bandwidth that improves the leave-one-out log score. Why can a spiky fit score worse on a held-out point?

Ten fixed synthetic observations; Gaussian kernels integrate to one. The score averages log density at each point after excluding that point. It is a density score in nats, not classification accuracy. The plot shows individual contributions divided by ten.

Method sources

PR 03 · Complexity versus shrinkage

Fit polynomial basis functions with ridge regression and separate training error from recovery of the underlying function.

Open experimentClose experiment
Predict first

Set λ = 0 and raise the degree from 5 to 6. Training error falls (0.0285 to 0.0239). What happens to the error against the true function?

Training mean squared error
0.0538
True-function grid error
0.029
Squared coefficient norm
5.403

Changing λ alters the fitted coefficients, not the observations. Large penalties can remove noise-fitting but also suppress the curvature needed to represent the signal.

Try this: At degree seven, increase regularization. Does the lowest training error also give the lowest underlying-function error?

The blue fit solves sum of squared residuals + λ times the squared non-intercept coefficients. Twelve fixed synthetic points have deterministic perturbations. The dashed curve is sin(πx); its grid error is available only because this is a known synthetic problem. Polynomial features are linear in their coefficients.

Method sources

PR 05 · Does maximum variance preserve the classes?

Rotate a one-dimensional projection and compare reconstruction error with class separation.

Open experimentClose experiment
Retained variance fraction
0.662
Mean squared reconstruction distance
1.917
Projected class-centroid gap
1

At 0°, both classes project to the same six positions. At 90°, within-class horizontal variation disappears and class centers remain distinct.

Try this: Find the projection that minimizes reconstruction error, then the one that separates the colored classes. Why are they different?

The centered synthetic observations have horizontal variation within each class and a vertical class difference. Principal component analysis maximizes retained variance without using labels. The centroid gap shown here is descriptive, not a complete classification metric.

Method sources

PR 06 · Match shape despite timing changes

Warp a signal in time and inspect a dynamic time warping cost matrix and its optimal alignment path.

Open experimentClose experiment
Aligned total cost
0.231
Same-index total cost
3.591
Path cells
14

Dark cells have larger amplitude mismatch; gray cells are forbidden by the band. The path minimizes accumulated allowed cost. A wider band cannot raise the optimum but can permit implausible repetitions.

Try this: Increase the time warp with a zero-width band, then widen the band. What does the lower cost allow the alignment to ignore?

Two twelve-sample sine signals: a(t)=sin(2πt), b(t)=sin(2πt^w). Local cost is squared amplitude difference. The path must start and finish at both endpoints; only diagonal, vertical and horizontal forward steps are allowed. Cost is a sum, not normalized by path length.

Method sources

PR 07 · Watch soft assignments move the means

Step through Gaussian-mixture EM and observe responsibilities, likelihood and the symmetric-initialization trap.

Open experimentClose experiment
Component means
-1.857 / 1.537
Log likelihood (nats)
-15.863
Gain since initialization
10.555

The E step recomputes soft memberships; the M step takes responsibility-weighted means. With identical initial means every responsibility is 1/2, so symmetry cannot break on its own.

Try this: Set initial separation to zero and add iterations. Why do the two means remain identical? Restart with a small separation.

A constrained two-component mixture: both weights stay 1/2 and both standard deviations stay at the selected σ. EM updates only the means. Each point is colored by its current responsibility for component 1; these are component memberships, not supervised labels.

Method sources

PR 08 · Which direction of KL are you minimizing?

Move and widen a Gaussian approximation to a two-mode target; compare penalties for missing mass and filling the valley.

Open experimentClose experiment
Predict first

The target has two separated bumps and q is a single Gaussian. Which divergence is smallest for a q that sits on just one bump?

KL(p ‖ q)
8.777
KL(q ‖ p)
0.69

KL(p‖q) weights mismatch where the target has mass; KL(q‖p) weights mismatch where the approximation puts mass. A single Gaussian may cover both modes or settle on one, depending on the objective.

Try this: With separated modes, compare a narrow approximation on one mode against a wide one covering both. Which KL prefers each?

Target p is an equal mixture N(−a,0.65²) and N(a,0.65²). Approximation q is N(μ,σ²). Both are numerically normalized on [−12,12] with spacing 0.02; KL values are grid approximations in nats. This is an adjustable distribution family, not an optimizer.

Method sources

PR 09 · Does averaging many unstable trees beat one tree?

Grow real decision trees on 300 noisy two-moons points, bootstrap them, let them vote, and watch the test error fall and then flatten as trees are added.

Open experimentClose experiment
Predict first

Fully grown trees (depth 14) are trained on 300 noisy moon points. Fifty more copies are trained on bootstrap resamples of the same points and vote on 1000 fresh test points. Compared with one fully grown tree trained on all 300 points, the vote will…

Loading image…

Decision map of the chosen view over the 300 training points (circle = class 0, square = class 1; paler colour = the trees disagree; hollow points = missing from that member’s bootstrap sample)
Test error of the bagged vote for every M from 1 to 100 (1000 held-out points); the ring marks the current M

Try this: Start with 50 trees at the fully grown depth 14 and drag M from 1 to 100: the vote falls from 14.60% to 11.10% over the first ten trees, then barely moves (10.25% at both 50 and 100). Switch the view to One member and drag M to see how ragged single bootstrap trees are, then back to the vote. Lower the depth to 2: the members now disagree on only 4.88% of the test points per pair, and the vote (10.70%) is no better than one tree (10.50%). Finally try depth 5: one tree already scores 9.30% against 10.15% for 50 bagged trees, a gap of less than one percentage point, about the noise of a 1000-point test set, so bagging does not replace choosing a sensible depth. Why does a vote of similar trees gain nothing?

Data: the two-moons set from scikit-learn make_moons with Gaussian noise 0.3, generated once and stored in the page: 300 training points (150 per class, data seed 2) and a separate held-out test set of 1000 points (seed 1002). The data seed is the one of 30 random draws whose bagging gain is closest to the average gain, so this is a typical case, not a favourable one. Trees: CART with Gini impurity and axis-aligned splits at the midpoint between neighbouring values; the depth limit comes from the slider (14 grows every tree here until its leaves are pure). Bagging: member b is trained on 300 rows drawn with replacement from one fixed seeded stream (seed 20260618), so moving M only adds or removes trees and the curve does not change. The vote is a plain majority; for an even M an exact tie counts as half an error. Test error is measured on the 1000 held-out points; its standard error is about 1 percentage point at an error of 10%, so differences smaller than that between two different settings are weak evidence on their own. Disagreement: for every pair of the M members, the share of the 1000 test points on which the two predict different classes, averaged over all pairs (0 means identical members; the closer to 0, the more alike the members, which is a proxy for how correlated their mistakes are). Ties between equally good splits are broken by a fixed rule (feature 1 before feature 2, lower threshold first), while scikit-learn visits the features in random order: in the tests, trees without such a tie give predictions identical to DecisionTreeClassifier, and the vote error of the full 100-tree ensembles differs from the same ensembles built with scikit-learn trees by at most 0.7 percentage points. Best possible 8.5% is the error of the ideal classifier on the distribution that make_moons samples, computed by numerical integration of the two class densities; the lab does not use it. Real bagging also gives each tree out-of-bag points for free error estimates; this lab uses the separate test set instead.

Method and image sources

PR 10 · Does confidence survive a shifted population?

Set a rejection threshold, shift both class distributions and compare accepted coverage with selective error.

Open experimentClose experiment
Accepted coverage
0.666
Error among accepted predictions
0.068
Deployment-optimal boundary
0

The classifier still places its decision boundary at zero, while the deployment-optimal boundary moves with δ. Rejecting uncertain predictions does not repair confidence under shift.

Try this: Hold the rejection threshold fixed while shifting the population. Can high reported confidence coexist with more accepted mistakes?

The fixed classifier uses training posterior sigmoid(2x), learned for equal-prior N(−1,1) and N(1,1). Deployment shifts both means by δ without updating the classifier. Coverage and accepted error are numerical integrals over the synthetic deployment population, not estimates from a benchmark.

Method sources

Machine Learning

Two hidden units can bend a decision region

Compare a fixed two-unit network with and without nonlinear activations on the XOR corners.

Open experimentClose experiment
Predict first

With the linear activation selected, can some output bias classify all four XOR corners correctly?

Correct corners
3 / 4
P(1 | 0,1)
0.881
P(1 | 1,1)
0.119

A linear hidden layer collapses to one affine decision boundary. Nonlinear hidden units create pieces that can assign the two opposite XOR corners to the same class.

Try this: Start with the linear activation and try every bias: can all four corners be right at once? Then switch to ReLU and tune the gain and bias. Why does the nonlinearity change the answer?

Weights are hand constructed, not trained: h₁=a(g(x+y)), h₂=a(g(x+y−1)), p=σ(4(h₁−2h₂)/g−2+b). Color is P(class 1); circles show known corner labels.

Method sources

Alignment, hard negatives and collapsed features

Move paired and unpaired unit vectors, then inspect the normalized matching loss used by contrastive learning.

Open experimentClose experiment
Positive probability
0.896
Matching loss, nats
0.109
Positive cosine
0.94

Loss = −log(exp(sim⁺/τ) / Σ exp(sim/τ)). Temperature sharpens whichever candidate already has the highest similarity. Collapsed unit vectors give equal probabilities and loss log(4).

Try this: Bring a negative closer to the anchor than the positive. Does a lower temperature help the correct pair? Then collapse all vectors.

One anchor, one positive and three negative candidates; vectors are controlled directly rather than trained. The same matching geometry explains image–text retrieval, but these are not CLIP features or semantic predictions.

Method sources

Selective state updates: remember the useful inputs

Compare a fixed recurrent decay with input-dependent write decisions while distractors pass through a sequence.

Open experimentClose experiment
Predict first

Policy = skip marked distractors, and you set retention a to 1 at the write positions (1 and 9). What is the state after the last position?

State h₁ → h₈ (before the second write)
0.75 → 0.75
Final state
-0.563
∂h₁₂ / ∂h₁
0.25
Largest |state|
0.75
Write opportunities
2

Fixed decay trades persistence against interference at every step. Selection can preserve state through irrelevant inputs, but the gate must identify relevance correctly; the oracle here removes that learning problem. Retention 1 preserves perfectly and writes nothing.

Try this: Compare the two policies: with selective updates the state written at position 1 survives the distractors, with the same update at every step the distractors overwrite it. Then raise the distractor amplitude and see which state changes. Finally set retention to one: a state that never changes keeps its old value perfectly, so nothing new can be written either. Why is that not memory?

Scalar recurrence hₜ=aₜhₜ₋₁+(1−aₜ)xₜ with h₀=0. Retention aₜ=1 keeps the old state, aₜ=0 replaces it with the input. Relevant positions 1 and 9 are provided by an oracle; in the selective policy every other position has aₜ=1. With retention 1 at the relevant positions the state stays at its starting value 0: perfect preservation, nothing written. This illustrates gating and Mamba’s input-selectivity motivation; it is not an implementation of Mamba, LSTM or GRU.

Method sources

Attention: which tokens are visible?

Compare bidirectional and causal attention using the same query, keys and values.

Open experimentClose experiment
Visible tokens
3
Weight sum
1
Attention output
[0.7517, 0.7517]

The score column includes temperature scaling. A masked score is −∞ and its weight is exactly zero. Changing the mask changes normalization over the remaining keys.

Try this: Choose the first query in causal mode: its weight must be 1 on itself. Switch to bidirectional mode, then lower the temperature.

All Q, K and V vectors are fixed, synthetic two-dimensional numbers, not learned language representations. Scores are Q · K / (√2 τ); τ is an extra temperature control and τ = 1 gives the usual scaling. Causal attention masks positions after the query before softmax. The output is the weighted sum of V. These weights illustrate routing, not an explanation of linguistic meaning.

Method sources

Same predictions, different probability quality

Apply one positive temperature to fixed validation logits and compare accuracy, log loss and squared probability error.

Open experimentClose experiment
Predict first

Raise the temperature T from 1 to 4. Does the classifier's accuracy change?

Accuracy
75%
Mean log loss, nats
0.498
Brier score
0.177

Positive temperature preserves logit ordering and binary decisions. It can adjust overconfidence or underconfidence, but cannot correct which examples the classifier gets wrong.

Try this: Find a temperature with lower log loss than T=1. Why does accuracy stay fixed even while confidence changes?

Eight synthetic validation observations, not a benchmark. Binary probabilities use σ(logit/T), with a fixed 0.5 decision threshold. Choosing T here illustrates validation tuning; it does not establish performance on an unseen test set.

Method sources

Why a smaller finite-difference step can be worse

Check the derivative of exp(x), separating truncation error from floating-point cancellation.

Open experimentClose experiment
Analytic derivative
2.71828183
Numerical derivative
2.71828183
Relative error
2.155e-11

Large h introduces approximation error. Very small h subtracts nearly identical floating-point values, losing useful digits. Central differences cancel the leading truncation term but still suffer roundoff.

Try this: Sweep h from 10⁻¹ to 10⁻¹⁶. Find the useful range for forward and central differences rather than choosing the smallest step.

Computed with JavaScript binary64 arithmetic; exact means the analytic derivative exp(x) evaluated in the same arithmetic. A zero measured error is rounded agreement, not proof of exact real arithmetic.

Method sources

ML 02 · Can a tiny network learn a curved boundary?

Train a network with 2 inputs, m tanh hidden units and 1 sigmoid output on the two moons and the two circles, one gradient-descent step at a time. Watch the decision map and the training and test loss: too few hidden units, too small a learning rate and too large a learning rate each fail in their own way.

Open experimentClose experiment
Predict first

Set the hidden units to 1 and leave everything else as it is. Can the network separate the two moons?

Loading image…

Two moons: decision map at the chosen step (blue circles = class 0, orange triangles = class 1; filled = training, hollow = test; ringed = wrong; black line = 0.5 boundary)
Two moons: cross-entropy loss against the step (solid = training, dashed = test)
Two circles: decision map at the chosen step (same symbols)
Two circles: cross-entropy loss against the step (solid = training, dashed = test)

Try this: Start from the default (m = 3, η = 1, 1000 steps), where both curved boundaries are learned. (1) Drag the steps slider up from 0: after 100 steps the moons are only 85.0% right (close to one tilted straight line, the loss on a plateau near 0.30), after 300 steps 94.5%, after 500 steps 98.0%. (2) Set η to 0.3 and lower m to 2, then 1: at 1000 steps the moons stay at 86.0% and 85.0%, the circles at 77.5% and 61.5%. (3) Go back to m = 3 and try η = 0.03, 0.1 and 0.3: at 1000 steps the moons are 84.5%, 85.0% and 94.5% right. A small η is not wrong, only slow: run offline for 5000 steps, beyond the 2000-step limit of this lab, η = 0.1 reaches 98.0% on the moons. (4) Try η = 3, 10 and 30: the loss now goes up on many steps, and with the largest η the training does not settle.

Data: scikit-learn make_moons (noise 0.2, random_state 0) and make_circles (noise 0.1, factor 0.5, random_state 0), 200 training points each, plus a held-out test set of 200 points each (random_state 1) that is never used for training. Network and loss as in ML 02: z₁ = W₁x + b₁, a₁ = tanh(z₁), z₂ = W₂a₁ + b₂, p = σ(z₂), binary cross-entropy averaged over the training points; the gradient comes from backpropagation (checked against finite differences and against an independent NumPy run). Training is plain full-batch gradient descent, θ ← θ − η∇L, with a fixed η: no momentum, mini-batches or regularization. Initial weights: Xavier normal (variance 2 / (fan-in + fan-out)) with zero biases, from one fixed seed (5), so a slider never re-rolls them; unit j always gets the same random numbers. Another seed changes the details: over seeds 1 to 20 (η = 1, 2000 steps), 3 hidden units reached at least 95% training accuracy on both datasets in all 20 runs, 1 unit never did, and 2 units reached it on the moons in at most 1 of the 20 runs and on the circles never. A step is one parameter update; step 0 is the untrained network. When the loss goes up and down the iteration is chaotic, so the curves at large η can differ in their details between browsers, though not in their story. The test-loss curve is sampled every 4 steps.

Method and image sources

A VAE trades reconstruction against prior agreement

Move and widen a Gaussian posterior, then compute expected decoder loss and the exact KL penalty.

Open experimentClose experiment
Expected decoder NLL
0.806
KL(q || prior), nats
0.638
NLL + β KL
1.444

Reconstruction favors latent values near x=1 and a narrow posterior. KL penalizes both moving away from zero and changing variance away from one. Prior agreement alone does not guarantee useful reconstruction.

Try this: Set q(z|x) equal to the prior. Why is KL zero but reconstruction still poor for x=1? Can shrinking σ improve one term while worsening the other?

One observation x=1, q=N(μ,σ²), prior=N(0,1), fixed decoder p(x|z)=N(z,0.5²). Expected negative log likelihood is analytic; no samples or network training. β=1 gives the negative evidence lower bound for this model; other β values are weighted objectives.

Method sources

Stretch space, change density: the Jacobian matters

Push a Gaussian through an invertible nonlinear map and preserve probability mass with its derivative.

Open experimentClose experiment
Derivative at zero
2
Output density at zero
0.199
Numerical area (±4 base σ)
0.99988

pₓ(f(z)) = p_z(z) / |f′(z)|. A local stretch spreads the same probability over more output space. Omitting the Jacobian would change the total probability mass.

Try this: Increase the nonlinear stretch near zero. Why does the output density at zero fall even though zero still maps to zero?

One-dimensional normalizing flow x=az+b tanh(z), with a>0 and b≥0. The derivative is always positive, so the map is invertible. Curves are computed from the exact change-of-variables formula, without fitting a model.

Method sources

A correct velocity field still needs a numerical solver

Transport particles through a known velocity field and compare integration error at equal step counts.

Open experimentClose experiment
Mean endpoint error
0.189926
Field evaluations / particle
4
Exact endpoint from x₀=0
1.532

Step count and function evaluations are different costs. Reducing integration error cannot fix an incorrect learned field; this oracle example keeps field error at zero so the two issues stay separate.

Try this: Set the rate to zero: why is one Euler step exact? Then use a nonzero rate and compare steps with actual field evaluations.

Known field v(x,t)=λx+1, integrated from t=0 to 1. This isolates solver error relevant to flow-matching sampling; no velocity network is trained and no generation quality is measured. The dashed paths use the analytic solution.

Method sources

ML 08 · What does a network see when it predicts noise, the image or a velocity?

Mix the astronaut photograph with fixed noise, look at the noise, clean-image and velocity targets side by side, then give the network a small error and see which conversion back to the clean image survives.

Open experimentClose experiment
Predict first

The picture x_t is almost pure noise (−20 dB). A network makes the same small error, 0.05, whichever target it predicts. After converting each prediction back to a clean image x̂₀, which one is the most error-sensitive?

Loading image…

x_t, the network input: the only picture it gets to see
ε, the noise that was added
x₀, the clean image
v = √ᾱ·ε − √(1−ᾱ)·x₀, the velocity
x̂₀ recovered from the network’s imperfect output of the chosen target
Error x̂₀ − x₀ (gain ×3, mid-gray = 0)

Try this: Start at −20 dB with an error of 0.05 in the noise output. Slide the signal level up to 0 dB: where do the three recoveries become equally good? Then go to +20 dB: which target is now the most error-sensitive, and why does the answer flip?

Image: the astronaut photograph (256 × 256, RGB), with pixel values mapped to [-1, 1] as in the chapter. The mix is x_t = √ᾱ x₀ + √(1−ᾱ) ε with a fixed seeded standard-normal ε, and v = √ᾱ ε − √(1−ᾱ) x₀ is the velocity of the ML 08 chapter (the diffusion velocity, not the flow-matching transport velocity). The slider sets the signal-to-noise ratio ᾱ/(1−ᾱ) of x_t in dB. The network is not trained: its output is the exact target plus a fixed seeded error, and the same error size (in the units of the [-1, 1] pixel scale) is applied whichever target is predicted. A real network does not make equal errors for every target, so this isolates the conversion, not the learning. x̂₀ is clipped to [-1, 1] as an image decoder would, and PSNR is measured against x₀ with both mapped to [0, 1] (peak 1). All panels share one gray mapping (gray = 0.5 + u/2.5).

Method and image sources

Factor a tiny image into conditional predictions

Enumerate all sixteen binary 2×2 images under a first-order pixel model, then change the generation order.

Open experimentClose experiment
Image probability
0.064
Bits per pixel
0.991
All-image probability sum
1

Joint probability multiplies normalized conditional factors. All sixteen image probabilities sum to one. A high-copy model prefers sequences with fewer transitions, which need not correspond to the same spatial pattern in each order.

Try this: Compare row stripes and a checkerboard when copying is likely. Switch raster to column order: which model assumptions changed?

First pixel has probability 1/2; each later pixel copies the previous one with probability c. The chain rule is exact, but this model deliberately retains only one previous pixel. Order dependence here comes from that restricted conditional model.

Method sources

Deep Learning for Computer Vision

DLCV 01 · How much of the image can one output unit see?

Pick a stack of convolution layers and a spot in a real photograph: the exact box of input pixels that one output unit depends on is drawn on the image, with its size, jump and the share of the image it covers.

Open experimentClose experiment
Predict first

Three stacked 3×3 layers give each output unit a 7 × 7 pixel receptive field. Which single change makes it grow the most?

Loading image…

Photograph with the receptive field of one output unit (yellow box). Click or drag to move the unit
Everything this unit can see, enlarged
The stack, layer by layer: r is the receptive field and j the jump after that layer; bars are traced exactly

Try this: Choose “Six 3×3 layers, stride 2 in every second” and click the image: the unit sees 29 × 29 pixels, 1.3% of the picture, and its neighbors are 8 pixels apart. Now choose “Four 3×3 layers, dilations 1, 2, 4, 8”: a similar field, 31 × 31 pixels (1.5%), but the jump stays 1, so every pixel keeps its own output unit. Finally click the top-left corner: why is the covered share smaller there than in the middle of the photograph?

Real photograph: skimage.data.camera at 256 × 256 (credit below). Each layer is a square convolution with kernel k, stride s and dilation d, padded with zeros by (k_eff − 1)/2 on each side, where the effective kernel is k_eff = (k − 1)·d + 1. The size and jump come from the recurrence r ← r + (k_eff − 1)·j and j ← j·s (r = j = 1 at the input), and the box on the image from tracing the index interval of one output unit back through every layer: both agree for every stack here, and an independent SciPy computation of which input pixels influence one unit gives the same boxes. The unit snaps to the nearest position on the output grid, whose spacing is the jump. Where the box leaves the image, the part outside is zero padding and the covered share counts only the part inside. This is the theoretical receptive field: the set of pixels that can change the unit. Measured influence is not uniform; it is concentrated near the center of the box, so the effective receptive field is smaller. That is not computed here, and no weights are trained.

Method and image sources

DLCV 02 · A decision threshold changes which mistakes you make

Move the decision threshold on fixed classifier scores. The confusion matrix changes, and the operating point slides along the ROC and precision-recall curves built from the same scores.

Open experimentClose experiment
Predict first

The threshold is 0.5. You lower it to 0.3. Where does the operating point move on the ROC curve?

Precision
0.66667
Recall
0.8
ROC AUC (all thresholds)
0.84
F1
0.72727
Accuracy
0.7
Mean error cost
0.3

True positives (TP): 4; false positives (FP): 2; false negatives (FN): 1; true negatives (TN): 3. Lowering the threshold can recover missed positives while admitting false positives. Precision and recall use different denominators; accuracy alone cannot express unequal error costs.

Try this: Find a threshold with no false positives and note where the dot sits on the ROC curve. How many positives are missed? Then raise the false-positive cost c: the curves do not move, but which point has the lowest mean error cost does.

Ten fixed synthetic scores with five positive labels. Predict class 1 when score ≥ threshold; no model is trained. Each vertex of the curves is one threshold (one of the ten scores); lines between vertices are only drawn to guide the eye. ROC: x is FP/(FP+TN), y is recall TP/(TP+FN); AUC is the trapezoid area. Precision-recall: precision is undefined if nothing is predicted positive, so no dot is drawn. Mean error cost = (c×FP+FN)/10, with false-negative cost fixed at 1. Adjusting c changes evaluation, not scores, decisions or curves.

Method sources

Boxes, overlap and suppression

When do overlapping detections represent one object? Inspect how greedy suppression makes that decision.

Open experimentClose experiment
Predict first

At the default positions, boxes A (score 0.92) and B (score 0.84) overlap with IoU about 0.55 and the IoU threshold is 0.5. How many of the three boxes survive?

IoU(A, B)
0.551
IoU(B, C)
0
Kept boxes
2

IoU = intersection area / union area. A suppressed box never suppresses another box; only previously kept boxes participate. A: kept · B: suppressed · C: kept.

Try this: Move B onto A, then raise the IoU threshold. Find a setting that keeps both. Can the rule tell duplicates from two nearby objects?

Three fixed-score boxes of the same class. First keep scores ≥ the score threshold; then visit boxes by descending score and suppress any box whose intersection-over-union with a kept box is strictly greater than the IoU threshold. Scores are toy inputs, not model accuracy.

Method sources

DLCV 03B · Is a small yaw error harmless?

Turn a predicted box a few degrees away from the ground truth in the bird’s-eye view and watch the exact rotated IoU fall while the centre distance stays at zero.

Open experimentClose experiment
Predict first

A detector predicts a bus-sized box (12 × 2.5 m) with exactly the right centre and size, but its heading is 10° off. What is the bird’s-eye-view IoU with the ground truth?

Loading image…

Bird’s-eye view: ground-truth box, predicted box and their exact overlap
Rotated IoU against yaw error for the three shapes, at the chosen centre offset

Try this: At the default setting (bus or truck, 12 × 2.5 m, yaw error 10°, centres identical) the rotated IoU is 0.657 although the centre distance is 0 m; the enclosing rectangles, which ignore the yaw, give 0.539. Lower the yaw error to 5° and the IoU is still only 0.808. Switch to the car (4.5 × 1.8 m) and the same 10° costs less, 0.796, then to the pedestrian (0.7 × 0.7 m): 0.863 at 10°, and never below 0.707 at any yaw error. The last row of the table gives the yaw error at which each shape falls below IoU 0.7: 8.5° for the bus and 16.55° for the car. Which class would a matching rule that uses an IoU threshold of 0.7 punish most for a heading mistake, and what does a rule that only looks at centre distance say about all three?

The ground-truth box is axis-aligned at the origin with its length along x. The prediction has exactly the right size and is turned by the yaw error about its own centre, then moved along the ground truth’s length by the centre offset. Sizes are illustrative, the ones used in the chapter’s Figures 4 and 6 (pedestrian 0.7 × 0.7 m, car 4.5 × 1.8 m, bus or truck 12 × 2.5 m), not measurements of any dataset. Rotated IoU is exact: the overlap polygon comes from Sutherland–Hodgman clipping and its area from the shoelace formula, and IoU = overlap / (area of the ground truth + area of the prediction − overlap); the code is the chapter’s Python snippet ported to TypeScript and tested against it. The axis-aligned IoU takes the smallest axis-aligned rectangle around each box and computes the ordinary IoU of those two rectangles; it ignores the real shape, so it is lower than the rotated IoU for small yaw errors (the enclosing rectangle of a turned box is larger than the box) and higher for large ones. The centre distance is the Euclidean distance between the two centres on the ground plane. IoU cannot see a 180° flip of the heading, so the yaw error is limited to 0° to 90°. Only the bird’s-eye view is compared here (no height). The nuScenes benchmark matches a detection to an object by the 2D centre distance on the ground plane, to decouple detection from object size and orientation, and scores heading in a separate error term; KITTI’s car threshold of 0.7 is a 3D box overlap.

Method and image sources

DLCV 04 · Does high pixel accuracy mean a good segmentation?

Simulate a segmentation network that predicts at a coarse output stride and compare pixel accuracy, mean IoU and the IoU of the smallest class.

Open experimentClose experiment
Predict first

A network predicts at output stride 8, so one score cell covers 8 × 8 pixels. Its cell scores are the best possible (the exact class fractions). The thin pole is 3 pixels wide and the small squares are 6 × 6. What do the scores look like?

Loading image…

Ground truth (5 classes)
Prediction of the simulated network
Wrong pixels in red
Pixel accuracy, mIoU and the IoU of every class

Try this: At the default output stride 8 the prediction gets about 95% of the pixels right, yet the small squares have an IoU of 0. Lower the stride to 4, then 2: at which stride does the thin pole come back, and why does it need a finer stride than the squares? Then raise the stride to 32: how many of the five classes still have a non-zero IoU? Finally switch the enlarging method to bilinear at stride 8: which numbers change, and which one stays at 0?

The label map is synthetic (128 × 128, drawn by code, no random numbers): a background of 85% of the pixels, a disk, a block, a 3-pixel pole and five 6 × 6 squares, all in separate classes. The network is idealized: at output stride s its score for each class in an s × s cell is the fraction of the cell’s pixels that belong to the class, which is the best a stride-s head can report. With nearest enlargement the class with the largest fraction labels the whole cell (ties go to the lowest class number); with bilinear enlargement the fractions are interpolated per class and the largest score labels each pixel. It is the construction of the chapter’s toy experiment on a smaller scene, so its numbers are this lab’s own and not the chapter’s. The scores follow the chapter: pixel accuracy is the share of correct pixels, the IoU of a class is true positives divided by true positives plus misses plus false alarms, and mIoU is the mean of the IoU over all five classes, the background included, computed from one confusion matrix.

Method and image sources

DLCV 05 · Is the argmax of a heatmap the keypoint?

Render a Gaussian heatmap at output stride 1, 2, 4 or 8 for 500 random sub-pixel keypoints and compare three ways to read the position back: the brightest cell, a quarter-cell offset and a Taylor step.

Open experimentClose experiment
Predict first

A network returns an almost perfect Gaussian heatmap at stride 4 (one cell = 4 input pixels) for a keypoint at a random sub-pixel position. You report the position of the brightest cell as the keypoint. On average, how far off is it?

Loading image…

Heatmap cells around one keypoint (values 0 to 1): true position, decoded position and the brightest cell
Mean error of the three decoders at each stride, over 500 keypoints
Histogram of the 500 errors for the chosen decoding and stride

Try this: At the default setting (stride 4, plain argmax) the 500 keypoints are off by 1.58 px on average, and the table of means shows the error doubling with every doubling of the stride: 0.37, 0.78, 1.58 and 3.12 px at strides 1, 2, 4 and 8, close to 0.383 times the stride. Switch the decoding to the quarter offset (about half of that) and then to the Taylor step: at stride 8 it brings 3.12 px down to 0.41 px. Now keep the Taylor step and raise σ from 2 to 4 cells: at stride 8 its error grows from 0.41 to 1.58 px. A wider blob should be easier to read, so why does the same noise hurt more?

A toy that mirrors the chapter’s code, with its own random numbers (so the means differ in the second decimal from the chapter’s run). 500 keypoints are drawn uniformly in [16, 240] × [16, 240] pixels of a 256 × 256 input. The heatmap at stride s is exp(−‖p − x/s‖² / (2σ²)) on integer cells p, centred on the exact unrounded position, plus Gaussian noise of standard deviation 0.01 per cell, floored at 10⁻⁶; a cell’s centre is the integer position, as in the chapter, so the brightest cell i maps to s · i. Argmax decodes s · (brightest cell). Quarter offset moves the brightest cell by ¼ cell on each axis toward the brighter neighbour. The Taylor step is one Newton step on the logarithm of the heatmap at the brightest cell, with central differences, kept only if the 2 × 2 Hessian is negative definite. The error is the Euclidean distance to the true position in input pixels. Only the (2R + 1)² cells around the true cell, R = ⌈2σ⌉ + 2, are generated; outside them the Gaussian is below 0.14 (against a peak of at least 0.77 and noise of 0.01), so the rest of the map cannot change the answer. The grid shows one fixed keypoint at (102.6, 78.9), the same noise for every decoder. Real networks add errors that this toy leaves out; the lab isolates the error that comes from the grid alone.

Method and image sources

DLCV 06 · How does CTC write a double letter?

Edit the blank probability in the frames between two equal letters of a toy recognizer output, and watch greedy best-path decoding and prefix beam search read it.

Open experimentClose experiment
Predict first

In the default matrix each of the two frames between the O’s says blank 0.45 and O 0.55, and every other frame is sure of its letter. What does greedy best-path decoding write for BOOK?

Loading image…

Toy network output: one row per frame, one column per symbol (orange bar: the frames between the repeated letters)
The five most probable texts, with all alignments of each added up

Try this: At the default setting (blank 0.45 and O 0.55 in each of the two frames between the O’s, greedy decoding) the largest entry of every frame gives the path B O O O O K, which merges to BOK: one O is lost. Yet the correct word has the largest total probability, 0.410, against 0.310 for BOK, because every alignment with a blank in either gap frame reads BOOK. Switch to prefix beam search (beam 3): it writes BOOK. Now move the blank probability. At 0.50 greedy decoding also writes BOOK (a tie goes to the blank). With beam search, 0.40 still gives BOOK, but at 0.35 it writes BOK (0.377 against 0.343 for BOOK). At 0.05 the network almost surely saw one long O: both decoders write BOK and the loss −ln p(BOOK) is 2.819. Finally choose SEE: greedy decoding flips at 0.50 and beam search between 0.35 and 0.40, just as for BOOK. Why can the sum over alignments recover a double letter that no single frame votes for?

A hand-made toy, not the output of a trained recognizer. Each letter of the word has one sure frame (the letter 0.85, the blank 0.15); between two equal letters there are two gap frames in which the network hesitates between the blank (probability q, the slider) and the letter (1 − q). Every other entry is 0, so the matrix asks one question only: blank or letter? BOOK has 6 frames (B, O, two gap frames, O, K) and SEE has 5. The collapsing map merges repeats first and then deletes blanks, so a double letter can be written only with a blank between its copies. Greedy decoding takes the largest entry of every frame; a tie goes to the blank, as NumPy’s argmax goes to the first index. The right-hand bars add up every alignment of each text by enumeration (4⁶ = 4,096 alignments for BOOK, 3⁵ = 243 for SEE); the probability of the correct word is also computed with the forward algorithm, and the tests check the two against each other. Prefix beam search keeps, for each prefix, the probability of alignments that end in a blank and of those that end in the last letter, and only the 3 best prefixes after every frame; pruning can lose some alignments, so its own probability can be a little below the exact one (0.366 against 0.377 for BOK at q = 0.35). The same code reproduces the 5-frame ‘cat’ example of the chapter (greedy ca, beam 3 cat 0.344, exact 0.363).

Method and image sources

DLCV 07 · Why do identities swap when objects meet?

Track three coloured blocks through 30 synthetic frames with a Kalman filter, a cost matrix and the Hungarian algorithm, and watch two identities swap when only box overlap is used.

Open experimentClose experiment
Predict first

Two walkers of the same size approach each other along one line, stand at the same spot and turn back. The tracker guesses each box with a constant-velocity Kalman filter. Which cue prevents the two tracks from swapping?

Loading image…

Synthetic frame: true boxes (white dashed), predicted boxes (dotted) and track boxes (solid, with trails), coloured by track id
Cost matrix of the inspected frame, with the Hungarian assignment outlined
Identity ribbon over the whole clip (click to choose the frame)

Try this: At the default setting (λ = 1, box overlap only) the orange and blue walkers swap tracks at frame 15, right after they meet: 2 identity switches, and 30 of the 90 object-frames carry the wrong id. Lower λ in steps of 0.05: the swap disappears between λ = 0.9 and 0.8, and at λ = 0.5 the mean IoU of the tracks with their own objects rises from 0.582 to 0.850. Now set the jitter to 0: overlap alone still swaps, so the turn and not the noise causes it. Finally set the jitter to 5 pixels: even λ = 0.5 gets 4 identity switches. At frame 13 the colour cost 1 − cos of track #1 is 0.47 for the blue walker and 0.61 for its own orange walker. Why is colour less reliable now?

Synthetic and tiny on purpose: everything is drawn by code, no photographs. Scene: three 30 × 40 pixel person-shaped blocks in a 160 × 128 frame over 30 frames (orange, blue and green shirts, each with its own trouser colour) plus Gaussian pixel noise of 0.03. The orange and blue walkers approach each other along one line at 4 pixels per frame, stand at the same x at frame 14 (the blue one 8 pixels lower and in front) and turn back; the green one walks along the bottom. The scene uses meet-and-turn rather than a pass-through because a pass-through at constant velocity is kept apart by motion alone and swaps only by chance under noise, whereas after a turn the straight-line guess points at the wrong walker (the case the chapter calls the bounce). The tracker follows the chapter: a Kalman filter on [cx, cy, w, h, vx, vy] (q = 1, r = 4), C = λ(1 − IoU) + (1 − λ)(1 − m·e) where e is the unit-length colour histogram (4 bins per channel) of the pixels inside the detection box and m a track memory updated with α = 0.9, and the Hungarian algorithm. Simplified: every frame has exactly one detection per object (no misses, false alarms, gating, birth or death); detections arrive in a rotating order and carry their true object only for scoring; the jitter is a fixed field of random numbers (seed 16) scaled by the slider, so it never re-rolls. Metrics: an identity switch is an object matched to a different track id than in the previous frame (the chapter’s definition), so two swapped tracks count 2; the mean IoU compares each track’s filtered box with the true box of the object the track started on; MOTA = 1 − (FN + FP + IDSW) / GT with FN = FP = 0 and GT = 90 object-frames, so two switches cost only 0.022 although the ids stay wrong for 15 frames. One jitter draw is one clip: over 200 other draws at σ = 2.5, overlap alone switches in 164 and λ = 0.5 in 12; at σ = 5, in 165 and 90. A NumPy and SciPy re-implementation reproduces every number.

Method and image sources

DLCV 04 · A sharp guess can be less faithful

A blurry observation fits two sharp clean signals. The blurry average has the lowest expected squared error; a sharp guess looks better but is usually farther from the truth.

Open experimentClose experiment
Predict first

Set P(A) = 0.9. The sharp copy of A is now very likely correct. Does the blurry mean still have the lower expected squared error?

Expected reconstruction MSE
0.5
Conditional-mean minimum MSE
0.25
Sharpness (adjacent total variation)
3

Averaging incompatible possibilities minimizes expected squared error, but can remove structures present in every valid clean outcome. Choosing one restores structure while risking the wrong structure.

Try this: Start from the sharp copy of A (commitment 1): its risk is higher than the blurry mean's. Slide the commitment to 0 and watch the risk fall to the minimum. Then raise the probability of A to 0.9: does the sharp copy of A catch up, and can it ever win?

Two fixed 8-sample patterns, A = 00110011 and B = 11001100, are the only clean signals that could have produced the same blurry observation. P(A|y) = p is supplied, not learned. The blurry mean is p·A + (1−p)·B. Your reconstruction blends the mean toward A: (1−m)·mean + m·A, so m = 1 is exactly A (a sharp sample) and m = 0 is the mean. Risk = the expected mean squared error, p·MSE(A, output) + (1−p)·MSE(B, output), averaged over which pattern is the truth. Sharpness is the total variation (sum of absolute differences between neighbouring samples), not a perceptual quality score.

Method sources

DLCV 09C · Gain cannot create missing photons

Simulate a dim photo with photon and read noise, then compare more light, more digital gain and a denoiser.

Open experimentClose experiment
Predict first

You double the digital gain while nothing clips. What happens to the signal-to-noise ratio?

Loading image…

The scene (what we want)
Sensor output after gain
After the enhancer

Try this: At 8 photons and gain 12 the picture is as bright as the scene, yet very noisy. Lower the gain to 1: the picture goes dark, but does the signal-to-noise ratio change? Now keep gain 12 and raise the photons to 100. What changes instead, and what happens to the highlights?

Camera test image, 256 × 256. Each pixel collects Poisson(photons × scene) electrons plus Gaussian read noise (σ = 2 electrons), then the output is clip(gain × electrons / 100). Noise fields are fixed, so sliders never re-roll them. SNR is the mean signal over the standard deviation of the noise, measured before clipping, in dB. The enhancers are a Gaussian blur (σ = 1.5) and a 3 × 3 median, stand-ins for a learned denoiser; they are not one. Large photon counts use a normal approximation of the Poisson distribution.

Method and image sources

DLCV 09C · How big must a 3D LUT be?

Bake one smooth color enhancement into an M × M × M table and apply it with a nearest-node lookup or with trilinear interpolation. Compare both with the exact transform.

Open experimentClose experiment
Predict first

A nearest-node table with M = 33 has about 49 times as many entries as a trilinear table with M = 9 (35,937 against 729 nodes). Which one reproduces the exact transform more closely on the smooth gradient?

Loading image…

Exact transform (the target)
LUT, nearest node
LUT, trilinear
Error map of the chosen lookup (8×, bright: wrong)

Try this: At the default (M = 9, the smooth gradient) the nearest lookup shows clear bands and the trilinear result looks like the exact one. Switch the error map to trilinear: where is the error largest, and why there? Then raise M to 33 for the nearest lookup: are the bands gone? Finally set M = 3 and compare the two interpolations.

The true enhancement is defined in code (src/lib/playground/labs/lut-trilinear.ts) and is the lab’s own toy: on each channel t, a tone curve g = t^0.9 followed by the S-curve (1 − a) g + a (3g² − 2g³) with a = 0.5; then a saturation gain of 1.15 that scales each color’s distance from its luma gray (luma 0.299 R + 0.587 G + 0.114 B); then a hue turn of 8° (a rotation of the RGB cube about its gray axis); then clipping to [0, 1]. The table has one entry per node of an M × M × M lattice over the RGB cube: node (i, j, k) stores the exact transform of the color (i/(M−1), j/(M−1), k/(M−1)), so it holds 3M³ numbers. A nearest lookup returns the stored color of the closest node. Trilinear interpolation blends the 8 corners of the cell around the color with weights that are products of the offsets, as in the chapter. Both results and the exact transform are rounded to 8-bit levels (0 to 255) before the errors are measured, and the error map is magnified 8 times. The photo is the astronaut test image reduced to 256 × 256; the gradient is a smooth blend of four corner colors computed in the lab, so it has no texture to hide banding.

Method and image sources

DLCV 10 · Does averaging two photos fuse them?

Two shots of one scene are each sharp in a different region. Average them, or pick every pixel from the shot with more local detail, and score both against the all-in-focus truth.

Open experimentClose experiment
Predict first

Shot A is sharp inside the disk and shot B is sharp outside it. Inside the disk, what does the plain average (A + B)/2 look like?

Loading image…

Shot A (sharp inside the disk)
Shot B (sharp outside the disk)
Average (A + B)/2
Fused by local detail
Decision map (white: take A; red line: true boundary)
Error of the fused image (4×, bright: wrong)

Try this: Start from the defaults: the average is blurry everywhere while the focus rule is sharp. Now lower the window to 0.5 and look at the decision map: why does it fill with specks, and where do they hurt the picture? Switch the detail measure to local variance at window 0.5 (it is far less forgiving), then raise the window and smooth the map by 2. What does smoothing buy, and what does a very large window cost at the circle’s edge?

Synthetic multi-focus pair (scripts/figures/playground_image_fusion.py), the recipe of the chapter’s snippet at half the size: the camera test image (256 × 256) is the all-in-focus truth; shot A keeps it inside a disk and is blurred (Gaussian, σ = 2) outside, shot B the other way round, each with Gaussian noise of standard deviation 0.01. Because we made the blur, the true decision map (the disk) is known, which real multi-focus pairs never provide. The average is (A + B)/2. The focus rule scores each shot at each pixel with a detail measure averaged over a Gaussian window of standard deviation σ (the “window”): the Laplacian energy G_σ * (∇²I)², with ∇²I = I(x−1,y) + I(x+1,y) + I(x,y−1) + I(x,y+1) − 4 I(x,y), or the local variance G_σ * I² − (G_σ * I)². The decision map marks the pixels where shot A scores higher, and F = w A + (1 − w) B with w the decision map, optionally blurred by a Gaussian of the chosen σ (0 keeps it binary). PSNR = 10 log₁₀(1/MSE) in dB against the truth, pixel values in [0, 1]. Map agreement is the share of pixels, counted only where the picture has texture (smoothed |Laplacian| of the truth above 0.01), on which the decision equals the true disk. These numbers belong to this toy.

Method and image sources

DLCV 11A · Is a bigger matching window always better?

Run block matching on a synthetic stereo pair with known disparity, and see small windows turn noisy while large windows fatten the foreground.

Open experimentClose experiment
Predict first

At the default noise a 3 × 3 window gives a noisy disparity map. Suppose you enlarge the window step by step up to 31 × 31. What happens to the bad-pixel rate (bad-1 over all scored pixels)?

Loading image…

Left image (with noise)
True disparity (dark: far, light: near; left strip not scored)
Estimated disparity (pink: invalidated; white line: true depth edge)
Error against window size at this noise level

Try this: At the default 3 × 3 window the map is noisy, most of all inside the textureless patch. Grow the window one step at a time and watch the three curves: where is the all-pixel error lowest? What happens to the white outline of the disc once the window reaches 31 × 31? Then raise the noise to 0.06 and see how much a 3 × 3 window suffers. Finally switch the left–right check on: which pixels turn pink, and what happens to bad-1?

Synthetic and tiny on purpose: the left image is 128 × 96 and is drawn by code, with fixed seeds so nothing flickers. A textured background plane has disparity 4, a brighter textured disc has disparity 12, and a flat gray rectangle lies on the background. The right image shifts every left pixel left by its disparity (the nearer surface wins) and fills the uncovered pixels with fresh texture; Gaussian sensor noise with the chosen standard deviation is added to both images. The matcher is the one of the chapter: for every disparity 0 to 15 the cost of a window is the mean absolute difference (SAD, normalized by the number of pixels that have a partner), and the winner-take-all rule takes the lowest cost. Bad-1 is the percentage of scored pixels whose disparity is off by more than 1 pixel. Scored pixels are those visible in both views and at least 16 columns from the left border (washed out in the pictures). The check matches again with the right image as reference and invalidates a pixel whose two answers differ by more than 1; bad-1 and the flat-patch score are then taken over the kept pixels. Disc fattening counts the visible background pixels labelled with the disc’s disparity (within 1), as a percentage of the disc’s true area. These are the lab’s own numbers on its own toy scene, not those of the chapter’s scene.

Method and image sources

DLCV 07 · Small dynamics errors compound in rollouts

Compare one-step predictions, open-loop rollout and periodic observation refresh under the same actions.

Open experimentClose experiment
One-step MSE
0.00195
Open-loop MSE
1.15919
Refreshed-rollout MSE
0.02323

One-step evaluation repeatedly supplies the true state. An open-loop rollout must consume its own imperfect predictions. Periodic observations limit drift without improving the transition model itself.

Try this: Set the model coefficient to the true value 0.96. Then change it to 1.02 and increase the horizon. How much can observation refresh correct?

True scalar state: x(t+1)=0.96x(t)+u(t), x(0)=1, u(t)=a sin(0.45t). The model replaces 0.96 with a chosen coefficient. Refresh feeds the true state every k steps; k=0 disables it. Fully observed deterministic dynamics, no learned video model or replanning.

Method sources

DLCV 14 · Why can an invisible change flip a prediction?

Train a linear classifier on noisy stripe images of 64, 256 or 1,024 pixels, then change every pixel by the same few levels, once along the loss gradient (FGSM) and once with random signs.

Open experimentClose experiment
Predict first

Every pixel of a 32 × 32 test image is changed by at most 8 of 255 levels, far below the pixel noise of 51 levels. Which change brings the accuracy of the trained classifier close to zero: signs that follow the loss gradient (FGSM), random signs, both, or neither?

Loading image…

Left: a clean test image (noisy stripes). Middle: the change, stretched to the full gray range for display (white = +ε, black = −ε). Right: the perturbed image. Under each, the logit and the class it gives.
Accuracy on 300 test images as ε grows

Try this: At the default setting (32 × 32 pixels, ε = 8, FGSM) accuracy falls from 99.3% to 2.7%, while random signs of the same size leave it at 99.3%. The mean logit shift is +5.8, against a median clean margin of 3.5. Now switch the size to 16 × 16 and then to 8 × 8 at the same ε: FGSM leaves 76.3% and 94.7%. Next find the ε at which FGSM first brings accuracy below 50%: ε = 5 for 32 × 32, ε = 11 for 16 × 16, and none up to ε = 16 for 8 × 8 (70.7% at ε = 16). The classes differ by the same total amount at every size, so why does the same ε matter more when the image has more pixels?

Synthetic and tiny on purpose: no photographs. Each image is 0.5 plus or minus an oblique cosine stripe pattern (two periods across, one down) plus Gaussian noise with standard deviation 0.2 (51 of 255 levels), clipped to [0, 1]; class A has the stripes in one phase and class B in the opposite phase. The stripe amplitude per pixel is 3 × 0.2 / √d for d pixels, so the two classes differ by the same total amount at every size (before clipping, the best possible classifier is equally accurate at every size); only the number of pixels that carry the evidence changes. The classifier is a logistic regression with one weight per pixel, trained in your browser by 50 steps of full-batch gradient descent (step 8, weight decay 0.001) on 300 images and tested on 300 other images (seeds 11 and 12). Its logit is z = w·(x − 0.5) + b, so the gradient of the cross-entropy loss with respect to the pixels is (p − y)·w and FGSM, x′ = clip(x + ε·sign(∇ₓL), 0, 1), moves every pixel by ε toward the wrong class. With no clipping the logit moves by exactly ε‖w‖₁; random signs move it by about ε‖w‖₂ and cancel on average. Here ‖w‖₂ is 7.5, 7.5 and 6.9 for 64, 256 and 1,024 pixels, while ‖w‖₁ is 54.4, 102.8 and 186.2. ε is counted in levels of 8-bit brightness (1 level = 1/255). This is a linear toy that isolates the argument of Goodfellow et al.; it is not the digits experiment of the chapter, a deep network is not linear, and one draw of data and training is shown, so read the trend, not the exact percentages.

Method and image sources

Optimization

Does the whole segment stay feasible?

Place two feasible points on a circle, slide along the segment between them, and compare a disk with an annulus.

Open experimentClose experiment
Predict first

Both endpoints lie on a circle of radius 0.85 and the annulus forbids radius below 0.5. When do the endpoints stay feasible but the segment between them does not?

Radius at weight λ
0.291
Smallest radius on the segment
0.291
Angle apart (degrees)
140

The chosen point lies in the hole although both endpoints are feasible: the annulus is not convex.

Try this: Choose the annulus and place both endpoints on the circle. Find two endpoints whose whole segment stays feasible, and two whose segment crosses the hole. How far apart in angle do they need to be?

Endpoints are P = 0.85(cos α, sin α) and Q = 0.85(cos β, sin β). The disk has radius 1; the annulus also requires radius ≥ 0.5. The segment is x(λ) = (1−λ)P + λQ. One counterexample disproves convexity; sampled examples alone do not prove it.

Method sources

Compare a function with its chord

Measure the Jensen gap while moving between two inputs. A negative gap supplies a concrete counterexample.

Open experimentClose experiment
Input
0.5
Chord − function
2.25

The function is no higher than this chord at the selected point.

Try this: Compare x² with −x² at weight 0.5. Why do both gaps become zero at the endpoints?

For endpoints a=−1 and b=2, gap = (1−λ)f(a)+λf(b)−f((1−λ)a+λb). Nonnegative gap on this single segment is not a proof on every domain.

Method sources

Why clipping a solution can be wrong

Compare coordinate clipping with the exact solution of a coupled quadratic under nonnegative constraints.

Open experimentClose experiment
Optimal objective
0.255
Clipped objective
0.5
Excess objective
0.245

Optimal (x,y)=(0, 0.3). Coupling changes the best boundary position.

Try this: Set coupling to zero, then increase it. When does clipping cease to minimize the objective?

Minimize ½[(x−a)²+2ρ(x−a)(y−1)+(y−1)²], x,y≥0. |ρ|<1 ensures a positive definite Hessian. The solver checks the interior and both boundary minima; it does not use a grid approximation.

Method sources

Build a primal–dual certificate

Change a candidate point and multiplier. A small gap is meaningful only with feasibility.

Open experimentClose experiment
Objective − dual
0.625
Constraint violation
0
Stationarity residual
0.5
Complementarity
0.5

The candidate is feasible. Its objective minus the dual bound is an upper bound on suboptimality.

Try this: Use b=1, x=0, λ=0. The displayed difference is zero: why is this not an optimality certificate?

Primal: minimize x²/2 subject to x≥b. L=x²/2+λ(b−x), λ≥0; dual g(λ)=λb−λ²/2. Stationarity is x−λ=0 and complementarity is λ(x−b)=0.

Method sources

One outlier, three meanings of best fit

Move one observation and compare squared, absolute and Huber losses for fitting a constant.

Open experimentClose experiment
Predict first

The sixth observation moves from 2 up to 12 while the other five stay put. Which fit is dragged furthest?

Fitted constant
1.333
Total loss
27.917

The loss defines what an error costs. Robust residual loss and parameter regularization solve different problems.

Try this: Increase the outlier from 2 to 12. Compare the movement of the squared-loss and Huber estimates.

Five fixed observations are −1, −0.5, 0, 0.5, 1. The sixth is adjustable. Squared loss includes ½; Huber uses δ=1. For absolute loss, any point in [0,0.5] minimizes the loss; the plot chooses 0.25.

Method sources

Follow a gradient down a narrow valley

See why a step size that works on a round bowl can diverge on a narrow one.

Open experimentClose experiment
Predict first

The curvature ratio is k = 5. Which step size makes the iterate blow up instead of settling?

Final objective
0.034
Converges when 0 < η <
0.4
Coordinate multipliers (x, y)
0.85, 0.25

Both coordinate multipliers have magnitude below 1, so the iterates converge. A negative multiplier produces an alternating path; a multiplier close to 1 or −1 produces slow progress.

Try this: Set k = 10. Compare step sizes 0.1, 0.2 and 0.3. What happens to the y coordinate at the stability boundary?

An exact quadratic: f(x,y) = (x² + ky²)/2, starting at (3,2). Updates are x ← (1−η)x and y ← (1−ηk)y. The contour plot has a fixed viewing window; displayed objective values are never clipped.

Method sources

Follow the central path to the optimal vertex

Raise the barrier weight t on a two-variable linear program and watch the central point, its dual bound and the gap m/t.

Open experimentClose experiment
Predict first

You multiply the barrier weight t by 10. What happens to the duality gap m/t?

Gap m/t
0.5
Suboptimality c·x − p*
0.1841
Distance to nearest edge
0.109

At t = 10 the central point is 0.109 from the nearest edge. Its suboptimality 0.1841 is at most the duality gap m/t = 0.5, and the gap computed from the dual point λ = 1/(t·slack) is 0.5.

Try this: Multiply t by ten. How do the distance to the boundary, the true suboptimality and the bound m/t change? Then rotate the objective: does the path still end at a vertex?

Minimize c·x over a convex pentagon with m = 5 edge constraints a_i·x ≤ b_i. The central point minimizes t·c·x − Σ log(b_i − a_i·x) (damped Newton). With λ_i = 1/(t s_i), where s_i is the slack, the dual objective is −b·λ and the duality gap equals Σ λ_i s_i = m/t exactly. The true suboptimality c·x − p* is at most m/t.

Method sources

Why a proximal step creates exact zeros

Compare soft thresholding with ordinary shrinkage and inspect its nonsmooth objective.

Open experimentClose experiment
Proximal solution
0.5
Objective
1
Residual v − x
1

Outside the threshold, the magnitude is reduced by λ.

Try this: Keep v=1 and increase λ through 1. Why does the optimum stay at exactly zero afterward?

Solve minₓ ½(x−v)²+λ|x|. The exact proximal solution is sign(v) max(|v|−λ,0). This scalar identity extends coordinatewise to an L1 penalty; it is not a complete total-variation denoiser.

Method sources

Information Theory

Separate entropy from model mismatch

Change the true probability and the model probability; measure the extra coding cost caused by a wrong model.

Open experimentClose experiment
Entropy
0.722
Cross-entropy
1
KL divergence
0.278

H(P,Q) = 0.722 + 0.278 bits. Matching probabilities removes the KL penalty, but cannot remove source uncertainty.

Try this: Fix p=0.8. Find the q that minimizes cross-entropy, then compare q=0.2.

For Bernoulli sources, H(p,q)=−p log₂q−(1−p)log₂(1−q)=H(p)+D(p∥q). q stays strictly between 0 and 1. These are expected ideal bit costs, not measured file sizes.

Method sources

How much information survives noise?

Separate source uncertainty, transmitted information and channel capacity.

Open experimentClose experiment
Predict first

Move the input probability q = P(X=1) away from 0.5 with the flip probability fixed. What happens to the channel capacity C?

H(X)
1
H(Y)
1
I(X;Y)
0.531
Capacity 1 − h₂(p)
0.531

The output has P(Y=1) = 0.5. I(X;Y) = H(Y) − h₂(p) = 1 − 0.469 = 0.531. Random output alone does not imply useful information: at p = 0.5 the output is independent of the input.

Try this: With crossover p = 0.1, move the input prior away from 0.5. Does the channel capacity change? Does the transmitted information change?

Independent bit flips define a binary symmetric channel. q = P(X=1), p = P(Y≠X), and P(Y=1) = p + q(1−2p). All values are bits per channel use. Capacity maximizes mutual information over q; here q = 0.5 achieves it.

Method sources

A likely block is not a typical set

Compare the probability mass of a typical set with its fraction of all possible binary blocks.

Open experimentClose experiment
Typical probability mass
0.6575
Fraction of all strings
0.03815
H(p) (bits)
0.722

Typicality concerns information per symbol. A single likely string and a set carrying most probability are different objects.

Try this: Fix p=0.2 and increase block length. Does the typical set contain most probability while using most of all binary strings?

Independent Bernoulli(p) blocks of length n. A block is typical when |−log₂ P(block)/n−H(p)|≤ε. Each bar aggregates all C(n,k) strings with k ones. Small n can give an empty typical set; AEP is asymptotic.

Method sources

Give common symbols shorter codes

Build a binary prefix code and compare its average length with source entropy.

Open experimentClose experiment
Source entropy
1.357
Expected code length
1.5
Redundancy (bits/symbol)
0.143

No codeword is the prefix of another, so concatenated bits can be decoded without separators. Expected length is Σ p(symbol) × code length = 1.5 bits/symbol. Entropy is 1.357; their gap is 0.143. Spaces in the example only show codeword boundaries.

Try this: Compare P(A) = 0.25 with P(A) = 0.7. Why does the optimal code change in discrete jumps while entropy changes smoothly?

Huffman coding repeatedly merges the two least probable nodes. Ties use alphabetical order. B, C and D each have probability (1−P(A))/3. The expected length assumes this source distribution; the example message has its own symbol counts. Codebook transmission cost is excluded.

Method sources

Allocate power where it carries more information

Compare equal allocation with water-filling across two parallel Gaussian channels.

Open experimentClose experiment
Optimal bits / joint use
1.17
Equal-allocation bits
1.085
Capacity gain
0.085

P₁=3.5, P₂=0.5. Active channels share a water level; a noise floor above it receives no power.

Try this: Give the second channel high noise. Increase the total power until it first receives a positive allocation.

Independent real Gaussian channels, noise variances N₁=1 and N₂, total power P. Capacity per joint use is ½Σ log₂(1+Pᵢ/Nᵢ); water-filling uses Pᵢ=max(μ−Nᵢ,0). This assumes optimal Gaussian signaling, not a finite code.

Method sources

How much rate buys a given error?

Inspect the Bernoulli rate–distortion curve and an attaining joint distribution under Hamming error.

Open experimentClose experiment
D
0.12
R(D)
0.352
P(Y=1)
0.237

Orange off-diagonal cells are reconstruction errors. Their total equals D; the joint distribution retains the selected source marginal.

Try this: Move distortion to its maximum. Why is sending no bits sufficient even though some symbols are wrong?

X~Bernoulli(p), p≤0.5, Hamming distortion D≤p. R(D)=h₂(p)−h₂(D). Construct X=Y XOR Z with independent Z~Bernoulli(D) and P(Y=1)=(p−D)/(1−2D). At p=D=0.5 choose constant Y=0. The curve is an asymptotic block-coding limit, not the rate of this single-symbol channel.

Method sources

Trade false alarms for missed alternatives

Slide a decision threshold across two exact binomial distributions and compute both error probabilities.

Open experimentClose experiment
False alarm
0.04735
Miss
0.15027
Detection probability
0.84973

Reject H₀ for k ≥ 6. Changing a threshold changes both errors; increasing evidence can improve their tradeoff.

Try this: Try thresholds that always accept or always reject H₀. Then increase n while keeping the threshold near n/2.

n independent binary observations. H₀:p=0.3, H₁:p=0.7; reject H₀ if number of ones k≥threshold. Curves show exact finite-sample probability masses. False alarm is conditional on H₀, miss on H₁; neither is a posterior probability.

Method sources

Two useless clues can be useful together

Compute information about a noisy XOR label from each independent bit and from their pair.

Open experimentClose experiment
Predict first

Each of A and B alone carries zero information about the label. With no label noise, how much do the two together tell you?

I(A;Y)
0 bits
I(B;Y)
0 bits
I(A,B;Y)
0.531 bits
I(A;Y|B)
0.531 bits

Averaging either row or column gives 0.5: a single feature does not change the label distribution. Observing both can.

Try this: Set label noise to zero. Why does each feature have zero information while their pair has one bit?

A and B are independent fair bits, Y=A XOR B XOR E, E~Bernoulli(ε) independent. Thus I(A;Y)=I(B;Y)=0 and I(A,B;Y)=1−h₂(ε). This population identity does not establish a finite-sample generalization bound.

Method sources