Full search page →

DLCV 01 · How much of the image can one output unit see?

IntermediateML 03 · Convolution, Image Filtering and CNNsImage lab

Description

Pick a stack of convolution layers and a spot in a real photograph: the exact box of input pixels that one output unit depends on is drawn on the image, with its size, jump and the share of the image it covers.

Try this: Choose “Six 3×3 layers, stride 2 in every second” and click the image: the unit sees 29 × 29 pixels, 1.3% of the picture, and its neighbors are 8 pixels apart. Now choose “Four 3×3 layers, dilations 1, 2, 4, 8”: a similar field, 31 × 31 pixels (1.5%), but the jump stays 1, so every pixel keeps its own output unit. Finally click the top-left corner: why is the covered share smaller there than in the middle of the photograph?

Predict first

Three stacked 3×3 layers give each output unit a 7 × 7 pixel receptive field. Which single change makes it grow the most?

Hints

The chapter section that uses this lab:

Explanation

Answer the prediction first, or open the explanation when you are ready.

Show the explanation

Stride 2 in the first layer wins: the field grows from 7 to 11, against 9 for a fourth layer and 9 for dilation 2 in the first layer. A stride doubles the jump, the pixel spacing between neighboring units, for every later layer, so each of the two remaining 3×3 layers now adds 4 pixels instead of 2. Dilation 2 on a layer fed by the input (jump 1) adds only 2 pixels, the same as one more layer. Where the stride sits matters: in the last layer it changes nothing, and the field stays 7.

Lab · DLCV 01 · How much of the image can one output unit see?

Predict first

Three stacked 3×3 layers give each output unit a 7 × 7 pixel receptive field. Which single change makes it grow the most?

Loading image…

Photograph with the receptive field of one output unit (yellow box). Click or drag to move the unit
Everything this unit can see, enlarged
The stack, layer by layer: r is the receptive field and j the jump after that layer; bars are traced exactly

Try this: Choose “Six 3×3 layers, stride 2 in every second” and click the image: the unit sees 29 × 29 pixels, 1.3% of the picture, and its neighbors are 8 pixels apart. Now choose “Four 3×3 layers, dilations 1, 2, 4, 8”: a similar field, 31 × 31 pixels (1.5%), but the jump stays 1, so every pixel keeps its own output unit. Finally click the top-left corner: why is the covered share smaller there than in the middle of the photograph?

Real photograph: skimage.data.camera at 256 × 256 (credit below). Each layer is a square convolution with kernel k, stride s and dilation d, padded with zeros by (k_eff − 1)/2 on each side, where the effective kernel is k_eff = (k − 1)·d + 1. The size and jump come from the recurrence r ← r + (k_eff − 1)·j and j ← j·s (r = j = 1 at the input), and the box on the image from tracing the index interval of one output unit back through every layer: both agree for every stack here, and an independent SciPy computation of which input pixels influence one unit gives the same boxes. The unit snaps to the nearest position on the output grid, whose spacing is the jump. Where the box leaves the image, the part outside is zero padding and the covered share counts only the part inside. This is the theoretical receptive field: the set of pixels that can change the unit. Measured influence is not uniform; it is concentrated near the center of the box, so the effective receptive field is smaller. That is not computed here, and no weights are trained.

Method and image sources

Open in the Playground →