Full search page →

DLCV 10 · Does averaging two photos fuse them?

AdvancedDLCV 10 · Image Fusion: Complementary Images, Useful OutputsImage lab

Description

Two shots of one scene are each sharp in a different region. Average them, or pick every pixel from the shot with more local detail, and score both against the all-in-focus truth.

Try this: Start from the defaults: the average is blurry everywhere while the focus rule is sharp. Now lower the window to 0.5 and look at the decision map: why does it fill with specks, and where do they hurt the picture? Switch the detail measure to local variance at window 0.5 (it is far less forgiving), then raise the window and smooth the map by 2. What does smoothing buy, and what does a very large window cost at the circle’s edge?

Predict first

Shot A is sharp inside the disk and shot B is sharp outside it. Inside the disk, what does the plain average (A + B)/2 look like?

Hints

The chapter section that uses this lab:

Explanation

Answer the prediction first, or open the explanation when you are ready.

Show the explanation

Inside the disk A equals the truth T and B is a blurred copy T′, so (A + B)/2 = (T + T′)/2. The blurred copy has very little fine detail, so the fine detail of T survives at only about half amplitude (a little more, because T′ keeps some detail and the noise adds some), on top of a soft ghost. Measuring the fine contrast as the root-mean-square of the image minus its Gaussian blur (σ = 2) inside the disk, the average keeps 61% of the truth’s and the focus-rule result 101%. Averaging cannot choose; selecting by local detail can.

Lab · DLCV 10 · Does averaging two photos fuse them?

Predict first

Shot A is sharp inside the disk and shot B is sharp outside it. Inside the disk, what does the plain average (A + B)/2 look like?

Loading image…

Shot A (sharp inside the disk)
Shot B (sharp outside the disk)
Average (A + B)/2
Fused by local detail
Decision map (white: take A; red line: true boundary)
Error of the fused image (4×, bright: wrong)

Try this: Start from the defaults: the average is blurry everywhere while the focus rule is sharp. Now lower the window to 0.5 and look at the decision map: why does it fill with specks, and where do they hurt the picture? Switch the detail measure to local variance at window 0.5 (it is far less forgiving), then raise the window and smooth the map by 2. What does smoothing buy, and what does a very large window cost at the circle’s edge?

Synthetic multi-focus pair (scripts/figures/playground_image_fusion.py), the recipe of the chapter’s snippet at half the size: the camera test image (256 × 256) is the all-in-focus truth; shot A keeps it inside a disk and is blurred (Gaussian, σ = 2) outside, shot B the other way round, each with Gaussian noise of standard deviation 0.01. Because we made the blur, the true decision map (the disk) is known, which real multi-focus pairs never provide. The average is (A + B)/2. The focus rule scores each shot at each pixel with a detail measure averaged over a Gaussian window of standard deviation σ (the “window”): the Laplacian energy G_σ * (∇²I)², with ∇²I = I(x−1,y) + I(x+1,y) + I(x,y−1) + I(x,y+1) − 4 I(x,y), or the local variance G_σ * I² − (G_σ * I)². The decision map marks the pixels where shot A scores higher, and F = w A + (1 − w) B with w the decision map, optionally blurred by a Gaussian of the chosen σ (0 keeps it binary). PSNR = 10 log₁₀(1/MSE) in dB against the truth, pixel values in [0, 1]. Map agreement is the share of pixels, counted only where the picture has texture (smoothed |Laplacian| of the truth above 0.01), on which the decision equals the true disk. These numbers belong to this toy.

Method and image sources

Open in the Playground →