Full search page →

Factor a tiny image into conditional predictions

AdvancedNLP 09 · Language Modeling with Transformers (Natural Language Generation)Interactive plot

Description

Enumerate all sixteen binary 2×2 images under a first-order pixel model, then change the generation order.

Try this: Compare row stripes and a checkerboard when copying is likely. Switch raster to column order: which model assumptions changed?

Lab · Factor a tiny image into conditional predictions

Image probability
0.064
Bits per pixel
0.991
All-image probability sum
1

Joint probability multiplies normalized conditional factors. All sixteen image probabilities sum to one. A high-copy model prefers sequences with fewer transitions, which need not correspond to the same spatial pattern in each order.

Try this: Compare row stripes and a checkerboard when copying is likely. Switch raster to column order: which model assumptions changed?

First pixel has probability 1/2; each later pixel copies the previous one with probability c. The chain rule is exact, but this model deliberately retains only one previous pixel. Order dependence here comes from that restricted conditional model.

Method sources

Open in the Playground →