Full search page →

A likely block is not a typical set

IntermediateIT 02 · Typical Sequences, AEP and Entropy RatesInteractive plot

Description

Compare the probability mass of a typical set with its fraction of all possible binary blocks.

Try this: Fix p=0.2 and increase block length. Does the typical set contain most probability while using most of all binary strings?

Hints

The chapter section that uses this lab:

Lab · A likely block is not a typical set

Typical probability mass
0.6575
Fraction of all strings
0.03815
H(p) (bits)
0.722

Typicality concerns information per symbol. A single likely string and a set carrying most probability are different objects.

Try this: Fix p=0.2 and increase block length. Does the typical set contain most probability while using most of all binary strings?

Independent Bernoulli(p) blocks of length n. A block is typical when |−log₂ P(block)/n−H(p)|≤ε. Each bar aggregates all C(n,k) strings with k ones. Small n can give an empty typical set; AEP is asymptotic.

Method sources

Open in the Playground →