A decoder that computes in rapidity

A spacecraft sends three tiny flashes. The receiver sees two clear flashes and one that might have been a reflection. Fortunately, the sender made a promise: every transmitted word contains an even number of ones.

That promise ties the flashes together. Two reasonably clear readings can help interpret the blurry third one. An error-correcting code creates such relationships deliberately, giving a receiver extra ways to recover a message after noise disturbs it.

Let’s build a receiver whose internal quantities already live in Epistemic Physics. Its two elementary instructions will sound familiar to anyone who has wandered through the evidence cone:

• To combine independent observations of the same bit, add their rapidities.
• To ask whether two hidden bits agree, multiply their velocities.

The second instruction turns this into a working decoder. Put the instructions together in a particular arrangement, and we can also watch information separate into easier and harder questions.

Give each reading a cone

A binary channel takes an input bit $X$, either 0 or 1, and produces an output $Y$. Write $W(y\mid x)$ for the probability of receiving the particular output $y$ when the transmitted bit was $x$.

For a received reading, choose the positive coordinates

The two coordinates are likelihoods. Their sum measures the total likelihood weight of this output; it supplies the scale for this use of the cone. The familiar coordinate changes are

Here $\tau$ and $x$ are the cone’s time and space coordinates, $v$ is its normalized direction, and $m$ is its mass. With equally likely input bits, the posterior probability of a zero is $(1+v)/2$.

Define the epistemic rapidity from this velocity, writing $r={\varphi }_{\mathrm{E}}$ for our decoder. Communication engineers often store a log-likelihood ratio, abbreviated LLR. It is exactly twice that rapidity:

The logarithm is natural. The function $\mathrm{artanh}$ is the inverse hyperbolic tangent, so $v=\tanh r$. Positive $r$ favors zero; negative $r$ favors one. A direction near either cone edge means the reading strongly distinguishes the alternatives.

The bridge is already operational: a decoder’s LLR register can be read as an epistemic rapidity register. The likelihood-to-boost derivation appears in When evidence takes different roads. Here we will wire those registers together.

Two messages, two different questions

First let two observations concern the same hidden bit. Assume their noise is independent once that bit is fixed. The joint likelihood pair is

$${\alpha }_{+}={\alpha }_1{\alpha }_2,{\beta }_{+}={\beta }_1{\beta }_2.$$

Taking a half-log ratio gives

$$\boxed{r_{+}=r_1+r_2}.$$

In velocity coordinates, the same calculation reads

$$\boxed{v_{+}=\frac{v_1+v_2}{1+v_1{v}_2}}.$$

Agreement adds two boosts in the same direction. Disagreement can partly or completely cancel them. This is how two independent sensors corroborate or challenge each other.

Now change the question. Two messages concern different hidden bits, and we ask whether the bits agree. Their parity is zero when they agree and one when they differ. Parity is also called exclusive-or, written $\mathrm{xor}$.

There are two ways to agree and two ways to differ. Sum the corresponding joint weights:

Add and subtract these two expressions. A striking factorization appears:

Consequently,

$$\boxed{v_{-}=v_1{v}_2},$$

and the decoder’s parity instruction becomes

$$\boxed{\begin{array}{c}{r}_{-}=\\ \mathrm{artanh}(\tanh r_1\tanh r_2)\end{array}}.$$

These gate weights are unnormalized. If the two hidden bits start independently and fairly, the conditional likelihoods given their parity each contain an additional factor $1/2$, because each parity allows two equally likely hidden assignments. That common factor rescales $\tau$ and $x$ together; all the displayed direction and rapidity formulas survive.

A parity check has a weakest-link geometry

Suppose both incoming velocities equal $0.8$. Each message assigns probability $0.9$ to zero. The parity velocity is $0.64$, giving probability $0.82$ that the bits agree.

That reduction has a simple explanation: either bit can spoil the parity prediction. More generally,

$$|v_1{v}_2|\le min(|v_1|,|v_2|).$$

A parity gate draws its output inward from the cone edge. In contrast, two corroborating same-bit messages push the result outward.

Try a prediction before moving on. One reading becomes completely uninformative, with $v_2=0$. What should the parity message do, however excellent the first reading might be?

Multiplication answers immediately: $v_{-}=0$. Knowing one bit perfectly leaves the parity of that bit and a fair unknown bit equally split. The geometry has found the missing piece in the logical question.

The same reasoning extends to a parity question involving $k$ independent bit beliefs:

$$v_{\mathrm{parity}}=\prod _{j=1}^k{v}_j.$$

If every magnitude is , the output magnitude is $q^k$. A long parity relation can be logically powerful while its first message is very weak. This gives us a quantitative reason to care about how a code distributes its constraints.

Let the other bits speak

Return to the spacecraft’s even-parity promise. Its valid words are

$$000,011,101,110.$$

Choose these four words with equal prior probability. Each bit passes through its own independent noisy channel. To improve the third bit’s reading, use the promise

$$X_3=X_1\mathrm{xor}{X}_2.$$

The first two readings therefore send a parity message about the third bit. Call its rapidity $c_3$, and add it to the third bit’s own channel rapidity:

$$\begin{array}{c}{c}_3=\mathrm{artanh}(\tanh r_1\tanh r_2),\\ r_{3,\mathrm{out}}=r_3+c_3.\end{array}$$

The check uses the other two readings. Its message is called extrinsic: information arriving from outside the destination’s own observation. This preserves the lineage of each contribution.

Suppose the receiver sees 001, the first two channels flip with probability $0.1$, and the third flips with probability $0.35$. The third reading alone gives probability $0.35$ to a hidden zero. After the check message, that probability rises to about $0.7104$. The code’s promise has made the blurry reading interpretable.

A parity check in the evidence cone

Help three bits keep their promise

Restore noisy third bit

The sender chooses equally among 000, 011, 101, and 110. Each word has an even number of ones. Click a received bit to change the reading, choose its channel's flip probability, then step a message toward your selected bit.

Received Y1 0Click to flip Flip probability 5%10%20%35%50%

Incoming r1: +1.099

Send message here

Received Y2 0Click to flip Flip probability 5%10%20%35%50%

Incoming r2: +1.099

Send message here

Received Y3 1Click to flip Flip probability 5%10%20%35%50%

Incoming r3: -0.310

Selected destination

Positive rapidity favors a hidden 0; negative rapidity favors a hidden 1. Each channel flips independently. A 50% flip channel contributes zero rapidity.

1. Read messages2. Apply parity3. Combine

Bit 3 hears its own channel

Its current rapidity is -0.310. The other two readings will now be combined through the even-parity promise.

$$r_{\mathrm{in}}=\frac{1}{2}\ln \frac{W(y\mid 0)}{W(y\mid 1)}$$

Next message

The demonstration also does the slow calculation: assign each valid word a weight equal to its prior probability times its three channel likelihoods, normalize the weights, and sum those with the desired bit value. Its answer agrees with message passing.

A factor graph draws variables and local relationships as separate kinds of nodes. A variable node combines incoming messages about one variable; a parity-check node enforces a relationship between several variables. This little graph is a tree, so its messages recover the exact marginal probabilities. The general construction is the sum-product algorithm of Kschischang, Frey, and Loeliger.

Seen through the cone, the two node types alternate addition of rapidities and multiplication of velocities. We have translated a working inference circuit into two geometric instructions.

Turn two wires into two questions

Now imagine designing the spacecraft’s encoder as well as its receiver. Start with independent fair bits $U_1,U_2$, and send

through independent channels $W_1,W_2$. The transformation is invertible: knowing $X_1,X_2$ lets us recover both original bits.

Decode in an interesting order. First ask for $U_1$, the parity of the transmitted bits. Then ask for $U_2$, using $U_1$ as side information. These define two synthetic channels, usually called minus and plus. A synthetic channel describes one inferential question about the combined experiment.

For fixed readings $y_1,y_2$, abbreviate the individual likelihoods as $a_t=W_1(y_1\mid t)$ and $b_t=W_2(y_2\mid t)$, where $t$ is an input bit. For the minus channel, average over the still-unknown fair $U_2$:

$$\begin{array}{c}{W}^{-}(y_1,y_2\mid u_1)\\ =\frac{1}{2}\sum _{u_2=0}^1{a}_{u_1\mathrm{xor}{u}_2}{b}_{u_2}.\end{array}$$

For the plus channel, the output includes the true first bit:

$$\begin{array}{c}{W}^{+}(y_1,y_2,u_1\mid u_2)\\ =\frac{1}{2}{a}_{u_1\mathrm{xor}{u}_2}{b}_{u_2}.\end{array}$$

Thus the two decoding equations are

The side information tells us whether to reverse the first rapidity before adding. If $u_1=0$, the transmitted bits agree. If $u_1=1$, they differ, so evidence for $X_1=0$ becomes evidence for $U_2=1$.

This is the elementary transform underlying Arıkan’s polar codes. A practical successive decoder inserts its earlier decision where the synthetic definition supplies the true bit. The definition lets us measure the individual questions; the running decoder must also manage the consequences of earlier mistakes.

The first polar transform

Two channels trade difficulty

Restore 10% and 20% flips

Send two fair, independent bits through two noisy wires. The encoder sends their parity on the first wire and the second bit on the second wire. Follow one possible journey, then inspect the information across every possible output.

Wire 1: flip probability 1%5%10%20%35%50%Wire 2: flip probability 1%5%10%20%35%50%

1. Encode2. Transmit3. Read parity4. Use side information

U1 = 0Click to changeU2 = 0Click to change

Encoded pair: 0 0

Before encoding, U1 and U2 each have probability ½ of being 0. These buttons choose a particular example from that input ensemble.

Next step

Account for all outputs

These channel averages include all fair inputs and all noise outcomes. They depend on the two flip probabilities, so choosing one journey leaves them unchanged.

Before: two wires

0.80908 bits

After: two questions

0.80908 bits

Bars use a common scale from 0 to 2 bits. Sky: wire 1; apricot: wire 2. Rose: parity question; teal: second bit with side information.

Minus: learn the parity

Information I

0.17325 bits

Average normalized mass Z

0.87727

Output reliability |v|, grouped by value

|v| = 0.480

100.0% of output probability

Plus: use the parity

Information I

0.63582 bits

Average normalized mass Z

0.48000

Output reliability |v|, grouped by value

|v| = 0.385

26.0% of output probability

|v| = 0.946

74.0% of output probability

$$I^{-}+I^{+}=I_1+I_2$$

0.17325 + 0.63582 = 0.53100 + 0.27807 bits

$$Z^{+}=Z_1{Z}_2$$

0.48000 = 0.60000 × 0.80000

The exact plus channel retains eight labeled outputs (true U1, Y1, Y2). The display groups outputs with equal |v|. Agreement and disagreement can have different reliabilities, so the plus channel carries a distribution of cone directions into the next stage.

The information budget survives the rearrangement

Let $I(A;B)$ denote mutual information, the average reduction in uncertainty about $A$ supplied by $B$, measured here in bits. Write $Y=(Y_1,Y_2)$ for the combined observations.

The mutual-information chain rule splits learning about a pair into learning about the first member, then the second:

$$\begin{array}{c}I(U_1,U_2;Y)=\\ I(U_1;Y)+I(U_2;Y\mid U_1).\end{array}$$

Because $U_1$ and $U_2$ start independently, the two terms are exactly the minus and plus channel information. Invertible encoding preserves the information about the input pair. Independence of the two physical channels then gives

$$\boxed{\begin{array}{c}I(W^{-})+I(W^{+})\\ =I(W_1)+I(W_2)\end{array}}.$$

For flip probabilities $0.1$ and $0.2$, the demo obtains

We have rearranged which question gets the information. Repeating such transforms creates the polarization that gives polar codes their name: many synthetic questions become nearly useless, while others become nearly noiseless. Information can be placed in the good positions, with predetermined values in the poor ones. Arıkan’s construction establishes the resulting coding theorem.

Mass keeps a second set of accounts

The normalized cone mass is

$$z=\frac{m}{\tau }=\sqrt{1-v^2}.$$

Across a channel’s output distribution, its average is the Bhattacharyya overlap $Z(W)$, which measures how much the two conditional output laws overlap:

$$\begin{array}{c}Z(W)\\ =\sum _y\sqrt{W(y\mid 0)W(y\mid 1)}.\end{array}$$

Insert the plus channel’s likelihoods. For either value of its side-information bit $u_1$, the square root contains

$$\begin{array}{c}\frac{1}{2}\sqrt{W_1(y_1\mid 0)W_1(y_1\mid 1)}\\ \cdot \sqrt{W_2(y_2\mid 0)W_2(y_2\mid 1)}.\end{array}$$

Summing over both $u_1$ values removes the half; summing over the two outputs separates into a product. Therefore

$$\boxed{Z(W^{+})=Z(W_1)Z(W_2)}.$$

For binary symmetric input channels, where each wire independently flips its bit with a fixed probability, the minus identity follows from $v^{-}=v_1{v}_2$:

$$\boxed{(Z^{-}{)}^2=Z_1^2+Z_2^2-Z_1^2{Z}_2^2}.$$

There is a lovely detail hidden in the plus average. With the demo’s $10\mathrm{\%}$ and $20\mathrm{\%}$ flip channels, the plus outputs have two reliability magnitudes: about $0.946$ with total probability $0.74$, and $0.385$ with probability $0.26$. Agreement and disagreement produce different cone directions. At equal flip probabilities, disagreement can produce exactly zero rapidity.

To carry the construction onward, carry that distribution onward too. Each output has its own direction and weight; their average mass alone cannot describe the next parity operation. The demo preserves the full eight-output plus channel and groups equal reliability magnitudes only for display.

Could a decoder become a little epistemic machine?

Picture a lattice of these gates. At one kind of junction, compatible reports add boosts. At another, a logical relationship multiplies velocities and sends the result along a new route. Constraints become devices for steering likelihoods through the cone.

This suggests a concrete design experiment. Give a network several parity checks of different sizes, but the same budget of connections. Predict early message strength from the products of their incoming velocities. Short checks should transmit stronger first messages; longer checks can connect otherwise distant pieces of the word. The interesting object is the tradeoff between local message strength and global constraint coverage.

A second experiment would let the machine choose where to measure next. A nearly zero incoming velocity can silence a whole parity message. Improving that weak reading might therefore help several distant bits at once. We could rank possible new trials by the information they would release through the constraint network, then compare that policy with simply remeasuring the locally blurriest bit.

The equations give us a small construction kit: rapidity for accumulating compatible observations, velocity for composing parity, and normalized mass for tracking the resulting ambiguity. The next adventure is to let that geometry help decide where the receiver should look.