A coin, a magnet, and a learning machine

Picture three rooms situated along the same laboratory hallway.

In the first room, a statistician flips a fair coin repeatedly, logging the sequence of heads and tails in a notebook.

In the second room, a condensed matter physicist places a crystal of tiny magnetic dipoles inside a variable magnetic field, heating and cooling it in a thermal bath.

In the third room, a computer scientist trains a neural network on text, adjusting synaptic weights so the machine can predict whether an incoming sentence ends with a question mark.

At first glance, these three researchers seem to inhabit entirely separate scientific worlds. One is studying combinatorial chance, another is measuring thermodynamics, and the third is optimizing artificial intelligence.

Yet when I set their mathematical models side by side, an astonishing realization emerges: all three rooms are running the exact same equations. Each room investigates a binary event with two possible outcomes. Each room parameterizes that event using the hyperbolic rapidity coordinate of spacetime. And each room discovers that the curvature of its underlying potential is governed by Einstein’s relativistic Lorentz factor.

Let us walk through all three rooms together, following a single binary trial as it evolves from a counting puzzle into a thermodynamic magnet, an optimal data compressor, and an adaptive learning machine.

The three rooms: Coins, spins, and bits

Let us establish a universal mathematical notation for all three rooms.

Consider any elementary trial with two mutually exclusive outcomes, which we label positive ($+1$) and negative ($-1$).

Let $p\in (0,1)$ denote the probability of the positive outcome, so that $1-p$ is the probability of the negative outcome.

In Epistemic Physics, we frequently switch between three complementary coordinates that describe this bias:

1. The raw probability $p\in (0,1)$.
2. The centered epistemic velocity $v\in (-1,1)$, which maps balance to the origin: $$v=2p-1=\mathbb{E}[S],$$ where $S\in \{-1,+1\}$ is the discrete random outcome.
3. The epistemic rapidity $\rho \in (-\mathrm{\infty },\mathrm{\infty })$, which represents half the log-odds: $$\rho =\frac{1}{2}\ln \left(\frac{p}{1-p}\right).$$

Inverting these definitions expresses probability and velocity in terms of rapidity:

$$p=\frac{e^{\rho }}{e^{\rho }+e^{-\rho }}=\frac{1}{1+e^{-2\rho }},v=\tanh \rho .$$

Now recall the relativistic Lorentz factor $\gamma (v)=1/\sqrt{1-v^2}$. Using the fundamental hyperbolic identity $1-{\tanh }^2\rho =1/{\cosh }^2\rho$:

$$\gamma =\frac{1}{\sqrt{1-{\tanh }^2\rho }}=\cosh \rho =\frac{e^{\rho }+e^{-\rho }}{2}.$$

Keep this hyperbolic cosine $\cosh \rho$ in mind: it is the master key that unlocks every room in the laboratory.

Counting histories: Microstates and Shannon entropy

Let us step inside the first room, where our statistician is recording coin flips.

Suppose a notebook contains a summary of $n$ trials, recording $Y$ positive outcomes (heads) and $N$ negative outcomes (tails), such that $n=Y+N$.

The summary records only the total counts, not the chronological sequence in which they arrived. How many distinct ordered histories (microstates) could have produced that exact summary?

This is the elementary binomial coefficient:

$$\mathrm{\Omega }(Y,N)=(\genfrac{}{}{0ex}{}{n}{Y})=\frac{n!}{Y!N!}=\frac{n!}{Y!(n-Y)!}.$$

Here the exclamation mark denotes the factorial ($n!=n\times (n-1)\times \cdots \times 1$).

Notice how the number of compatible histories changes with balance:

• If a sequence of six trials yields six heads and zero tails, there is only $(\genfrac{}{}{0ex}{}{6}{6})=1$ possible history: $(+,+,+,+,+,+)$.
• If the six trials split evenly into three heads and three tails, the number of compatible histories explodes to $(\genfrac{}{}{0ex}{}{6}{3})=\frac{6\times 5\times 4}{3\times 2\times 1}=20$ distinct orderings.

Balance opens up maximal combinatorial freedom.

When the number of recorded trials $n$ becomes large, we can approximate the logarithm of the microstate count using Stirling’s asymptotic formula, $\ln (k!)\approx k\ln k-k$:

$$\ln \mathrm{\Omega }(Y,N)\approx (n\ln n-n)-(Y\ln Y-Y)-(N\ln N-N)=n\ln n-Y\ln Y-N\ln N.$$

Divide both sides by $n$, and substitute the observed frequencies $p=Y/n$ and $1-p=N/n$:

$$\frac{1}{n}\ln \mathrm{\Omega }\approx \ln n-\frac{Y}{n}\ln Y-\frac{N}{n}\ln N=-\frac{Y}{n}\ln \left(\frac{Y}{n}\right)-\frac{N}{n}\ln \left(\frac{N}{n}\right).$$

This yields Claude Shannon’s famous binary entropy function, $h(p)$:

$$\boxed{\frac{1}{n}\ln \mathrm{\Omega }\approx h(p)=-p\ln p-(1-p)\ln (1-p).}$$

Binary entropy measures the logarithmic volume of possible histories per trial. When natural logarithms are used, entropy is measured in nats; dividing by $\ln 2$ converts the units into bits.

Entropy peaks at complete balance ($p=1/2$), where $h(1/2)=\ln 2\approx 0.693\text{ nats}=1.0\text{ bit}$. At the boundaries ($p\to 0$ or $p\to 1$), entropy plunges to zero: a completely deterministic process produces zero surprise and permits only a single history.

The history cabinet

Make four histories from one notebook

Reset six trials

Tap a head and a tail to swap them. Find 4 distinct orderings while preserving the notebook's totals.

The notebook always reads 3 H and 3 T

HHHTTT

Choose an H, then a T. Their positions will trade.

Your history album · 1/20 slots

HHHTTT

There are 20 compatible histories in total. Album slots are a display limit, not a limit on possible histories.

Change the notebook and count all possible histories

Notebook size

4 trials6 trials8 trials12 trials24 trials

One fewer headOne more head

Exact log-count / trial

0.720 bits

Shannon limit / trial

1.000 bits

$$\mathrm{\Omega }=(\genfrac{}{}{0ex}{}{n}{Y}),{\log }_2\mathrm{\Omega }/n⟶H_2(Y/n)$$

A Beta ledger for independent identically distributed trials retains counts, so all these orders yield the same final weights. Their tracelet order is still different. Exact finite log-count and the Shannon large-sample expression differ. An all-heads or all-tails notebook has only one history.

Thermal physics: The paramagnet in a heat bath

Now walk into the second room. Put away the coin, and inspect the physicist’s apparatus: a collection of non-interacting magnetic atoms (a paramagnet).

Each atom has an intrinsic magnetic dipole moment that can point in one of two directions along an applied magnetic field: spin up ($s=+1$) or spin down ($s=-1$).

An external magnetic field applies an energy bias $\epsilon$. The energy of a spin state is:

$$E_s=-\epsilon s.$$

If , pointing up is energetically favored ($E_{+1}=-\epsilon$), while pointing down is penalized ($E_{-1}=+\epsilon$).

The crystal sits inside a thermal reservoir at absolute temperature . According to the Boltzmann distribution of statistical mechanics, the thermal probability of finding a dipole in state $s$ is proportional to its Boltzmann factor $e^{-E_s/(k_{\mathrm{B}}T)}$, where $k_{\mathrm{B}}\approx 1.38\times {10}^{-23}\text{ J/K}$ is Boltzmann’s constant.

The sum of all state weights is the canonical partition function, denoted $Z$:

$$Z=\sum _{s\in \{-1,+1\}}{e}^{-E_s/(k_{\mathrm{B}}T)}=e^{\epsilon /(k_{\mathrm{B}}T)}+e^{-\epsilon /(k_{\mathrm{B}}T)}.$$

Now compare this formula with the rapidities from our first room!

Define the dimensionless physical rapidity as the ratio of magnetic field energy to thermal energy:

$$\rho =\frac{\epsilon }{k_{\mathrm{B}}T}.$$

Substituting $\rho$ into the partition function:

$$Z=e^{\rho }+e^{-\rho }=2\cosh \rho .$$

Look at that: the physical partition function is twice the relativistic Lorentz factor!

The thermal probability of finding a spin pointing up is:

$$p=\frac{e^{\epsilon /(k_{\mathrm{B}}T)}}{Z}=\frac{e^{\rho }}{e^{\rho }+e^{-\rho }}=\frac{1}{1+e^{-2\rho }}.$$

The net magnetization (the average alignment per spin) is:

$$⟨s⟩=(+1)p+(-1)(1-p)=2p-1=\tanh \rho =v.$$

The magnetic alignment of a crystal is mathematically identical to the centered epistemic velocity of a coin!

When the temperature is high ($k_{\mathrm{B}}T\gg \epsilon$), thermal agitation dominates: rapidity vanishes ($\rho \to 0$), and the spins point up and down in equal numbers ($v=0$). When the temperature is low ($k_{\mathrm{B}}T\ll \epsilon$), the magnetic field freezes the dipoles into uniform alignment ($\rho \to \mathrm{\infty },v\to 1$).

Thermodynamic potentials: The Lorentz factor as curvature

In statistical physics, all thermodynamic properties of a system can be derived by differentiating a characteristic potential: the free energy, or its dimensionless counterpart, the log-partition function:

$$\mathrm{\Lambda }(\rho )=\ln \left(\frac{Z}{2}\right)=\ln \cosh \rho .$$

Let us differentiate this potential with respect to rapidity $\rho$:

The first derivative gives the expected velocity (magnetization):

$${\mathrm{\Lambda }}^{\prime }(\rho )=\frac{d}{d\rho }\ln \cosh \rho =\frac{\sinh \rho }{\cosh \rho }=\tanh \rho =v.$$

The second derivative gives the response sensitivity (magnetic susceptibility, or variance):

$${\mathrm{\Lambda }}^{″}(\rho )=\frac{d}{d\rho }\tanh \rho =1-{\tanh }^2\rho =1-v^2=\frac{1}{{\gamma }^2}.$$

Look at that identity:

$$\boxed{{\mathrm{\Lambda }}^{″}(\rho )=\frac{1}{{\gamma }^2}.}$$

The curvature of the thermodynamic log-partition function is the inverse square of the Lorentz factor!

Near the balanced center ($v=0,\gamma =1$), the curvature is at its maximum (${\mathrm{\Lambda }}^{″}=1$): a tiny increase in the applied field $\rho$ produces a large, immediate change in alignment. Near saturation ($|v|\to 1,\gamma \to \mathrm{\infty }$), the curvature vanishes: the spins are already pinned, and further field increases produce almost no change.

Now let us perform a Legendre transformation to switch perspectives. Instead of driving the system with an external force $\rho$, suppose we specify the internal state $v$ and ask for the corresponding entropy cost.

Define the entropy deficit function $\mathrm{\Phi }(v)=\ln 2-h((1+v)/2)$, which measures how much order has been imposed relative to maximum entropy:

$$\mathrm{\Phi }(v)=v\rho -\mathrm{\Lambda }(\rho )=v\mathrm{artanh}(v)-\ln \cosh (\mathrm{artanh}v).$$

Differentiating $\mathrm{\Phi }(v)$ with respect to velocity $v$:

$${\mathrm{\Phi }}^{\prime }(v)=\rho =\mathrm{artanh}(v).$$

Differentiating once more gives the stiffness (the resistance of the system to being displaced from balance):

$$\boxed{{\mathrm{\Phi }}^{″}(v)=\frac{d}{dv}\mathrm{artanh}(v)=\frac{1}{1-v^2}={\gamma }^2.}$$

The two curvatures are exact reciprocal partners:

$${\mathrm{\Lambda }}^{″}(\rho )\times {\mathrm{\Phi }}^{″}(v)=\left(\frac{1}{{\gamma }^2}\right)\times ({\gamma }^2)=1.$$

The curvature of order in state space is the relativistic Lorentz factor squared!

Where the system is easy to move in rapidity, it is flat in entropy deficit. Near the speed of light, the entropy deficit approaches the finite ceiling $\ln 2$, while its slope and curvature climb toward infinity. The surprise is an infinitely steep edge to a finite hill: absolute certainty demands unbounded rapidity!

Optimal coding: Cross-entropy and the cost of being wrong

Now enter the third room. Here we meet the computer scientist, who is compressing data and training machine learning models.

Suppose the true environment produces positive outcomes with probability $p$. However, our observer or predictive model holds a subjective belief that positive outcomes occur with probability $q$.

According to Shannon’s source coding theorem, if an encoder designs an optimal prefix code tailored to an expected probability $q$, the ideal code length for an outcome that was expected with probability $q$ is:

$$L=-\ln q\text{ nats}=-{\log }_{2}q\text{ bits}.$$

Outcomes expected to be common are assigned short code words; outcomes expected to be rare receive long code words.

Now, what is the expected code length per trial when the true data source actually produces outcomes according to probability $p$?

This expected cost is the cross-entropy, $C(p,q)$:

$$C(p,q)=-p\ln q-(1-p)\ln (1-q).$$

We can decompose cross-entropy into two distinct components:

$$C(p,q)=h(p)+D_{\mathrm{KL}}(p\parallel q).$$

Here $h(p)=-p\ln p-(1-p)\ln (1-p)$ is the true binary entropy of the source: the unavoidable, irreducible randomness of the physical process itself.

The excess penalty, $D_{\mathrm{KL}}(p\parallel q)$, is the Kullback-Leibler (KL) divergence, also called relative entropy:

$$D_{\mathrm{KL}}(p\parallel q)=p\ln \left(\frac{p}{q}\right)+(1-p)\ln \left(\frac{1-p}{1-q}\right).$$

Look at what KL divergence measures: it is the informational tax you pay for using the wrong model of reality.

By Gibbs’ inequality, $D_{\mathrm{KL}}(p\parallel q)\ge 0$ always, with equality if and only if $q=p$.

The code kitchen

Pack a message with fewer wasted bits

Reset recipes

Pick a prediction recipe for this source. Open its coding bill and see which part of the surprise you can remove.

Source 1 of 3

The source sends plus 80% of the time.

20% plusPrediction recipe50% plusPrediction recipe80% plusPrediction recipe

Open the coding bill

Free recipe mixing and the thermal connection

Thermal bias ρ

0.69

Recipe q

50.0%

Use the source's exact recipe $$p=\frac{1}{1+e^{-2\rho }},C(p,q)=H(p)+D_{\mathrm{K}\mathrm{L}}(p\Vert q)$$

ρ is field energy divided by thermal energy in the two-state model. Costs here use base-two logs and are expected ideal code lengths for long independent blocks. There are no sampled trials or invented game points. The bars share one cost scale. Choosing a new recipe closes its bill until you inspect it.

Test this in the interactive lab above. Adjust the true probability $p$ and the model’s prediction $q$. Watch how the total message length expands when the model’s expectations diverge from reality.

Let us see this in concrete numbers:

Suppose the world produces positive outcomes $90\mathrm{\%}$ of the time ($p=0.90$). The irreducible entropy of this source is:

$$h(0.90)=-(0.90){\log }_2(0.90)-(0.10){\log }_2(0.10)\approx 0.469\text{ bits per trial}.$$

1. If the model knows the true distribution ($q=0.90$), its average description length is $0.469\text{ bits}$. The penalty is zero.
2. If the model is completely agnostic ($q=0.50$), it assigns 1 bit to every outcome. The penalty is $D_{\mathrm{KL}}=1.000-0.469=0.531\text{ bits}$ wasted per trial.
3. If the model is dogmatically wrong, predicting positive outcomes only $10\mathrm{\%}$ of the time ($q=0.10$), the expected code length leaps to: $$C(0.90,0.10)=-(0.90){\log }_2(0.10)-(0.10){\log }_2(0.90)\approx 3.005\text{ bits per trial}.$$ The penalty is $D_{\mathrm{KL}}\approx 3.005-0.469=2.536\text{ bits}$ wasted on every single observation!

Dogmatic misprediction is immensely expensive.

Gradient descent: How learning machines follow hyperbolic curves

How does an artificial neural network fix its mistakes?

Let the machine represent its prediction using its own internal rapidity parameter, $\eta$ (eta):

$$\eta =\frac{1}{2}\ln \left(\frac{q}{1-q}\right)⟹q=\frac{e^{\eta }}{e^{\eta }+e^{-\eta }},v_q=\tanh \eta .$$

Here $\eta$ is an unconstrained parameter generated by the network’s final linear layer. It is half the usual binary logit $z=\ln [q/(1-q)]$: $z=2\eta$.

Now express the cross-entropy loss $C(p,q)$ directly as a function of the model’s internal parameter $\eta$:

$$C(p,\eta )=-p\ln \left(\frac{e^{\eta }}{e^{\eta }+e^{-\eta }}\right)-(1-p)\ln \left(\frac{e^{-\eta }}{e^{\eta }+e^{-\eta }}\right).$$

Using elementary algebra:

$$C(p,\eta )=-p[\eta -\ln (2\cosh \eta )]-(1-p)[-\eta -\ln (2\cosh \eta )].$$

Combining terms:

$$C(p,\eta )=\ln (2\cosh \eta )-(2p-1)\eta =\ln (2\cosh \eta )-v\eta .$$

Look at that formula:

Cross-entropy loss in machine learning uses the same conjugate expression that builds the Legendre transform of a thermal paramagnet! The transform maximizes $v\eta -\ln (2\cosh \eta )$ over $\eta$; minimizing its negative, our loss, gives $h(p)$. At any prediction, the excess above that minimum is exactly the mismatch penalty $D_{\mathrm{KL}}(p\parallel q)$.

Now compute the gradient of this loss with respect to the network’s parameter $\eta$:

$$\frac{\partial C}{\partial \eta }=\frac{d}{d\eta }\ln (2\cosh \eta )-v=\tanh \eta -v=v_q-v.$$

Look at how extraordinarily simple and elegant this gradient is:

$$\boxed{\frac{\partial C}{\partial \eta }=v_q-v.}$$

The learning force that drives gradient descent is simply the difference between the model’s predicted velocity $v_q$ and the environment’s true velocity $v$!

When the training data consists of individual discrete observations $S\in \{-1,+1\}$, the stochastic gradient on a single trial is:

$$g=v_q-S.$$

If the network predicts $v_q=+0.80$ and observes a positive trial ($S=+1$), the gradient is a small adjustment: $g=0.80-1.00=-0.20$.

If instead the network encounters an unexpected negative trial ($S=-1$), the gradient is a massive kick: $g=0.80-(-1.00)=+1.80$.

The machine takes a large stride in rapidity precisely when an observation strongly contradicts its prediction.

Rapidity steering

Steer a learner into the prediction gate

Reset course

Choose the size of each learning step. Land within 0.01 extra bits per trial of the source. Big steps can cross the target.

Course 1 of 3 · Cross the track · step 0

●

0%100%

Apricot learner

10.0%

Sky target

90%

Small stepRapidity gain 0.25Steady stepRapidity gain 1Bold stepRapidity gain 2.8

The learner starts far from the target. How large a first step would you choose?

Undo last stepNext course

Inspect your route and the gradient rule

Custom target

90%

Custom starting forecast

10%

0: 10.0%

$$g=\tanh \eta -(2p-1),{\eta }_{\mathrm{n}\mathrm{e}\mathrm{x}\mathrm{t}}=\eta -\lambda g$$

Current rapidity η = -1.099; gradient g = -1.600.

These are deterministic gradient steps against the source's expected outcome, rather than sampled trials or Beta evidence accumulation. The target measures excess expected log loss. A large gain can overshoot and need not reduce loss every time. Your route is capped at 30 steps; undo and replay remain available.

Spreading convictions: A physical lattice of learning minds

We have now traveled through all three rooms:

• In the coin room, we discovered that microstate volume generates Shannon entropy $h(p)$.
• In the magnet room, we found that thermal partition functions $Z=2\cosh \rho$ govern physical dipoles, with response sensitivity given by the inverse Lorentz factor ${\mathrm{\Lambda }}^{″}={\gamma }^{-2}$.
• In the neural room, we showed that cross-entropy loss reproduces the exact same potential, driving gradient descent through the velocity error $v_q-v$.

Now imagine combining all three rooms into a single physical system: a distributed lattice of learning agents.

Picture a two-dimensional grid where each node maintains an evidence notebook $(\alpha ,\beta )$. Each node observes an incoming stream of noisy binary tracelets from its local environment, while communicating with its immediate neighbors.

When neighboring nodes disagree, the cross-entropy mismatch generates an informational shear force proportional to $v_i-v_j$. This force drives their internal rapidities toward consensus, exactly like the exchange interaction between magnetic spins in an Ising model:

• At low informational temperature, the lattice exhibits spontaneous symmetry breaking: agents coalesce into vast, coherent domains of shared conviction.
• At high temperature, random noise disrupts communication, dissolving the lattice into an uncommitted paramagnetic gas.
• At the critical temperature, long-range correlations emerge: a single surprising observation at one corner can propagate like a phase wave across the entire collective network.

Such a world possesses three distinct physical stiffnesses:

• Saturation stiffness: An element near certainty responds weakly to bias changes because its susceptibility vanishes (${\gamma }^{-2}\to 0$).
• Inertial stiffness: A thick notebook with massive epistemic rest mass $m_{\mathrm{E}}$ turns slowly because each incoming trial is divided by $\tau +1$.
• Information tension: A mismatched prediction incurs a steep cross-entropy cost, generating mechanical pressure to realign.

These are distinct physical mechanisms that we can place side by side and simulate.

Build the smallest toy version: two neighboring notebooks, a shared stream of binary events, and a coupling proportional to prediction mismatch. Watch whether agreement spreads like heat, travels like a wave, or freezes into competing domains.

Coins, magnets, and neural networks are not merely loose metaphors for one another. They are distinct physical manifestations of the same underlying information geometry. The equations have given us parts for a little physical world.

In our next excursion, we will follow this geometry into the cosmic fabric of spacetime in A universe written in two inks, where positive and negative evidence lay down the light cones of physical reality.