--- title: "How much can one trial teach?" author: "Miraia S. Chiou" creator: "Miraia Chiou" published: "2026-09-13" identifier: "ep_field_0e57d074738d48a585f77faa80c55c43" source: "https://epistemicphysics.com/fields/information-theory/how-much-can-one-trial-teach/" citation: "https://epistemicphysics.com/cite/?work=ep_field_0e57d074738d48a585f77faa80c55c43" --- # How much can one trial teach? Separate surprise from learning, watch mass cancel Fisher curvature, and find pi in the price of coding an unknown coin. [Read the illustrated article](https://epistemicphysics.com/fields/information-theory/how-much-can-one-trial-teach/) Three expedition notebooks predict a coin will land heads half the time. The first belongs to someone who has barely met the coin. The second records a modest collection of trials. The third has watched it for ages. Tomorrow’s flip still catches all three off guard: heads and tails remain equally likely. Would that flip teach each notebook the same amount? The question sends us across a useful border. **Predicting the next outcome** and **learning the mechanism that produces outcomes** are different jobs. Epistemic Physics keeps a coordinate for each side of this distinction: the mean tells us where a belief points, while its total trial amount helps determine how much it can change. We will find two equations waiting at that border. In the first, normalized mass cancels Fisher information, leaving a simple scale for learning from one trial. In the second, the Fisher length $\pi$ reappears in the extra bits needed to encode a coin whose bias we must learn. ## A coin inside a notebook Call the coin’s unknown heads rate $B$. Put a Beta distribution over it: $$ B\sim\operatorname{Beta}(\alpha,\beta). $$ The positive shapes $\alpha,\beta$ include the prior and whatever supporting and opposing trial amounts the notebook has collected. Its mean and total amount are $$ \mu=\frac{\alpha}{\alpha+\beta}, \qquad \tau_{\mathrm E}=\alpha+\beta. $$ A fresh outcome $S$ is 1 for heads and 0 for tails. Given $B=b$, heads has probability $b$. Averaging over the notebook’s possible rates gives $\Pr(S=1)=\mu$. We use natural logarithms first. **Surprisal** is $-\ln$ of an outcome’s predicted probability. Its average is **entropy**: $$ h(p)=-p\ln p-(1-p)\ln(1-p). $$ Natural logarithms measure information in **nats**. Divide by $\ln2$ to obtain bits. At $p=1/2$, either outcome has one bit of surprisal and the outcome entropy is one bit. Now imagine a friendly laboratory sprite whispers the exact value of $B$ before the flip. You still cannot generally predict the outcome perfectly. A coin with known rate $b$ retains entropy $h(b)$. The average advantage of hearing the sprite’s answer is therefore $$ I(B;S)=h(\mu)-\mathbb E[h(B)]. $$ This quantity is **mutual information**: how much learning one variable reveals about the other, on average. Read the equation in either direction. Knowing the rate helps predict the flip; seeing the flip helps learn the rate. The same joint distribution supplies both readings. The three notebooks can now submit different answers: | Notebook | Next-outcome entropy | Expected learning about the rate | | ------------------------------ | -------------------- | -------------------------------- | | $\operatorname{Beta}(1/2,1/2)$ | 1 bit | 0.442695 bits | | $\operatorname{Beta}(5,5)$ | 1 bit | 0.068546 bits | | $\operatorname{Beta}(50,50)$ | 1 bit | 0.007177 bits | The last notebook has nearly settled the rate while leaving the individual flip unpredictable. Balanced support and opposition can make a belief resistant to change even when its next answer remains a toss-up. [One outcome, two amounts of learning](https://epistemicphysics.com/fields/information-theory/how-much-can-one-trial-teach/#trial-information-lab) [Open the trial-information experiment in the playground](https://epistemicphysics.com/lab/?demo=trial-information#experiment). ## Surprise opens one door; learning measures the step After heads, the notebook becomes $\operatorname{Beta}(\alpha+1,\beta)$. After tails, it becomes $\operatorname{Beta}(\alpha,\beta+1)$. These are the familiar supporting and opposing trial updates. To measure how far an update moves the whole probability distribution, use **Kullback-Leibler divergence**, abbreviated KL. For a posterior density $f_s(b)$ and starting density $f(b)$, define $$ G_s=\int_0^1 f_s(b)\ln\frac{f_s(b)}{f(b)}\,db. $$ The subscript $s$ identifies the observed outcome. $G_s$ is the realized information gain for that outcome, with the posterior on the left side of the density ratio. Bayes’ rule makes the ratio beautifully simple. If the outcome is heads, $$ \frac{f_1(b)}{f(b)}=\frac b\mu. $$ The posterior favors rates that gave the observed heads greater probability. Averaging the logarithm of this ratio over possible heads and tails gives $$ \mathbb E_S[G_S]=I(B;S). $$ So the gold quantity in the demo can be calculated two ways: subtract the entropy left after knowing the rate, or average the KL movement caused by each possible observation. The expected-information viewpoint is central to [Lindley’s measure of what an experiment provides](https://www.mathnet.ru/eng/mat101). Try the demo at a predicted heads probability of $90\%$. Tails has surprisal $-\log_2(0.1)\approx3.322$ bits in both notebooks. Its posterior KL depends strongly on their trial amounts. An outcome can be a rare visitor without rearranging much furniture. ### An exact Beta calculation The Beta density’s normalizing integral is $$ \mathcal B(\alpha,\beta)= \int_0^1 b^{\alpha-1}(1-b)^{\beta-1}\,db. $$ Differentiating a power brings down a logarithm. For example, differentiating the same integral with $\alpha$ replaced by $\alpha+1$ gives the average of $B\ln B$: $$ \mathbb E[B\ln B] =\mu\,[\psi(\alpha+1)-\psi(\tau_{\mathrm E}+1)]. $$ Here $\psi(x)=d\ln\Gamma(x)/dx$ is the **digamma function**, the derivative of the logarithm of the Gamma function. The Gamma function extends factorials; $\Gamma(n+1)=n!$ for whole $n$. Digamma packages the logarithmic averages our calculation needs. Repeating the step for $(1-B)\ln(1-B)$ yields $$ \begin{aligned} \mathbb E[h(B)] ={}&\psi(\tau_{\mathrm E}+1)\\ &-\mu\psi(\alpha+1)\\ &-(1-\mu)\psi(\beta+1). \end{aligned} $$ For one revealed heads, the exact information gain is $$ G_1=\psi(\alpha+1)-\psi(\tau_{\mathrm E}+1)-\ln\mu. $$ Replace $\alpha$ with $\beta$ and $\mu$ with $1-\mu$ for tails. These formulas let the demo separate an outcome’s surprisal from its actual posterior movement. ## Watch mass cancel curvature Now bring the notebook into the evidence cone. Its centered mean is the velocity-like coordinate $$ v_{\mathrm E}=2\mu-1. $$ Write normalized mass as $z$ and the Lorentz factor as $\gamma_{\mathrm E}$: $$ \begin{aligned} z&=\sqrt{1-v_{\mathrm E}^2}\\ &=2\sqrt{\mu(1-\mu)},\\ \gamma_{\mathrm E}&=\frac1z. \end{aligned} $$ The Beta variance then becomes $$ \operatorname{Var}(B) =\frac{\mu(1-\mu)}{\tau_{\mathrm E}+1} =\frac{z^2}{4(\tau_{\mathrm E}+1)}. $$ This measures the spread of possible rates. A second ingredient measures how strongly neighboring rates can be distinguished by a trial. For a Bernoulli outcome, its **Fisher information** is $$ J(\mu)=\frac1{\mu(1-\mu)} =4\gamma_{\mathrm E}^2. $$ The same quantity appears as negative entropy curvature: $h''(\mu)=-J(\mu)$. Expand $h(B)$ around the mean. The average linear term vanishes because $\mathbb E[B-\mu]=0$. Keeping the quadratic term gives $$ I(B;S)\approx \frac12 J(\mu)\operatorname{Var}(B). $$ Put the two cone formulas side by side: $$ \begin{aligned} \text{spread}&:\quad \frac{z^2}{4(\tau_{\mathrm E}+1)},\\ \text{curvature}&:\quad 4\gamma_{\mathrm E}^2=\frac4{z^2}. \end{aligned} $$ Multiply them and the directional factors disappear: $$ \frac12J(\mu)\operatorname{Var}(B) =\frac1{2(\tau_{\mathrm E}+1)}. $$ The cancellation is exact. Using that product for mutual information is a large-concentration approximation, with the mean held inside $(0,1)$ or bounded away from its endpoints. Moving toward an extreme mean narrows the Beta spread, while increasing how sharply a trial distinguishes nearby rates. At leading order, those changes balance. The total amount sets the remaining learning scale: $$ I(B;S)\sim\frac1{2\tau_{\mathrm E}} \quad\text{nats}. $$ This invites an alternative clock for a learning system. A clock that counts trials advances once for every incoming tracelet. A clock that accumulates expected information slows as repeated observations settle the same question. New contexts or new mechanisms could make it hurry again. The displayed equations give us a concrete starting point for exploring such a clock. ## A message arrives with no instruction manual Imagine sending a long record of coin flips through a storage channel. An encoder assigns a short description to likely sequences and a longer description to unlikely ones. Its ideal logarithmic bill is $-\log_2$ of the sequence probability. If both sender and receiver know the true heads rate $p$, an ordered sequence with $k$ heads in $n$ trials has probability $$ P_p^n(s^n)=p^k(1-p)^{n-k}. $$ Here $s^n$ means the whole ordered message. The expected bill is $n h(p)/\ln2$ bits. An encoder without $p$ can learn as it goes. Start with $\operatorname{Beta}(1/2,1/2)$, predict each next trial from the earlier ones, and multiply those predictions. The resulting sequence probability is $$ Q_n(s^n)= \frac{\mathcal B(k+1/2,n-k+1/2)} {\mathcal B(1/2,1/2)}. $$ There is no binomial coefficient here: we are pricing one ordered message. When averaging over all possible messages with $k$ heads, their multiplicity $\binom nk$ enters the average. The sequential Beta code is explored in [When the numbers refuse to fit](https://epistemicphysics.com/fields/statistics/when-the-numbers-refuse-to-fit/). Our new question is its **redundancy**: the expected extra cost of learning compared with already knowing the true rate. In nats, that cost is $$ R_n(p)= \mathbb E_{S^n\sim P_p^n} \left[\ln\frac{P_p^n(S^n)}{Q_n(S^n)}\right]. $$ This is another KL divergence, now between two distributions over complete messages. It is nonnegative, although a particular unusual message can receive a shorter bill from the learning encoder. [What does an unknown rate add to the bill?](https://epistemicphysics.com/fields/information-theory/how-much-can-one-trial-teach/#learning-code-cost-lab) [Open the learning-code-cost experiment in the playground](https://epistemicphysics.com/lab/?demo=learning-code-cost#experiment). ## Pi sends an invoice Why did the learner begin with half a trial on each side? Recall the Fisher geometry of the coin: $$ \sqrt{J(p)}\,dp=\frac{dp}{\sqrt{p(1-p)}}. $$ Substitute $p=\sin^2(\theta/2)$. The right-hand side becomes $d\theta$, and the interval $0