The world inside a normalizer

A probability density usually arrives wearing a small, forgettable denominator. The numerator gives the curve its interesting shape. The denominator makes its area equal one. It is tempting to divide, tidy the expression, and move on.

Keep that denominator on the desk. It can predict the next observation, count possible histories, compare models, and generate a geometry of statistical distinguishability. After a change of coordinates, it becomes a partition function, the object a statistical physicist uses to organize a whole thermal system.

There is another surprise waiting at the end. Even a complete probability distribution over velocities leaves a journey unfinished. To find where a traveler ends up, we also need to know how long each velocity lasts.

Three objects that travel together

Assume an unknown binary rate $P$ has a proper Beta distribution, with positive shapes $\alpha ,\beta$. Recenter it to $U=2P-1$, so possible values $u$ lie between $-1$ and $1$. Throughout this article, write the prior-inclusive evidence coordinates without their usual epistemic subscripts:

$$\tau =\alpha +\beta ,x=\alpha -\beta ,v=\mathbb{E}[U]=\frac{x}{\tau },m=2\sqrt{\alpha \beta }.$$

The random variable $U$ describes the whole cloud of possible velocities. Its mean $v$ describes one feature of that cloud. The exact identity ${\tau }^2-x^2=m^2$ puts these parameters inside a Lorentz cone.

Our traveling kit contains three objects:

$$q(u)=(1+u{)}^{\alpha -1}(1-u{)}^{\beta -1},Z={\int }_{-1}^{1}q(u)du,f_U(u)=\frac{q(u)}{Z}.$$

Here $q$ is an unnormalized kernel, $Z$ its total weight, and $f_U$ the probability density. Think of $q$ as a recipe’s relative ingredients, $Z$ as the recipe’s total, and $f_U$ as the proportions.

Substitute $p=(1+u)/2$ into the integral. Because $du=2dp$, the powers of two collect into

$$Z=2^{\tau -1}\mathrm{B}(\alpha ,\beta )=2^{\tau -1}\frac{\mathrm{\Gamma }(\alpha )\mathrm{\Gamma }(\beta )}{\mathrm{\Gamma }(\tau )}.$$

The Beta function $\mathrm{B}$ is an integral; $\mathrm{\Gamma }$ extends the factorial, with $\mathrm{\Gamma }(k+1)=k!$ for nonnegative integers. Euler’s Beta integral supplies the identity used here.

The change of variables also explains a common puzzle. Stretching the probability interval from $[0,1]$ to $[-1,1]$ halves the density’s height:

$$f_U(u)=\frac{1}{2}{f}_P\left(\frac{1+u}{2}\right),{\int }_{-1}^1{f}_U(u)du=1.$$

This factor is the Jacobian, the correction for a coordinate’s changing length scale. Omitting it doubles the area. A coordinate change preserves total probability when the density and measure travel together.

The first row is the Lorentz-factor shape. Its normalized version is the Jeffreys prior $j(u)=1/[\pi \sqrt{1-u^2}]$. Thus $\gamma (u)=\pi j(u)$: the same shape, with explicitly different areas.

Multiplying $q$ by any positive constant multiplies $Z$ by that constant and leaves $f_U$ unchanged. A raw area acquires meaning only after we specify the kernel and integration measure. We have now done that.

A lattice decorated with π

Begin with Jeffreys half-counts and collect $Y$ support trials and $N$ opposition trials:

$$\alpha =Y+\frac{1}{2},\beta =N+\frac{1}{2},n=Y+N,\tau =n+1.$$

For integer counts, the half-integer Gamma formula simplifies the area to

$$Z_{Y,N}=\pi \frac{(2Y)!(2N)!}{2^{Y+N}Y!N!(Y+N)!}.$$

Every lattice site gets a rational multiple of $\pi$. The prior has $Z=\pi$; one support trial still has $Z=\pi$; two support trials give $3\pi /2$; one of each gives $\pi /2$.

Why do balanced and one-sided trials send the area in opposite directions? Multiplying by $(1+u)$ rewards the positive end. Repeating that reward amplifies a region already favored by earlier trials. A balanced pair multiplies by $1-u^2$, which never exceeds one.

Two particularly tidy paths are

$$Z_{k,k}=\pi \frac{(\genfrac{}{}{0ex}{}{2k}{k})}{4^k},Z_{n,0}=\pi \frac{(\genfrac{}{}{0ex}{}{2n}{n})}{2^n}.$$

For large counts these behave as $\sqrt{\pi /k}$ and $2^n\sqrt{\pi /n}$, respectively. Remember that the first path has $2k$ trials and the second has $n$. At any common total, exchanging support and opposition leaves the area unchanged; greater imbalance makes it larger.

The next prediction hides in a neighboring area

A support trial changes $(\alpha ,\beta )$ to $(\alpha +1,\beta )$. Gamma recurrence immediately gives

$$\frac{Z(\alpha +1,\beta )}{Z(\alpha ,\beta )}=\frac{2\alpha }{\tau }=1+v.$$

But a support observation’s likelihood is $(1+u)/2$. Its predictive probability therefore includes one more factor of $1/2$:

$$P(+\mid \alpha ,\beta )=\frac{Z(\alpha +1,\beta )}{2Z(\alpha ,\beta )}=\frac{1+v}{2}.$$

The opposition formula replaces $v$ by $-v$. Neighboring normalizers know our next prediction.

A support-opposition pair gives a richer identity:

$$\frac{Z(\alpha +1,\beta +1)}{Z(\alpha ,\beta )}=\frac{4\alpha \beta }{\tau (\tau +1)}=\frac{m^2}{\tau (\tau +1)}=\tau \mathrm{Var}(U).$$

The same ratio is $\mathbb{E}[1-U^2]$. It joins the mass-like quantity, the cloud’s variance, and a two-observation update. Divide it by four to get the probability of one specified ordered pair, such as support followed by opposition.

A sequence, a count, and a bet

Suppose a particular sequence $D$ contains $Y$ support trials and $N$ opposition trials. Averaging its likelihood over the Jeffreys prior gives

$$P(D)=\frac{Z_{Y,N}}{\pi 2^n}.$$

An unordered count includes $(\genfrac{}{}{0ex}{}{n}{Y})$ different sequences, so its probability is $(\genfrac{}{}{0ex}{}{n}{Y})P(D)$. For two observations, the particular sequence $+,-$ has probability $1/8$; the count “one of each” has probability $1/4$ because $-,+$ also qualifies.

A fixed fair coin assigns the specified sequence probability $2^{-n}$. Thus

$$\frac{P(D\mid \text{unknown rate, Jeffreys prior})}{P(D\mid \text{fixed fair coin})}=\frac{Z_{Y,N}}{\pi }.$$

This probability ratio is a Bayes factor. Here the raw area has a concrete comparison attached to it. A value above one favors the unknown-rate model relative to the fair-coin model for these data. Prior odds between models are a separate input.

There is also a coding interpretation. The ideal information length of this sequence is

$$\mathcal{L}(D)=-{\log }_{2}P(D)=n+{\log }_2\pi -{\log }_2{Z}_{Y,N}.$$

Each observation adds its predictive surprise, $-{\log }_{2}P(\text{next outcome})$, in bits. The half-count predictor is the binary rule studied in Krichevsky and Trofimov’s universal encoding paper; a finite implemented code also has rounding and overhead.

For a playful third reading, imagine betting a fraction $v$ of current wealth on support at even odds, allowing a negative fraction to back opposition. Outcome $s=\pm 1$ multiplies wealth by $1+sv$. Iterating the neighboring-area rule gives

$$\frac{Z_n}{\pi }=\prod _{i=1}^n(1+s_i{v}_{i-1}).$$

Under a fair coin, each factor has conditional mean one. The resulting wealth process is a martingale: its expected next value, given its history, equals its present value. This interpretation follows from the specified fair-coin experiment. Shared testimony still needs its lineage tracked; repeating one trial’s report does not create a fresh independent trial.

Before opening the lab, predict what happens after two support trials and after one of each. Then select bars with the same total count. Watch which quantities are probabilities and which are ratios.

Normalizer laboratory

One sequence, a whole packet of orderings

Reset lab

Tap a tile to change its outcome. Your strip is one particular sequence. A count packet includes every ordering with the same number of support trials.

1+2−3+4−

Move first tile to the end Try six trials

Predict: which gets more probability, this exact sequence or its whole count packet? 2 support, 2 opposition.

The exact sequence The whole packet They tie

Choose another count packet

Keep 4 trials. Tap a bar to change how many support outcomes the packet contains.

01234

Support count. Swipe horizontally to see every packet.

Open the normalizer ledger

Raw area Z / π, a Bayes factor

0.375

Next support probability

50.0%

Sequence code length

5.42 bits

The prior contributes ½ to each Beta weight once: α = 2.5, β = 2.5. Z/π compares the unknown-rate model against a fixed fair coin; it can exceed one because it is a probability ratio.

Give the same belief three different clocks

Now hold that velocity distribution fixed

All three journeys have mean velocity 0.000 and velocity variance 0.167. Rose keeps one draw forever; teal redraws at random refresh times; gold redraws every 0.25 time units.

λ = 0.25 / time unitλ = 1 / time unitλ = 4 / time unit

− ¼ time + ¼ time Jump to 10

At time 5.00, mean displacement = 0.000 for every model.

Fixed draw: position variance

4.167

Poisson refresh: position variance

1.336

Independent bins: position variance

0.208

Analytic curves, with X(0) = 0 and stationary initial velocities; no sampled trajectories. Time and position use dimensionless model units. The refresh clock is an extra assumption: the Beta shapes alone do not choose it.

Rapidity opens a thermal door

Unroll the bounded velocity interval using rapidity $\varphi =\mathrm{artanh}u$, or $u=\tanh \varphi$. Two elementary identities do the work:

$$1\pm u=\frac{e^{\pm \varphi }}{\cosh \varphi },du={\mathrm{sech}}^2\varphi d\varphi .$$

Here $\mathrm{sech}\varphi =1/\cosh \varphi$. Carrying the Jacobian transforms our entire unnormalized measure:

$$q(u)du=e^{x\varphi -\tau \log \cosh \varphi }d\varphi ,Z={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{x\varphi -\tau \log \cosh \varphi }d\varphi .$$

This is also a form of the hyperbolic Beta integral. The right tail decays like $e^{-(\tau -x)\varphi }$ and the left like $e^{(\tau +x)\varphi }$. Both integrate to a finite value precisely when . The evidence cone is also the domain where this partition function exists. A cone-shaped domain alone does not make $Z$ invariant under Lorentz boosts.

Hold the mean $v=x/\tau$ fixed and define an energy landscape

$$H_v(\varphi )=\log \cosh \varphi -v\varphi .$$

The rapidity density becomes the exact Gibbs form

$$f_{\mathrm{\Phi }}(\varphi )=\frac{e^{-\tau H_v(\varphi )}}{Z(\tau ,\tau v)}.$$

In this dimensionless thermal reading, $\tau$ is inverse temperature. The landscape’s minimum solves $\tanh \varphi =v$. Its minimum is therefore the mean-state rapidity $\mathrm{artanh}v$, also the mode of this rapidity density. Generally it differs from $\mathbb{E}[\mathrm{\Phi }]$.

The landscape measures a familiar prediction mismatch. Set $p_v=(1+v)/2$ and $p_{\varphi }=(1+\tanh \varphi )/2$. Subtract the energy at its minimum and simplify:

$$H_v(\varphi )-H_v(\mathrm{artanh}v)=D_{\mathrm{K}\mathrm{L}}(\mathrm{Bern}(p_v)\Vert \mathrm{Bern}(p_{\varphi })).$$

The KL divergence on the right is the expected log probability ratio between two Bernoulli models. A thermal well and excess predictive loss now appear in the same equation.

At fixed interior $v$ and large $\tau$, the well looks quadratic and $\mathrm{Var}\mathrm{\Phi }\approx 1/[\tau (1-v^2)]$. Increasing $\tau$ along that ray sharpens the distribution. An actual observation usually changes $v$ too. For example, one opposition trial changes $\mathrm{Beta}(9.5,0.5)$ to $\mathrm{Beta}(9.5,1.5)$ and raises velocity variance from $19/1100$ to $19/484$. Surprising evidence can broaden a belief while increasing its trial amount.

Even “raise the density to a power” needs a coordinate:

The last row naturally expresses fractional trial weight. A physical temperature calibration would add units and a specified physical model to the Gibbs correspondence.

Differentiate the denominator

Let $A=\log Z$. Differentiating its integral reveals

$${\partial }_{x}A=\mathbb{E}[\mathrm{\Phi }],{\partial }_x^{2}A=\mathrm{Var}(\mathrm{\Phi }),{\partial }_{\tau }A=-\mathbb{E}[\log \cosh \mathrm{\Phi }].$$

An applied bias changes a mean; the rate of that response equals a fluctuation. This is the partition function’s great organizational trick.

More fully, the matrix of second derivatives in coordinates $(x,\tau )$ is the covariance matrix of $(\mathrm{\Phi },-\log \cosh \mathrm{\Phi })$. It is the family’s Fisher information matrix, measuring how distinguishable neighboring distributions are. These are the standard exponential-family derivative identities, specialized to this integral. The positive Fisher metric and the indefinite form $d{\tau }^2-d{x}^2$ perform different jobs: statistical distinguishability and evidence-cone geometry.

The thermal derivatives also need the correct route through parameter space. At fixed $v$, use a total derivative along $x=\tau v$:

$${\mathcal{F}}_v=-{\tau }^{-1}\log Z(\tau ,\tau v),{\mathcal{U}}_v=-\frac{d}{d\tau }\log Z(\tau ,\tau v)=\mathbb{E}[H_v],C_v={\tau }^2\mathrm{Var}(H_v).$$

These are the chosen model’s free energy, mean energy, and heat capacity. The fixed-$x$ partial derivative answers a different question. Likewise, $-\log P(D)$ is data surprise; connecting it to this free energy requires the prior and reference-likelihood factors already displayed above.

Normalization retains the shapes, trial amount, and malleability encoded in the distribution. If a separate material density $\rho$ matters, write $\rho f_U$; its integral is $\rho$. There is no need to assign that role to an arbitrary kernel area.

Spheres and zeta values appear along the way

For a balanced state $\alpha =\beta =a$, the kernel is $(1-u^2{)}^{a-1}$. It has a geometric realization: the first coordinate of a uniformly chosen point on a sphere in $d=2a+1$ dimensions has this density, whenever $d$ is an integer at least two. To see the exponent, write that coordinate as $u=\cos \theta$. The sphere’s angular area factor is ${\sin }^{d-2}\theta d\theta$; changing to $du$ removes one sine and leaves $(1-u^2{)}^{(d-3)/2}$.

A circle gives the Jeffreys arcsine law. An ordinary sphere in three dimensions gives a uniform coordinate. Balanced Jeffreys trials $Y=N=k$ correspond to $d=2k+2$: each balanced pair advances this distributional representation by two dimensions. It is a lovely way to visualize concentration without asserting that physical space gains dimensions.

The same normalizer also generates rapidity moments:

Derivatives at zero of the logarithm are cumulants: the first is the mean, the second the variance, and higher ones measure departures from Gaussian shape. For balanced shapes, odd cumulants vanish. The even ones become

$${\kappa }_{2r}=2^{1-2r}(2r-1)!\zeta (2r,a),$$

where $\zeta (s,a)=\sum _{k=0}^{\mathrm{\infty }}(k+a{)}^{-s}$ is the Hurwitz zeta function. This follows by differentiating log Gamma and using its polygamma series. At Jeffreys, the rapidity density is $\mathrm{sech}\varphi /\pi$ and its variance is ${\pi }^2/4$. These identities connect fluctuations to zeta values at positive even integers; questions about nontrivial zeta zeros require further structure.

Which integral gives position?

Several integrals have now appeared. Put their meanings side by side:

The third integrates across possibilities; the fourth integrates along a history. A cloud is a photograph. A process is a film.

Even along an evidence history, the ratio $v=x/\tau$ and incoming tangent bias $j=dx/d\tau$ differ:

$$\frac{dx}{d\tau }=v+\tau \frac{dv}{d\tau },\tau \frac{dv}{d\tau }=j-v.$$

Integrating $j$ reconstructs the evidence path. Integrating $v$ generally constructs a different trajectory. The tracelets in a lineage carry ordered increments; the final Beta state alone does not specify their timing.

Choose a velocity process and set $X(0)=0$. Then

$$\mathbb{E}[X(t)]={\int }_0^t\mathbb{E}[U(s)]ds,\mathrm{Var}X(t)={\int }_0^t{\int }_0^t\mathrm{Cov}(U(s),U(r))dsdr.$$

The covariance tells us whether a velocity now resembles the velocity later. That is the missing clock.

The lab compares three explicit choices, all with the same stationary Beta velocity distribution, mean $v$, and variance ${\sigma }_U^2$:

A Poisson clock has independent exponentially distributed waiting times, averaging $1/\lambda$. The chance that no redraw separates two times is $e^{-\lambda |s-r|}$, so its velocity covariance is ${\sigma }_U^2{e}^{-\lambda |s-r|}$. Inserting this into the double integral gives the middle row.

At short times, persistence produces quadratic, ballistic variance growth. Over many refreshes it crosses toward linear, diffusive growth. The fixed-bin model supplies a useful limiting surprise: shrinking $\mathrm{\Delta }t$ to zero while retaining bounded velocities makes position variance vanish. A nontrivial white-noise limit requires a different scaling.

Edit the movie while keeping every snapshot

Cast four travelers in a short film. Every frame contains the same four velocities, but you decide which traveler receives each one. Rotate the cast to make the travelers reunite or spread apart. This finite, discrete example makes the missing ingredient tangible: a sequence of identical velocity histograms can support very different correlations across time.

The velocity casting room

Edit four journeys from the same snapshots

Reset the film

Every frame contains the same four velocities. Tap a frame to rotate which traveler gets which velocity. Can you bring everyone together without changing a single frame's histogram?

Make a reunionMake the widest fan

Invite all four travelers back to position zero after four frames. Click a frame to cycle its cast; edits rewind the film.

Frame 1 ↻A -0.75B -0.25C +0.25D +0.75Frame 2 ↻A -0.75B -0.25C +0.25D +0.75Frame 3 ↻A -0.75B -0.25C +0.25D +0.75Frame 4 ↻A -0.75B -0.25C +0.25D +0.75

Play one frameShow the whole filmRewind

Traveler ATraveler BTraveler CTraveler D

The film is paused. Arrange the remaining casts, or play the next frame.

Each frame's velocity variance

0.3125

Final position variance

Play to reveal

Same marginals, different temporal coupling

This exact finite ensemble uses four equally weighted velocities −0.75, −0.25, 0.25, and 0.75 in each unit-time frame. It illustrates the clock principle with a discrete distribution, rather than sampling a Beta law. All travelers begin at zero; positions add their assigned velocities. Rotations preserve every one-frame histogram and its zero mean.

Changing assignments changes correlations between a traveler's velocities at different times. Persistent assignments produce variance 16 × 0.3125 = 5 after four frames. Giving every traveler each velocity once produces zero final position variance. The same logic is why a stationary Beta cloud alone cannot specify its displacement process.

Conservation, biology, and a partition function for histories

There is a local conservation law hiding in normalization. Let $P(z,u,t)$ be the joint position-velocity density for the refresh process and $\rho (z,t)=\int Pdu$ its position density. Transport and refreshing obey

$${\partial }_{t}P+u{\partial }_{z}P=\lambda [f_U(u)\rho -P].$$

Integrate over velocity. Since $\int f_{U}du=1$, the refresh terms cancel:

$${\partial }_t\rho +{\partial }_{z}J=0,J=\int uPdu.$$

The flux $J$ moves probability around without creating it. Replacing $f_U$ by raw $q$ would create an artificial source $\lambda (Z-1)\rho$; retaining $q/Z$ restores conservation. Every trajectory also stays inside $|X(t)|\le t$, because $|U|\le 1$. Relating this trajectory cone to the evidence-parameter cone requires an explicit mapping between their coordinates.

Biology offers another clock. Neutral Wright-Fisher diffusion with recurrent mutation has a Beta stationary distribution, as developed in the population-genetics treatment by Steinrücken, Wang, and Song. One recentered convention is

$$dU=D(x-\tau U)dt+\sqrt{2D(1-U^2)}d{W}_t.$$

Here sets the time scale and $W_t$ is standard Brownian motion. With the usual mutation-diffusion boundary behavior, its stationary density is our $f_U$. Its stationary covariance decays as ${\sigma }_U^2{e}^{-D\tau |t|}$. Choosing $\lambda =D\tau$ makes its integrated position variance match the refresh model exactly, despite one velocity process being continuous and the other jumping.

Matching a distribution and covariance still leaves different movies. Jump timing, continuity, and higher joint moments can distinguish them. In the biological model the shapes describe mutation relative to genetic drift; reading them as observation counts adds a separate interpretation.

Selection supplies another useful correspondence. Reweight a population by fitness $w(u)\ge 0$:

$$f_{\mathrm{n}\mathrm{e}\mathrm{w}}(u)=\frac{w(u)f(u)}{{\mathbb{E}}_f[w]}.$$

The divisor is mean fitness. Replace fitness by a likelihood and the same operation is Bayesian updating, with predictive evidence as divisor. For exponential fitness $e^{su}$, differentiating its log normalizer gives the selected mean and variance. The resulting density generally extends beyond the ordinary Beta family.

To encode an entire journey, choose its Markov generator $L$, an operator describing instantaneous changes of functions of velocity, and form

$$G(u,t;k)={\mathbb{E}}_u\left[e^{k{\int }_0^{t}U(s)ds}\right].$$

Conditioning on a short first step gives ${\partial }_{t}G=(L+ku)G$, with $G(u,0;k)=1$. For refreshing, $Lg=\lambda ({\mathbb{E}}_f[g]-g)$. For the diffusion, $Lg=D(x-\tau u)g^{\prime }+D(1-u^2)g^{″}$. Derivatives of $\log G$ in $k$ now organize displacement cumulants. This is a calculable route from a static normalizer to a partition function over histories.

A further adventure would couple many such beliefs and ask whether a collective transition emerges. One finite Beta partition function is smooth inside its domain; two finite competing model evidences also combine smoothly by addition. A sharp transition needs a specified interaction and limiting process. That choice could create new physics, just as the refresh clock created a new journey.

Reading further

• Bayesian Special Relativity introduces the evidence-cone coordinates and rapidity construction.
• DLMF Beta integrals and polygamma functions give the integral and derivative identities behind the normalizer.
• Exponential families develops the connection between partition derivatives, fluctuations, and Fisher information.
• The performance of universal encoding provides the historical coding connection.
• Wright-Fisher diffusion with recurrent mutation and selection develops the biological stationary-distribution setting.