When the numbers refuse to fit

Three teams return from a survey. Each has asked two of the same three yes-or-no questions. Their individual answer frequencies agree perfectly. Every pair disagrees 70% of the time.

The spreadsheets look tidy. Their correlation matrix even passes a standard mathematical test. Shall we combine them into one picture of the population?

Before boarding that picture, try to write down one person’s three answers. How many pairs can disagree?

This little puzzle will take us into negative probability, the cost of cancellation, and an experiment that lets us stop whenever we like. Epistemic Physics gives us a useful traveling kit: positive cones, explicit questions, traceable observations, and rapidity. We will use each tool to ask a different question of a statistical model.

Three reasonable pairs arrive at an impossible party

Call the binary answers $A,B,C$, and encode yes as $+1$ and no as $-1$. Each answer is individually balanced:

For this experiment, treat the 70% disagreement rate as a specified probability. The product $AB$ equals $+1$ when the two answers agree and $-1$ when they disagree. Its expected value is therefore

The symbol $\mathbb{E}$ means an average weighted by probabilities. Since our answers have mean zero and variance one, these product averages are also their correlations, measures of how they vary together.

Our three surveys propose

Collect those numbers in the correlation matrix

A correlation matrix must be positive semidefinite: every real vector $\mathbf{u}$ must satisfy ${\mathbf{u}}^{\mathsf{T}}\mathcal{C}\mathbf{u}\ge 0$. This ensures that every linear combination of the variables has nonnegative variance. The superscript $\mathsf{T}$ means transpose.

Our matrix passes. Its eigenvalues, the scale factors along its principal directions, are

All are positive. A Gaussian model could happily use this correlation matrix. But our variables must take the two values $+1$ and $-1$. Those discrete possibilities impose an additional structure.

Write down three answers. They are either all the same, giving zero disagreeing pairs, or one differs from the other two, giving two disagreeing pairs. Consequently,

Our proposed average is $-6/5$. Equivalently, the surveys demand an average of $3(7/10)=2.1$ disagreeing pairs, when an actual triple can have at most two.

The matrix’s geometry and the possible answer assignments are asking different compatibility questions. Passing one has left the other unresolved.

A negative entry marks the missing room

Let us deliberately allow signed weights and see what they accomplish. For a particular assignment $(a,b,c)\in \{-1,+1{\}}^3$, define

There are eight assignments. The table is short enough to inspect:

The weights sum to one. Summing over an unwanted answer gives exactly the requested pair table. Summing over two answers gives the fair individual probabilities.

A marginal is the distribution obtained by forgetting some variables. This signed joint table has ordinary, nonnegative marginals for every question our teams actually asked. Its negative weights record the obstruction to treating those local answers as parts of one fixed binary assignment.

The relation between local marginals and signed global representations is developed by Abramsky and Brandenburger. Here, the eight-entry calculation gives us a small example we can hold in our hands. Read their paper on negative probabilities and no-signalling models.

There is an exact amount of negativity we must pay. Let $w$ be the total weight of the two all-equal assignments. Then

Setting this equal to $-6/5$ forces $w=-1/20$. Every signed extension with these pair expectations must have at least $1/20$ total negative weight. Our symmetric table reaches that minimum.

The negative entry is doing a particular job: it lets a compact joint representation preserve all three pairwise stories.

How much mixing makes the stories fit?

Add a fresh ingredient: with probability $\epsilon$, replace a pair survey by two independent fair answers. With probability $1-\epsilon$, use the original pair model.

The independent pair has correlation zero, so each mixed correlation becomes

The equal-correlation boundary for a classical binary triple is $c^{\prime }=-1/3$. Solving gives

At this mixture, each pair disagrees with probability $2/3$. The two all-equal assignments have zero weight, and the six remaining assignments each have weight $1/6$. We have constructed a positive joint distribution.

This fraction is a robustness measure: it quantifies how much of this particular independent-pair mixture removes the incompatibility. Another way of changing the reports could have a different cost.

For unequal pair correlations $x=\mathbb{E}[AB]$, $y=\mathbb{E}[BC]$, and $z=\mathbb{E}[CA]$, the four pairs of opposite assignments have total weights

Each $g_i$ must be nonnegative for a positive joint distribution to exist. These four inequalities enclose a tetrahedron in correlation space. A tetrahedron is the solid with four triangular faces. The positive-semidefinite correlation region extends beyond it.

This suggests an EP compatibility instrument with several lenses. One checks positive state geometry. Another checks the possible answers to the chosen questions. A third could preserve the source graph and inspect whether the surveys refer to the same population and conditions.

In a real investigation, those distinctions suggest different next actions: collect more trials to narrow uncertain pair estimates, identify a changed context, or adopt a model in which asking a question changes the state. The geometry can help choose the next experiment.

Even compatible pairs can forget a whole pattern

Try another pair of survey folders. Now use 0 and 1 for no and yes. Each row records one person, and each folder contains four different people:

Pick any two columns. Both folders contain 00, 01, 10, and 11 exactly once. Every single-question summary and every pair table agrees. Both folders describe perfectly ordinary joint worlds. Their three-way patterns differ.

We can turn that difference into a complete posterior. Assume independent people drawn from one unknown distribution over the eight answer patterns. Put a Dirichlet prior, a distribution over those eight probabilities, on that world. Give each pattern prior shape $1/8$, preserving total prior amount one. Let $q$ be the population probability of answering yes to all three questions. Counting the complete rows gives

Their posterior means are $1/40$ and $9/40$. The same pair summaries have concealed different answers to the triple question.

This is where lineage becomes a reconstruction tool. Retaining which answers belong to the same person preserves the rows needed for the likelihood. FLogic’s tally logic compiles those joined observations into counts of compatible answer patterns, then reconstructs the query distribution. With partially observed rows, its construction can retain several Beta components, weighted by the possible completions. Try the report-joining experiment.

A useful observer therefore asks two successive questions: can these summaries fit together, and which joint details have they forgotten? The first investigates compatibility. The second tells us what the next observation should recover.

Negative weights can spend computer time

A signed table also creates an ordinary sampling problem.

Write its entries as $q_i$, with $\sum _i{q}_i=1$. Define

The quantity $\mathcal{N}$ is total negative weight. The quantity $Z$, called total variation here, counts the absolute amount before positive and negative parts cancel.

Suppose we want the signed average $\sum _i{q}_{i}f(i)$ of a quantity $f$. We can sample using entirely ordinary probabilities:

The random index $I$ selects an assignment. The function $\mathrm{sign}$ restores whether its weight was positive or negative. Multiplying out the average gives

The estimator is unbiased: its average over repeated runs equals the target. If $|f(i)|\le 1$, then

For our triangle, $Z=11/10$. Choose $f=AB$, so the target average is $-2/5$. The signed estimator has variance

Direct sampling from the positive $AB$ marginal has variance $1-4/25=21/25$. For these two estimators of this particular quantity, signed sampling needs $25\mathrm{\%}$ more draws to give its sample mean the same variance.

Imagine two buckets of tokens, one positive and one negative. Larger buckets can produce the same net total, yet require more sampling to resolve the cancellation. Negativity has acquired a computational price.

An ambitious next step would search among signed representations that preserve the required observable predictions and choose one with cheaper cancellation. The best representation might depend on which questions we need to answer.

A model can fit today’s table and stumble tomorrow

Compatibility examines whether claims can coexist. Prediction asks what a model expects next.

Set the survey triangle aside and pick up a coin. Our reference model says each flip is independent and has heads probability

A competing model allows an unknown fixed heads probability $p$, initially distributed as

After $S_n$ heads and $F_n$ tails in $n=S_n+F_n$ flips, its next-heads probability is

The half-trials belong to the initial prior and are included once. Each observed flip contributes one new trial. The next prediction uses only answers already seen.

How well did these two models predict the whole sequence? The reference assigns likelihood $p_0^{S_n}(1-p_0{)}^{F_n}$. The learning model averages over its initial Beta distribution, assigning likelihood

The Beta function $B(a,b)={\int }_0^1{p}^{a-1}(1-p{)}^{b-1}dp$ supplies the normalization. We are evaluating one ordered sequence, so no binomial coefficient is needed.

Their likelihood ratio is

A value above one favors the learning model for the observed sequence; below one favors the reference. This is evidence between the two specified predictive models.

Build the stopping rule into the path

Suppose the reference model is correct. Given the past, the next likelihood multiplier is

Its conditional average is

Therefore $\mathbb{E}[E_{n+1}\mid \text{past}]=E_n$. Such a process is a martingale: under the reference model, the current value is the expected next value given the information already available.

Our process is nonnegative and starts at $E_0=1$. Ville’s inequality gives

The symbol $sup$ means the largest value reached over all times. With $\delta =1/20$, the probability of ever crossing $E=20$ is at most $5\mathrm{\%}$ under the reference model. We may inspect the path after every flip and choose when to stop. Shafer and coauthors explain the links among these test martingales, likelihood evidence, and sequential probability bounds. Read their paper.

The entire trajectory has become the statistical object. A path can cross and later return; its present value and its maximum answer different questions.

The same path speaks in rapidity and bits

Assign a rapidity to the likelihood odds between our two models:

This is the rapidity of the model comparison. The unknown coin probability has its own coordinate $\frac{1}{2}\log [p/(1-p)]$; the two objects play different roles.

The stopping boundary becomes

Each new likelihood factor adds a rapidity increment. Epistemic Physics makes the accumulation easy to picture: evidence follows a path in half-log odds.

There is also a coding interpretation. An ideal predictive code assigns a sequence with probability $q$ a length $-{\log }_{2}q$ bits. A bit is one binary unit of information. The reference code and learning code therefore differ by

Evidence, rapidity, and saved bits are three readings of the same calculation:

Seven heads in a row give $E_7=3432/128$, already above twenty. Balanced sequences often favor the fixed fair-coin model because it placed its predictions directly at one half while the learning model spread its initial attention over many possibilities.

A percentage about a posterior has a different job

A $95\mathrm{\%}$ equal-tailed posterior interval leaves $2.5\mathrm{\%}$ posterior probability below it and $2.5\mathrm{\%}$ above it. That is a statement about the distribution of $p$ after the observed sequence.

Try a different rule: inspect this interval after every flip, and stop when it excludes one half. What is the probability of stopping by flip 500 if the coin is actually fair?

We can calculate it by carrying the probability of every still-running path forward through heads and tails, removing paths as soon as they cross. The exact finite recursion, evaluated numerically, gives:

The rules answer different questions. One controls a crossing event across time. The other repeatedly inspects posterior mass at a sequence of observation times. Naming the percentage’s job helps us choose the tool for the experiment we intend to run.

Race the stopping rules across every possible experiment

How much safety does the crossing rule buy, and what does it cost? Give one hundred simulated labs the same flips and three verdict rules: an unadjusted check at every look, a pre-registered schedule that splits its $5\mathrm{\%}$ error budget across one hundred looks, and the evidence path crossing $E\ge 20$. The forward recursion above prices each rule exactly, over every possible sequence at once.

With no real bias, unadjusted peeking eventually declares a difference in over a third of the labs, and its rate keeps climbing as the horizon grows. The pre-registered schedule stays safe only until its planned looks run out. The evidence path holds one lifetime guarantee at every horizon, and it finds a solid bias at nearly the same trial count as the schedule. The freedom to keep watching is what the martingale bought.

Test a contextual world by the questions actually asked

Return to our three survey teams. Suppose the experimental context determines which pair is asked, and a model predicts a valid distribution for the answers in that context. Its predictions may come from a changing matrix state, a hidden source graph, or a signed representation of several local tables.

We can still test the observable predictions. At step $i$, let $u_i$ name the chosen question, $y_i$ its answer, and $h_i$ the recorded history before that answer. Compare a proposed conditional prediction $q_i(y_i\mid h_i,u_i)$ with a reference $p_i(y_i\mid h_i,u_i)$:

Let the question depend on the past. Require both models to give normalized, nonnegative answer probabilities, with the reference positive wherever the proposed model is positive. Hold the current history and question fixed, and shorten the notation to $q_i(y)$ and $p_i(y)$. Under the reference, the next factor again has average one:

The same martingale argument applies. We have moved from independent coin flips to adaptive questioning through a specified history.

This suggests a larger EP research instrument. Preserve the identities of actual observations. Track the questions that produced them. Let different models propose their next answer distributions, then compare the resulting evidence paths. A copied report can be predicted from its parent; it does not need to masquerade as another independent trial.

The directional-learning article gives the observer a way to choose its next direction. Evidence taking different roads supplies source graphs and state-changing questions. Here we have added two further instruments: a compatibility test for a proposed world, and a predictive path for the world encountered through time.

One ambitious experiment would let those instruments talk to each other. A compatibility failure could suggest a new context variable. An adaptive question could distinguish the resulting models. Their next predictions would determine which new description earns its place.