A learning machine that grows its own concepts

A gate has two switches, one rose and one blue. Sometimes it opens. Sometimes it stays shut.

You hire two diligent observers. One watches the rose switch; the other watches the blue switch. After a balanced tour of all four settings, they return with identical reports: “Up or down, the gate opens half the time.”

They collect more trials. Their reports become better supported. Their predictions stay at one half.

Then someone looks at the switches together.

The gate opens whenever the switches match.

That small discovery changes the learner’s vocabulary. It can now ask same or different? A relationship has become a thing the learner can notice, remember, and use in another relationship. Let’s build a machine around that moment.

A clue hiding between the clues

Call the switch positions $a$ and $b$. Each is $+1$ when up and $-1$ when down. Let $Y=1$ mean that the gate opens and $Y=0$ mean that it stays shut.

Here is the whole first world:

We visit each row equally often. Among trials with the rose switch up, half open and half stay shut. The same is true with the rose switch down, and for either position of the blue switch. Writing $P$ for probability, all four single-switch predictions are

The vertical bar means “given.” Each observer has correctly summarized the question they were allowed to ask.

Now multiply the two signs:

$$c=ab.$$

The new variable $c$ is $+1$ when the switches match and $-1$ when they differ. Through this question, the same observations look very different:

$$P(Y=1\mid c=+1)=1,P(Y=1\mid c=-1)=0.$$

This is an interaction: the effect of one clue depends on the other. With the blue switch up, raising the rose switch opens the gate. With the blue switch down, raising the rose switch closes it.

Before trying the experiment, make a prediction. If we keep asking only the separate questions, will another hundred balanced trials reveal the rule?

Give the learner another question

The learner below starts with three possible ways to organize its observations: put every outcome in one drawer, sort outcomes by switch A, or sort them by switch B. Each drawer keeps a belief about how often the gate opens there.

Try Observe four more a few times. In the pair world, both single-switch views remain balanced. Then press Try “same or different.” The new question sorts the same trials into a different pair of drawers.

A new question can become a new part

Teach a learner to notice a relationship.

Start again

Two switches control a gate. Four trials are already on the table. Choose a switch pair, watch the learner's prediction, then let it try a shared question.

The pair matters A alone matters

1. Look at the trials

Select a row to ask about that setting.

ABObserved outcome

↓ ↓ 1 × opened↓ ↑ 1 × stayed shut↑ ↓ 1 × stayed shut↑ ↑ 1 × opened

Observe four more Undo four

4 trials, balanced across the four settings.

2. Grow a shared question

A ↑Tap to flip B ↑Tap to flip

+ Try “same or different”Connect the two clues

At first, the learner can ask about A, B, or neither. Each question has its own outcome drawers.

3. Predict this setting

Chance the gate opens with A up and B up.

Separate questions50.0%

Current learner50.0%

Try the gate

Choose a setting above, make your prediction, then test it.

See how much each question contributes

Models start with equal prior weights. Each drawer starts with Beta(1, 1); its forecast is the posterior mean. The final forecast averages the models.

One shared drawer 37.5% weight 50.0% open

Ask A alone 31.3% weight 50.0% open

Ask B alone 31.3% weight 50.0% open

A teaching model with two deterministic worlds and four predefined candidate questions. “Grow” adds the relation candidate to the comparison; it does not search for arbitrary concepts. Model weights come from exact Beta integrals of the observed outcomes, with each trial used once per alternative model.

Open the concept-growth experiment in the playground.

Flip either switch and test the gate. The relation changes when one switch changes. Flip both, and the relation returns to its old value.

Then switch to A alone matters. A more elaborate question is available, but the observations favor the simple one. A useful growing machine needs room for both outcomes: sometimes a relationship earns a role, and sometimes the existing question already does the job.

This is a deliberately small teaching model. We have supplied the candidate relation in advance. The learner determines how much to rely on it; searching a large world for good candidate relations is a further problem.

What a new part remembers

Each drawer begins with a uniform belief over its opening probability $p$. We write that belief as

$$p\sim \mathrm{Beta}(1,1).$$

The symbol $\sim$ means “has this distribution.” The two Beta parameters record support for opening and staying shut. If a drawer receives $s$ openings and $f$ closures, its updated belief is

$$p\mid \text{trials}\sim \mathrm{Beta}(s+1,f+1).$$

The extra ones belong to the starting belief. The forecast for the next opening is its posterior mean:

$$\hat{p}=\frac{s+1}{s+f+2}.$$

An empty drawer predicts one half. A drawer containing eight openings and no closures predicts $9/10$. The learned forecast can leave room for an exception even though we, as designers of this particular world, know its rule is deterministic.

The drawer also remembers how many trials support the forecast. A belief built from two trials is easier to change than one built from two hundred. That difference in malleability matters when a machine takes its new concept somewhere unfamiliar. See A belief is more than a number for the geometry behind these two coordinates.

In the pair world, the new part has two addresses:

The part has learned something reusable: the answer depends on a relationship that survives a simultaneous reversal of both switches.

It also made a choice about what to forget. “Both up” and “both down” now share a drawer. If a later task cares which of those happened, the original switch values still matter. A concept is a useful way of grouping experiences for a question.

How a candidate earns its place

The demo compares four small models. A model here is a rule for assigning observations to drawers, together with a Beta belief in each drawer:

Before seeing outcomes, the models in each comparison receive equal prior weights. The three-model view excludes the relation; the grown view admits it as a fourth alternative.

How should the observations change those weights? A candidate earns support by assigning probability to the outcomes that actually arrived. For one drawer, average over every possible opening probability under its starting Beta belief. The result is

$$E(s,f)={\int }_0^1{p}^s(1-p{)}^{f}dp=\frac{s!f!}{(s+f+1)!}.$$

Here $E$ is the probability of that particular sequence of outcomes under the drawer’s model, and $n!$ means the product of the integers from $1$ through $n$, with $0!=1$. We condition on the observed switch settings. There is no extra factor for rearranging outcomes into other sequences.

For model $j$, multiply these quantities over its drawers to obtain $E_j$. Its updated weight is

$$w_j=\frac{E_j}{\sum _k{E}_k}.$$

The index $k$ runs over the competing models. Their weights sum to one. Averaging their next-trial forecasts gives

$$P(\text{next gate opens}\mid \text{trials},a,b)=\sum _j{w}_j{\hat{p}}_j(a,b).$$

The integral rewards a model whose possible opening rates made the observed outcomes plausible. It also accounts for learning those rates: every additional drawer begins with its own uncertainty. We do not simply pick the most flattering fitted percentage.

After sixteen balanced trials in the pair world, the relation receives about 99.92% of the model weight in this four-model comparison. Yet its opening forecast for matching switches is still about 90%. Those percentages answer different questions. The first compares ways of organizing the observations; the second predicts the gate’s next outcome under that comparison.

This exact calculation is also what you get by making each model predict every trial before adding that trial to its drawers, then multiplying the probabilities it assigned. Because the candidate menu was fixed in advance, the observations can judge the models without a hidden search through thousands of alternatives. A larger learner must account for that search too.

The same product appears in physics

The relation $ab$ has another home. Represent two magnetic spins by signs $a,b\in \{-1,+1\}$. A simple interaction energy is

$$E_{\mathrm{spin}}=-Jab,$$

where $J$ is a positive coupling energy. Matching spins have lower interaction energy than opposing spins.

Place the two uses next to one another:

In both descriptions, a product carries a property of the pair. Neither individual sign determines it. Reversing both signs preserves it.

The gate’s rule is a toy predictive relationship; the spin formula is a physical interaction model. Their shared algebra suggests a useful design question: could a learner build new parts that respond to relationships, just as an interaction term responds to a configuration?

That is one route explored by the EP research prototype eplant. Its broader design allows relationships between existing parts to become parts in their own right. The small learner on this page isolates that idea with exact Beta calculations. The prototype uses a broader architecture and a different scoring method.

A concept can become somebody else’s clue

Imagine four switches instead of two. One new part asks whether the first pair matches. Another asks whether the second pair matches. A third can ask whether both pairs match.

The third part sees two meaningful inputs. Those inputs were themselves constructed from earlier inputs. This is depth of construction: a description built from other descriptions.

The same pattern is tempting in many settings. A small patch of an image becomes an edge; relations between edges suggest a corner. A sequence of sounds becomes a recurring fragment; relations between fragments suggest a word. Two individually weak measurements can jointly reveal a failure mode.

Each proposed part needs a predictive reason to exist. Each also needs a record of which observations support it. If two parts learned from the same trial, their agreement does not turn that trial into two independent witnesses. The alternatives in our demo are averaged as models; their trial counts are never added together. Larger networks need equally careful lineage bookkeeping.

Where growing relationships has paid off

Two experimental EP architectures, MorphoPlant MP5 and epibrain v4, already show substantial gains on tasks built around relationships. Here are selected wins recorded on October 29, 2026, against the histogram gradient-boosting baseline used in each experiment, a learner that combines many decision trees.

The score is log loss: the penalty for assigning too little probability to the answer that occurs. Lower is better. These values use natural logarithms and are averaged per test example, in units called nats.

On MONK-1 and MONK-2, MorphoPlant also reached 100% test accuracy, compared with the tree baseline’s 87.7% and 61.6%. Those runs used 124 and 169 training examples, respectively, and tested on the 308 and 263 combinations absent from training.

The parity task extends our gate: among sixteen binary inputs, five determine whether the count of ones is odd or even. Ten percent of the labels are flipped. epibrain identified the relevant five inputs in all five generated datasets, reaching 89.4% accuracy, compared with the tree baseline’s 50.1%. A compact relationship carried what the separate clues concealed.

The noisy board task ties the label to a particular arrangement: two adjacent pieces with a supporting piece below the left one, anywhere on the board. With 2,000 training examples, MorphoPlant’s 0.478 log loss approached the 0.477 score of a predictor given the world’s generating rule. The parity and board results average five generated datasets; tic-tac-toe averages five train/test splits, while each MONK result uses one split. These are concrete examples of the advantage a learner can gain when it finds the relationships a task depends on.

Return to the first pair of observers. They had accumulated plenty of evidence. What changed their ability was a new question about how the clues fit together.

Now imagine a machine that can keep proposing such questions, carrying uncertain answers, and letting useful relationships become the material of its next questions. The little gate gives us a place to start watching that process.