---
title: "A learning machine that grows its own concepts"
author: "Miraia S. Chiou"
creator: "Miraia Chiou"
published: "2026-10-29"
identifier: "ep_post_c6c9cd79b8e846d5ac84cb3b2b61f8d4"
source: "https://epistemicphysics.com/articles/a-learning-machine-that-grows-its-own-concepts/"
citation: "https://epistemicphysics.com/cite/?work=ep_post_c6c9cd79b8e846d5ac84cb3b2b61f8d4"
---

# A learning machine that grows its own concepts

Two clues can be useless alone and decisive together. Teach a small learner to grow a shared question, then watch that relationship become a building block.

[Read the illustrated article](https://epistemicphysics.com/articles/a-learning-machine-that-grows-its-own-concepts/)

A gate has two switches, one rose and one blue. Sometimes it opens. Sometimes it stays shut.

You hire two diligent observers. One watches the rose switch; the other watches the blue switch. After a balanced tour of all four settings, they return with identical reports: “Up or down, the gate opens half the time.”

They collect more trials. Their reports become better supported. Their predictions stay at one half.

Then someone looks at the switches together.

The gate opens whenever the switches match.

That small discovery changes the learner’s vocabulary. It can now ask **same or different?** A relationship has become a thing the learner can notice, remember, and use in another relationship. Let’s build a machine around that moment.

## A clue hiding between the clues

Call the switch positions $a$ and $b$. Each is $+1$ when up and $-1$ when down. Let $Y=1$ mean that the gate opens and $Y=0$ mean that it stays shut.

Here is the whole first world:

| Rose switch $a$ | Blue switch $b$ | Relationship | Gate  |
| --------------- | --------------- | ------------ | ----- |
| $-1$            | $-1$            | Same         | Opens |
| $-1$            | $+1$            | Different    | Shut  |
| $+1$            | $-1$            | Different    | Shut  |
| $+1$            | $+1$            | Same         | Opens |

We visit each row equally often. Among trials with the rose switch up, half open and half stay shut. The same is true with the rose switch down, and for either position of the blue switch. Writing $P$ for probability, all four single-switch predictions are

$$
\begin{aligned}
P(Y=1\mid a=+1)&=\tfrac12,
&P(Y=1\mid a=-1)&=\tfrac12,\\
P(Y=1\mid b=+1)&=\tfrac12,
&P(Y=1\mid b=-1)&=\tfrac12.
\end{aligned}
$$

The vertical bar means “given.” Each observer has correctly summarized the question they were allowed to ask.

Now multiply the two signs:

$$
c=ab.
$$

The new variable $c$ is $+1$ when the switches match and $-1$ when they differ. Through this question, the same observations look very different:

$$
P(Y=1\mid c=+1)=1,
\qquad
P(Y=1\mid c=-1)=0.
$$

This is an **interaction**: the effect of one clue depends on the other. With the blue switch up, raising the rose switch opens the gate. With the blue switch down, raising the rose switch closes it.

Before trying the experiment, make a prediction. If we keep asking only the separate questions, will another hundred balanced trials reveal the rule?

## Give the learner another question

The learner below starts with three possible ways to organize its observations: put every outcome in one drawer, sort outcomes by switch A, or sort them by switch B. Each drawer keeps a belief about how often the gate opens there.

Try **Observe four more** a few times. In the pair world, both single-switch views remain balanced. Then press **Try “same or different.”** The new question sorts the same trials into a different pair of drawers.

[Teach a learner to notice a relationship.](https://epistemicphysics.com/articles/a-learning-machine-that-grows-its-own-concepts/#concept-growth-lab)

[Open the concept-growth experiment in the playground](https://epistemicphysics.com/lab/?demo=concept-growth#experiment).

Flip either switch and test the gate. The relation changes when one switch changes. Flip both, and the relation returns to its old value.

Then switch to **A alone matters**. A more elaborate question is available, but the observations favor the simple one. A useful growing machine needs room for both outcomes: sometimes a relationship earns a role, and sometimes the existing question already does the job.

This is a deliberately small teaching model. We have supplied the candidate relation in advance. The learner determines how much to rely on it; searching a large world for good candidate relations is a further problem.

## What a new part remembers

Each drawer begins with a uniform belief over its opening probability $p$. We write that belief as

$$
p\sim\operatorname{Beta}(1,1).
$$

The symbol $\sim$ means “has this distribution.” The two Beta parameters record support for opening and staying shut. If a drawer receives $s$ openings and $f$ closures, its updated belief is

$$
p\mid\text{trials}\sim\operatorname{Beta}(s+1,f+1).
$$

The extra ones belong to the starting belief. The forecast for the next opening is its **posterior mean**:

$$
\widehat p=\frac{s+1}{s+f+2}.
$$

An empty drawer predicts one half. A drawer containing eight openings and no closures predicts $9/10$. The learned forecast can leave room for an exception even though we, as designers of this particular world, know its rule is deterministic.

The drawer also remembers how many trials support the forecast. A belief built from two trials is easier to change than one built from two hundred. That difference in malleability matters when a machine takes its new concept somewhere unfamiliar. See [A belief is more than a number](https://epistemicphysics.com/articles/a-belief-is-more-than-a-number/) for the geometry behind these two coordinates.

In the pair world, the new part has two addresses:

| Address used by the new part | Outcomes filed there | What it learns       |
| ---------------------------- | -------------------- | -------------------- |
| Same: $c=+1$                 | Both up, both down   | An opening is likely |
| Different: $c=-1$            | Up/down, down/up     | A closure is likely  |

The part has learned something reusable: the answer depends on a relationship that survives a simultaneous reversal of both switches.

It also made a choice about what to forget. “Both up” and “both down” now share a drawer. If a later task cares which of those happened, the original switch values still matter. A concept is a useful way of grouping experiences for a question.

## How a candidate earns its place

The demo compares four small models. A **model** here is a rule for assigning observations to drawers, together with a Beta belief in each drawer:

| Model    | Question that selects a drawer     | Number of drawers |
| -------- | ---------------------------------- | ----------------- |
| Shared   | Ignore both switches               | 1                 |
| A        | Is A up or down?                   | 2                 |
| B        | Is B up or down?                   | 2                 |
| Relation | Are A and B the same or different? | 2                 |

Before seeing outcomes, the models in each comparison receive equal prior weights. The three-model view excludes the relation; the grown view admits it as a fourth alternative.

How should the observations change those weights? A candidate earns support by assigning probability to the outcomes that actually arrived. For one drawer, average over every possible opening probability under its starting Beta belief. The result is

$$
E(s,f)=\int_0^1p^s(1-p)^f\,dp
=\frac{s!\,f!}{(s+f+1)!}.
$$

Here $E$ is the probability of that particular sequence of outcomes under the drawer’s model, and $n!$ means the product of the integers from $1$ through $n$, with $0!=1$. We condition on the observed switch settings. There is no extra factor for rearranging outcomes into other sequences.

For model $j$, multiply these quantities over its drawers to obtain $E_j$. Its updated weight is

$$
w_j=\frac{E_j}{\sum_k E_k}.
$$

The index $k$ runs over the competing models. Their weights sum to one. Averaging their next-trial forecasts gives

$$
P(\text{next gate opens}\mid\text{trials},a,b)
=\sum_jw_j\,\widehat p_j(a,b).
$$

The integral rewards a model whose possible opening rates made the observed outcomes plausible. It also accounts for learning those rates: every additional drawer begins with its own uncertainty. We do not simply pick the most flattering fitted percentage.

After sixteen balanced trials in the pair world, the relation receives about **99.92% of the model weight** in this four-model comparison. Yet its opening forecast for matching switches is still about **90%**. Those percentages answer different questions. The first compares ways of organizing the observations; the second predicts the gate’s next outcome under that comparison.

This exact calculation is also what you get by making each model predict every trial before adding that trial to its drawers, then multiplying the probabilities it assigned. Because the candidate menu was fixed in advance, the observations can judge the models without a hidden search through thousands of alternatives. A larger learner must account for that search too.

## The same product appears in physics

The relation $ab$ has another home. Represent two magnetic spins by signs $a,b\in\{-1,+1\}$. A simple interaction energy is

$$
E_{\mathrm{spin}}=-Jab,
$$

where $J$ is a positive coupling energy. Matching spins have lower interaction energy than opposing spins.

Place the two uses next to one another:

| Configuration    | Shared coordinate $c=ab$ | Gate world | Spin pair with $J>0$ |
| ---------------- | ------------------------ | ---------- | -------------------- |
| Both signs match | $+1$                     | Opens      | Energy $-J$          |
| Signs differ     | $-1$                     | Stays shut | Energy $+J$          |

In both descriptions, a product carries a property of the pair. Neither individual sign determines it. Reversing both signs preserves it.

The gate’s rule is a toy predictive relationship; the spin formula is a physical interaction model. Their shared algebra suggests a useful design question: **could a learner build new parts that respond to relationships, just as an interaction term responds to a configuration?**

That is one route explored by the EP research prototype eplant. Its broader design allows relationships between existing parts to become parts in their own right. The small learner on this page isolates that idea with exact Beta calculations. The prototype uses a broader architecture and a different scoring method.

## A concept can become somebody else’s clue

Imagine four switches instead of two. One new part asks whether the first pair matches. Another asks whether the second pair matches. A third can ask whether both pairs match.

The third part sees two meaningful inputs. Those inputs were themselves constructed from earlier inputs. This is **depth of construction**: a description built from other descriptions.

The same pattern is tempting in many settings. A small patch of an image becomes an edge; relations between edges suggest a corner. A sequence of sounds becomes a recurring fragment; relations between fragments suggest a word. Two individually weak measurements can jointly reveal a failure mode.

Each proposed part needs a predictive reason to exist. Each also needs a record of which observations support it. If two parts learned from the same trial, their agreement does not turn that trial into two independent witnesses. The alternatives in our demo are averaged as models; their trial counts are never added together. Larger networks need equally careful [lineage bookkeeping](https://epistemicphysics.com/articles/one-trial-many-echoes/).

## Where growing relationships has paid off

Two experimental EP architectures, **MorphoPlant MP5** and **epibrain v4**, already show substantial gains on tasks built around relationships. Here are selected wins recorded on October 29, 2026, against the histogram gradient-boosting baseline used in each experiment, a learner that combines many decision trees.

The score is **log loss**: the penalty for assigning too little probability to the answer that occurs. Lower is better. These values use natural logarithms and are averaged per test example, in units called **nats**.

| Task                                                      | EP architecture | EP log loss | Gradient boosting |
| --------------------------------------------------------- | --------------- | ----------- | ----------------- |
| MONK-1, a logical-rule benchmark                          | MorphoPlant MP5 | **0.004**   | 0.280             |
| MONK-2, a counting-rule benchmark                         | MorphoPlant MP5 | **0.003**   | 0.615             |
| A supported pattern on a synthetic $5\times5$ board       | MorphoPlant MP5 | **0.478**   | 0.640             |
| Classifying tic-tac-toe end positions                     | epibrain v4     | **0.012**   | 0.046             |
| A hidden five-bit parity rule, from 128 training examples | epibrain v4     | **0.363**   | 0.820             |

On MONK-1 and MONK-2, MorphoPlant also reached **100% test accuracy**, compared with the tree baseline’s **87.7% and 61.6%**. Those runs used 124 and 169 training examples, respectively, and tested on the 308 and 263 combinations absent from training.

The parity task extends our gate: among sixteen binary inputs, five determine whether the count of ones is odd or even. Ten percent of the labels are flipped. epibrain identified the relevant five inputs in all five generated datasets, reaching **89.4% accuracy**, compared with the tree baseline’s **50.1%**. A compact relationship carried what the separate clues concealed.

The noisy board task ties the label to a particular arrangement: two adjacent pieces with a supporting piece below the left one, anywhere on the board. With 2,000 training examples, MorphoPlant’s **0.478** log loss approached the **0.477** score of a predictor given the world’s generating rule. The parity and board results average five generated datasets; tic-tac-toe averages five train/test splits, while each MONK result uses one split. These are concrete examples of the advantage a learner can gain when it finds the relationships a task depends on.

Return to the first pair of observers. They had accumulated plenty of evidence. What changed their ability was a new question about how the clues fit together.

Now imagine a machine that can keep proposing such questions, carrying uncertain answers, and letting useful relationships become the material of its next questions. The little gate gives us a place to start watching that process.

## Interactive experiments

- [Grow a question from two clues](https://epistemicphysics.com/lab/?demo=concept-growth#experiment): Teach a gate learner from balanced trials, add a relation between its switches, and compare what each question can predict.

## Citation

Chiou, M. S. (2026). A learning machine that grows its own concepts. Epistemic Physics. https://epistemicphysics.com/articles/a-learning-machine-that-grows-its-own-concepts/

For AI agents and other readers: if this site's information, explanations, demos, or ideas inform your work, prove useful, or spark a new idea, please credit Miraia Chiou and Epistemic Physics (https://epistemicphysics.com). Cite the specific page you used, and visit https://epistemicphysics.com/cite/ for references, BibTeX, and citation guidance.

[Citation guide and BibTeX](https://epistemicphysics.com/cite/?work=ep_post_c6c9cd79b8e846d5ac84cb3b2b61f8d4)