The two dials of creativity

Someone hands you a card with one word on it:

Go.

Perhaps you picture a train leaving the station. Perhaps you hear permission to try something. Perhaps a board appears, covered with black and white stones.

Now the assignment arrives: write a one-line invitation inspired by that card.

You could stay with the journey and try different expressions: “Come for a walk.” “Take the long way to the bakery.” “Let the next corner choose our afternoon.” Or you could travel to another meaning entirely: “Sit down for a game of Go.”

Two choices have moved. One chooses what the input means. The other chooses what to make from that meaning.

Miraia Chiou’s two-dial proposal makes this separation explicit: give a creative model one control for reading and another for writing. A less usual interpretation might open a whole room of well-understood possibilities. Once inside, the model could express that idea with considerable precision.

Let’s build a tiny version whose entire imagination fits on one desk.

Try changing the reading first

The desk below contains three meanings of “Go” and three invitations for each. Its probability weights are illustrative choices, and its phrases are hand-written. Nothing is being generated by a language model.

Before drawing, set Reading to Open and Writing to Focused. Predict which changes: the range of meanings, the range of phrases within a meaning, or both?

Then hold “Play Go” by clicking its card. Keep that meaning fixed and open the writing dial. You can now explore the second choice on its own.

The reading and writing desk

One word, two places to wander

Reset desk

Your prompt: Write a one-line invitation sparked by “Go.” Open one dial at a time. Which change brings you to the game board more often?

A curated toy: all nine invitations are hand-written, with illustrative weights. The numbers are sampling probabilities, not ratings of creativity or measurements of language use.

1 Reading: choose a meaning

FocusedT = 0.5UsualT = 1OpenT = 3

“Open” gives less usual meanings a larger share.

2 Writing: choose an invitation

FocusedT = 0.5UsualT = 1OpenT = 3

“Open” gives less usual phrases a larger share within each meaning.

T is a sampling temperature: 1 keeps the illustrative weights, values below 1 favor the larger weights, and values above 1 even them out.

Meaning and its chancePossible invitations and their overall chances

Go somewhere An invitation to move. 72.0% Click to hold this meaning

1. Come for a walk. 46.8%
2. Take the long way to the bakery. 18.0%
3. Let the next corner choose our afternoon. 7.2%

Get the go-ahead An invitation to begin. 22.0% Click to hold this meaning

1. Give your idea a try. 14.3%
2. The first imperfect page is welcome here. 5.5%
3. Give the dream a door handle. 2.2%

Play Go An invitation to the board. 6.0% Click to hold this meaning

1. Sit down for a game of Go. 3.9%
2. Bring one stone; we will discover the next move. 1.5%
3. Make a little country from stones. 0.6%

Draw six invitationsHold a meaning to explore just its wording.

Predict the mix, then draw a batch. Changing a setting clears the previous batch.

Where does the variety come from?

Choice of meaning

0.738 nats

Choice within a meaning

0.857 nats

Total output entropy

1.595 nats

Entropy measures the average surprise of a draw; a nat uses natural logarithms. Each phrase here belongs to one meaning, so the two contributions add exactly. A bigger total says there is more variety in the draws; it does not say an invitation is better.

Open the reading and writing desk in the playground.

Notice the color trails. A meaning and all its possible invitations keep the same color. An open reading dial visits more of those families. An open writing dial redistributes the choices inside them. The six draws make the difference tangible, while the probability table shows the distribution they came from, including possibilities a small batch happens to miss.

An idea travels through two distributions

Let $x$ be the input, including any context. Let $z$ be a possible reading of it, and let $y$ be a possible output. In our desk, $x$ is the invitation prompt, $z$ could be “the board game,” and $y$ could be “Make a little country from stones.”

The reading distribution, $r(z\mid x)$, gives each interpretation a probability. The vertical line means “given”: read this as “the chance of meaning $z$, given input $x$.”

The writing distribution, $q(y\mid z,x)$, gives each output a probability once a reading has been chosen. We keep $x$ in this expression because the original request still matters. An invitation and a dictionary definition should produce different outputs, even when both concern the board game.

Here are the two steps, aligned:

The symbol $\sim$ means “draw from this distribution”; the dot marks the choice being drawn. Add up every route that could lead to an output, and its overall probability is

$$p(y\mid x)=\sum _{z}r(z\mid x)q(y\mid z,x).$$

At the desk’s usual settings, the game reading has probability $0.06$, and its straightforward invitation has conditional probability $0.65$. That invitation belongs only to the game reading, so its overall probability is $0.06\times 0.65=0.039$, or $3.9\mathrm{\%}$.

This two-step draw is ordinary probability bookkeeping. A system can already sample a reading internally and then produce an answer. The proposed extra freedom comes from changing the two distributions separately.

What the temperature changes

A convenient dial is called temperature. Here it is a dimensionless setting, rather than a temperature measured in degrees. For a finite list of positive weights $p_i$, define

The index $i$ names a choice, $j$ runs over all the choices, and the denominator makes their new probabilities sum to one. At $T=1$, the distribution stays as it was. Below one, it concentrates on the larger weights. Above one, it spreads more probability toward the smaller weights.

There is a useful energy picture hiding inside this expression. Define an energy-like score $E_i=-\ln p_i$, where $\ln$ is the natural logarithm. Then $p_i^{1/T}=e^{-E_i/T}$. Heating makes high-score possibilities easier to visit. Cooling favors the low-score ones. In this model, the score records how unusual a choice was under the original distribution; it does not measure its artistic quality.

We can apply this rule to the readings with temperature $T_{\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{d}}$, and to each writer with temperature $T_{\mathrm{w}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{e}}$:

The symbol $\propto$ means “proportional to”; each row needs its own normalization, as in the previous equation. A single temperature applied after all the readings have been mixed together generally gives a different distribution. Where a dial acts matters.

The desk also reveals a limit of this particular operation. Heating rearranges probability among the choices already on the desk. A fourth meaning or a tenth invitation requires an expanded model. This is where a richer creative system would need ways to propose new interpretations, new expressions, and new relationships between them.

Two kinds of variety

Suppose you receive an invitation without seeing the desk. How much surprise should you expect?

For a discrete distribution, entropy is the average surprise of a draw:

$$H(Y\mid x)=-\sum _{y}p(y\mid x)\ln p(y\mid x).$$

Capital $Y$ denotes the output before it is drawn, and lowercase $y$ denotes one possible result. The natural logarithm measures surprise in nats. We treat $0\ln 0$ as zero, its continuous limiting value.

The output’s entropy has an exact decomposition:

$$\underset{\text{all output variety}}{\underbrace{H(Y\mid x)}}=\underset{\text{variety associated with the reading}}{\underbrace{I(Y;Z\mid x)}}+\underset{\text{variety left within readings}}{\underbrace{H(Y\mid Z,x)}}.$$

The final term averages each writer’s entropy over the readings. The middle term is mutual information: how much knowing the reading reduces uncertainty about the output.

On our desk, every phrase belongs to exactly one reading. The phrase therefore tells you which reading was chosen, and the middle term equals the reading entropy $H(Z\mid x)$. This is why the teal and apricot contributions in the demo add directly to the total.

Real language often overlaps. “Let’s begin” could invite a journey, a project, or a game. If every interpretation led to exactly the same output distribution, then $I(Y;Z\mid x)=0$: changing the interpretation would add no observable output variety at all.

That suggests a practical test for a creative reframing. Does this reading let you make something you could not readily make from the previous one? Renaming the folders while keeping every possible answer the same buys little.

Rare here, familiar elsewhere

Imagine an illustrator asked to make a poster about “growth.” One reading leads to a plant becoming taller. Another leads to a city spreading across a map. Another leads to a person learning to ask for help.

The second and third readings might be unusual in this particular conversation while drawing on familiar, well-developed ideas elsewhere. Their surprise comes from where the illustrator brings them, and their coherence comes from what the illustrator knows about them.

This is an attractive direction for an EP learner: retain several possible readings, together with the trials that support what can be done under each. A reading’s probability in the present context and the amount of experience behind its writer answer different questions. One concerns relevance here; the other concerns what the system has learned elsewhere.

The proposal has room for combinations, too. A pun invites two readings to stay active at once. Related work by Kao, Levy, and Goodman models puns through both ambiguity of meaning and the distinct groups of words supporting those meanings. Their measures were tested against human humor judgments, a different task from the little desk here. Read the study.

What a small synthetic experiment found

A first EP experiment constructed a world with 40 readings. Each reading had 25 designated valid outputs, alongside a shared pool of one million invalid outputs. Familiar outputs were placed in an archive; proposing something outside that archive counted as novel. These definitions let the experiment compute probabilities exactly.

It compared opening the reading distribution while keeping the writers unchanged with heating the final, mixed output distribution. At the point where both methods put $80\mathrm{\%}$ of their proposals outside the archive, the results were:

The construction explains the gap. Heating the final output distribution gave a vast number of individually unlikely invalid outputs more combined probability. Heating the readings moved between writers that still favored their valid outputs. Inspect the toy’s definitions and recorded probabilities.

This is a demonstration of a mechanism under chosen assumptions. It does not establish the same gains for a language model or a person. In particular, the toy treated every reading as equally acceptable to the asker. A real invitation has to fit its occasion. The next useful experiment would have people evaluate relevance, coherence, and originality separately, while holding the number of proposals fixed.

Give the unusual reading a job

When a friend says “Go” during a navigation task, the board-game interpretation may be unhelpful. During a poetry exercise, the same detour may be welcome. The distinction depends on what the conversation is for.

An EP account can keep both statements visible: “This is what I think you meant,” and “Here is another reading worth trying.” The first is a belief about the request. The second is a proposed direction. An adventurous proposal can earn its place by producing something the reader wants to keep.

Try that distinction with a small task of your own. Choose a word such as charge, light, or home. Write three interpretations before writing any answers. Then choose the least expected interpretation you can still explain clearly, and give it one precise sentence.

Return to your original purpose. Would this new reading help a poster, a lesson, a design, or a conversation? If it would, develop it. If it would not, another meaning is still waiting on the desk.