Why give a language model a tree?

Ask a language model a hard question and it answers the way it was built to: one word at a time, in a single unbroken stream.

Notice what it cannot do inside that stream. It cannot lay two ideas side by side. It cannot put an idea down, try another, and come back to the first. It has no scratchpad except its own words — so when a problem requires tracking a precise, changing state, the words drift, and the model begins believing things about the situation that are no longer true.

Once that happens, more thinking doesn't help. It produces a longer, more confident version of being lost.

What we watched happen

We captured this failure verbatim, under conditions designed to remove every excuse.

A rated chess tactics puzzle — position 03kkE, Lichess rating 1809, roughly strong-club-player difficulty. One move wins; every alternative lets the opponent escape. No time limit, no cost budget, a 16,000-token output allowance, and the generation streamed and archived as it happened.

The model — a fast one, in a July 2026 snapshot — started well. It parsed the board correctly and listed sensible ideas. Then the drift began. It circled back to the same candidate a third time ("Let me go back to considering Rxc8+"). It "double-checked" the board and misremembered it ("wait — the king can't be on g7 if there's a pawn there"). After 127 seconds and 12,974 output tokens it committed, politely and confidently, to a move that was not merely wrong but illegal — its own knight was standing on the path. Its final verification step repeated the hallucination instead of checking it.

That last detail is the whole problem in miniature. The model did verify. It verified against its own corrupted memory of the board.

The fix is not a better prompt

Reason Tree is a small open-source controller built around a different picture of the same task. Instead of one long stream, the problem becomes a tree of real, executable states:

• candidate actions are enumerated by code, not recalled from memory;
• each candidate is played out on a real copy of the state;
• something pushes back on every branch — an adversary's best reply, a failing test, a counter-example;
• branches are scored, weak ones pruned, and a hard budget of time and nodes caps the search;
• a verifier — executed code, never another opinion — holds the last word.

Only then does the model speak: one short call to explain the branch that survived.

The design rule underneath all of it is deliberately stubborn:

The model never gets the last word on anything a machine can check.

The model's job moves from deciding to explaining. This distinction matters for how you read every number below. Nothing here makes a model smarter. A deterministic search chooses; the model narrates the choice.

The adapter contract is five functions

Chess is the microscope, not the product. It is simply the one domain where nobody can argue with the scoreboard. The controller itself is domain-neutral, and an adapter for a new domain is five functions:

state → candidate actions → executable transition → score / check → stop rule

That's the whole contract. For code repair: repository state → candidate patches → apply in a worktree → run tests → test/time budget. For scheduling: calendar state → candidate slots → timezone conversion → constraint check → first feasible slot. For data work: a query → execute on a sample → invariant check → stop.

When a domain cannot supply a real transition or a real check, the contract doesn't get faked. The output is labeled unverified or underdetermined and says which measurement would settle it.

The chess result, both tiers

The claim under test was not "this makes the model better." It was: does the same unchanged controller rescue models at whatever level they fail? Measured on frozen, pre-registered Lichess holdouts (CC0 data), answer key sealed until scoring, configuration tuned only on a separate development split and then frozen.

The paired breakdown matters more than the totals. At the first tier: both correct 1, tree-only 20, raw-only 0, both failed 4. At the second tier: both correct 3, tree-only 15, raw-only 0, both failed 7. Across both tiers, the number of puzzles the raw model solved that the tree missed is zero.

At each tier, the first ten "model failed, tree solved" cases were re-run end to end with the model as the explanation layer: 10/10 correct both times. At the higher tier that path ran at a mean $0.008 per case — roughly 5× faster and 12× cheaper than the raw attempts it replaces. The controller never had to override the model's restatement of the selected move.

A cross-provider observation, reported as measured and not explained: a different vendor's frontier model showed a different failure profile — a gradual decline with no cliff, 6/25 raw on the harder holdout — and got the same kind of uplift from the same adapter (tree-only 14, raw-only 2).

"It was about to answer"

The fairest objection to any wall-clock cap is that the model was one sentence from the solution. We closed it with a three-case uncapped probe: 600-second wall, no cost budget, streams archived verbatim.

Every call terminated on its own. The failure is not starvation of compute — it is miscalibrated confidence. Given unlimited time, the raw model converges, usually to a wrong move, once to an impossible one, each time behind a confident closing justification. On the one puzzle it did solve, it took 191 seconds and $0.103 against the controller's 11 seconds and $0.0146 for the same answer.

Three cases, one trial each. It bounds the "just wait longer" objection; it does not estimate an uncapped accuracy rate.

The negative results

These are the part of the project we'd keep if we had to discard everything else. A technique that only publishes its wins is a marketing claim wearing a lab coat.

Where the tree lost.

The AIME-with-tools result is the cleanest refutation of the broad pitch. Given tools, a one-shot agent writes its own checker and nearly saturates the benchmark. There is nothing left for an external verifier to contribute, and the tree adds only cost plus a schema/protocol failure surface. Against matched-compute sampling without tools, plain sampling beat the tree. These runs are exploratory — mostly one trial per condition — but their direction is unambiguous, and it rejects the claim that explicit tree search generically improves a tool-using model.

Prompt-only trees solved nothing. Telling the model to "think in branches," even handing it every legal move, failed exactly like the raw call: 0/2 on re-test, both hitting a 90-second cap. The structure has to be in the machinery. Phrasing is not architecture.

Deeper search scored worse. Before opening the harder holdout, we compared adapter configurations:

Odd depths end the line on the searcher's own move — a textbook horizon effect — and the longer configurations increasingly hit the wall clock with an incomplete root. Bounded and complete beat big and truncated.

An operational lesson that changed a result. An earlier rescue run used five parallel workers; CPU contention pushed two searches into the time cap with incomplete roots and produced two wrong selections. Wall-clock-capped searches are only reproducible when run serially. Both runs are archived. A node-cap-only mode is the correct fix for parallel harnesses.

And the tree cannot certify judgment. For a decision with no mechanical verifier — ship the fix now or wait a week? — advocate branches argue each option and a skeptic pass cross-examines both against every stated fact, but the recommendation ships with its assumptions labeled and marked unverified. A single stream may notice a buried collision between two dated facts. A skeptic pass must check. That is a real improvement and it is not verification, and the output says so.

How this relates to Tree of Thoughts, MCTS, and friends

Every primitive here has ancestry. Branching and scoring over reasoning states is Tree of Thoughts [1]. Graph-structured refinement is Graph of Thoughts. Adversarial state search with budgets is the Monte Carlo tree search lineage that produced AlphaGo Zero and AlphaZero [2][3]. Self-critique is Reflexion.

The deliberate difference is one line: in Tree-of-Thoughts-family methods, the model grades its own thoughts. Huang et al. examined that assumption and found that self-correction without external feedback fixes wrong answers about as often as it breaks right ones [4]. Search over a noisy evaluator inherits the noise and multiplies the cost.

We probed the difference directly with our own emulation of a propose-×3-and-vote harness. On chess rescues it went 1/2, at roughly $0.20 and four minutes per puzzle. On a correlated-evidence probability trap it named the correct conclusion — "that argues for a range, not a specific guess" — and then answered "15.4%" anyway.

Branches without a verifier are opinions with better formatting.

A full head-to-head against the official Tree-of-Thoughts and Graph-of-Thoughts codebases hasn't been run — not by us, and to our knowledge not by anyone for this verifier-gated setup. It's the obvious next experiment, and the repository is set up to run it.

Status and honest scope

Reason Tree is an early, honestly-scoped prototype, published open-source under MIT and now maintained under NextSense AI. It began as independent, AI-safety-adjacent research: the underlying question — when should a system be allowed to act on a model's confidence? — is a safety question before it is a performance one.

The known limits are stated plainly. The chess holdout is 25 cases from one narrow rating slice, one dated model snapshot, one trial per condition. The adapter is intentionally small and still missed 4/25. Six searches returned a partial root frontier at the time cap; iterative deepening should guarantee a complete shallower frontier first. The ten-case rescue table is a selected demo, not an independent accuracy estimate. And the broader AIME and ARC experiments showed no general reasoning uplift.

The claim that survives every control we ran is narrow, and we'd rather publish the narrow one:

Externalizing real state transitions, counter-branches, scoring, and stop rules turns a fast model's opaque attempt into a bounded, inspectable, checkable workflow. The gain lives in the executable adapter — and where no check can be executed, the output says so instead of borrowing confidence.

The full protocol, every cost table, the archived thought streams, and the frozen manifests are in the repository. The illustrated tour walks through four captured experiments end to end, and the rescue case puts the raw monologue and the bounded tree side by side on the same puzzle.

The same rule carries into a sibling project on the monitoring side: Risk Tree forecasts where a conversation could go next and scores the reachable states against a curated store, rather than trusting a model's read of the message on screen. For the wider research context — how training-time and inference-time approaches to reasoning diverged — see Three roads to reasoning in language models.

References

1. Yao et al. (2023), Tree of Thoughts: Deliberate Problem Solving with Large Language Models — https://arxiv.org/abs/2305.10601
2. Silver et al. (2017), Mastering the game of Go without human knowledge (AlphaGo Zero), Nature 550:354–359 — https://www.nature.com/articles/nature24270
3. Silver et al. (2017), Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (AlphaZero) — https://arxiv.org/abs/1712.01815
4. Huang et al. (2023), Large Language Models Cannot Self-Correct Reasoning Yet — https://arxiv.org/abs/2310.01798
5. Lichess open puzzle database (CC0) — https://database.lichess.org/