Why give a language model a tree?

When you ask an AI model a hard question, it answers the way it was built to: one word at a time, in a single unbroken stream. Researchers call this autoregressive generation, and the "let me think step by step" text you sometimes see is called a chain of thought. It looks like reasoning, and often it works. But notice what the model can't do in that stream: it can't lay two ideas side by side. It can't put an idea down, try another, and come back. And it has no scratchpad other than its own words — so when a puzzle needs it to track a changing situation precisely, the words drift, and the model starts believing things about the situation that are no longer true. Once that happens, more thinking doesn't help. It just produces a longer, more confident version of being lost.

ReasonTree is a small open-source controller built around a different picture. Instead of one long monologue, the problem becomes a tree: the current situation is the trunk, each candidate idea is a branch, and every branch is played out on a real copy of the situation — in software terms, a state that actually updates, not a description of one. For each branch, something pushes back: an opponent's best counter-move, a failing test, a counter-example. Branches get scores, weak ones are pruned, and a strict budget of time and compute (bounded search) decides when to stop. Wherever a claim can be checked mechanically, a verifier — real executed code, not another opinion — gets the final word. Only then does the language model speak: one short call to explain the branch that survived.

Below are four real, captured experiments — a chess puzzle, a probability question, a debugging session, and a business decision. Each one exercises a different part of the tree, so along the way you'll see which specific feature earns its keep: executable states, verifiers, scoring, or the adversarial pushback. At the end, we place ReasonTree honestly among its research ancestors — Tree of Thoughts, Monte Carlo Tree Search, and friends — including a small head-to-head we ran for this page.

No chessboard here — this is the kind of question that lands in a Slack channel. A company's mail system sees about 1 phishing message per 100. It runs two phishing scanners. Each scanner, measured on its own, catches 90% of phishing emails and false-alarms on 5% of legitimate ones. This morning, both scanners flagged the same message, and the security lead asks the natural question: "Okay — what's the probability this one is actually phishing?" To be precise about what's being asked: of all the messages that ever get double-flagged, what fraction are truly phishing? Statisticians call this the posterior probability; everyone else calls it "how sure are we."

There is a textbook way to compute it, and it's genuinely tempting. Phishing is rare (1 in 100), so even a good scanner cries wolf a lot — that's the famous base-rate trap. But two independent alarms fix that: the odds that two unrelated scanners both false-alarm on the same innocent email are tiny (5% of 5%). Run the arithmetic — Bayes' theorem — and out comes a satisfying 76.6%. One catch. That word independent. These two scanners come from the same vendor and share code and training data. They are two witnesses who went to the same school and read the same books: when one is fooled, the other tends to be fooled by the very same trick. How much does their agreement really add, then? Here is the uncomfortable truth: the numbers given do not say. Nobody has measured how often the two scanners fail together, and no amount of cleverness can conjure that missing measurement out of thin air.

Watch what the raw model does with this. Midway through its thinking, it spots the trap — genuinely, in its own words:

ReasonTree's structure changes who is allowed to have the last word. "Assume independence and multiply" isn't the method here — it's just one branch, sitting next to a rival branch that asks: what do the stated facts actually pin down? That second branch goes to a verifier — a small piece of exact mathematics (bounds under unknown dependence, for the technical reader) that computes every probability consistent with the given facts. The answer it returns is not a number. It's a range, and the range is enormous:

Both ends of that range are real possibilities, and each has a plain-English story. The bottom (13.9%): if the scanners' mistakes overlap almost completely, the second flag adds nearly nothing — you effectively have one scanner wearing two badges, and one flag on a rare event mostly means "false alarm." The top (100%): if their false alarms happen never to coincide, then a double flag is practically proof. Between those extremes, any value fits the facts. So the only honest answer to "give me the exact probability" is: underdetermined — you cannot know from this data, and here is precisely the measurement that would settle it (how often do both scanners flag the same legitimate email?). ReasonTree calls this the gap rule: when two fact-compatible worlds disagree, name the gap instead of picking a world. That refusal is a feature. A wrong-but-confident 77% could justify auto-deleting customer email; "between 14% and 100%, and here's what to measure" leads somewhere useful.

A Python service ships a small feature: every user can set their own request timeout. One day a customer sets theirs to 60 seconds — and suddenly every user's timeout is 60. Settings are leaking across users who have never touched them. Here's the entire relevant code:

Two things in this snippet deserve a plain-language introduction. @functools.lru_cache() is Python's "remember the answer" sticker: the first time get_config("prod") runs, the result is stored; every later call skips the work and returns the stored result. Useful — but note the subtlety: it remembers the object itself, not a photocopy. Every caller receives the same dictionary. The second thing is overrides={} — a mutable default argument, one of Python's most famous gotchas, the thing every code reviewer has been trained to pounce on the moment they see it. It looks guilty. That's precisely what makes this case interesting: there's a well-known suspect standing right next to the actual culprit.

Credit where due: asked raw, Haiku correctly blamed the cache this time (19.64s, 1758 tokens, $0.0133). But look at what that answer is: a plausible story about code it never ran. You'd still have to verify it yourself before shipping the fix — and if it had picked the decoy instead, the answer would have read exactly as confident.

ReasonTree treats a debugging session as hypothesis testing. Each suspect becomes a branch, and the verifier executes the actual program. Branch one — "the cache hands out one shared dict" — is confirmed by reproduction: run the code, watch user 2 inherit user 1's timeout, observe that get_config("prod") really does return the identical object twice. Branch two — the famous decoy — is cleared by direct observation: after the failure, the default dict is still empty; it was never touched. Then the fix branch: apply copy-on-return, re-run the same scenario, watch the symptom disappear. The fix isn't recommended; it's demonstrated. All of the code execution took under a millisecond; one short model call writes it up. Total: 7.32s and $0.007 — roughly half the time and cost of the raw attempt, with proof attached instead of plausibility.

The pattern generalizes to most "how is this even possible?" bugs: settings that leak between users, tests that pass alone and fail together, caches that serve stale data, "works on my machine." Wherever a hypothesis can be turned into a few lines of executable check, the tree turns detective stories into experiments.

Our last case has no verifier at all — no chess engine, no formula, no test suite can tell you the right answer. That's deliberate: most real decisions live here, and it's important to see what a reasoning tree can and cannot honestly do in this territory.

The setup: you advise a six-person B2B software startup with 40 customers. A bug is corrupting non-English characters (ç, ö, 漢字…) in the data-export feature; two customers are affected. Option A: a hotfix can ship tomorrow — but deploying it means disabling every customer's data export for up to 24 hours, starting tomorrow morning. Option B: wait for next week's scheduled maintenance window and fix it with zero downtime. Sounds like a judgment call between speed and disruption — until you read the account notes: one of the two affected customers is mid contract-renewal, worth 30% of the company's revenue, and their procurement team runs a data-export compliance test tomorrow afternoon as part of the renewal checklist. Now look again. Option A knocks out the very feature that customer will be testing, during the very hours they'll be testing it:

To raw Haiku's credit, this time the single stream caught it — its very first step was to call the customer and move the test out of the deployment window:

The tree makes the catch structural instead of lucky. Two advocate branches each argue one option as hard as they can — and are required to list the biggest risks of their own plan against every stated fact. Then a skeptic pass cross-examines both branches, specifically hunting for facts that break them. In the captured run it found both of the things that matter. Against Option B, the fatal one:

And against Option A — subtler and easy to miss — the skeptic caught the plan contradicting its own numbers:

The shape to remember: any decision where dated facts can collide. A kitchen renovation scheduled just before a family event; switching payroll vendors the month of a tax deadline; accepting a job offer whose start date quietly overlaps a signed obligation. One stream may notice. A tree with a skeptic pass must check.

None of the individual ideas here are new, and it matters to say so plainly. Researchers have been giving language models more structure for years, and ReasonTree borrows from most of them:

So what is actually different? One design rule, applied stubbornly: the model never gets the last word on anything a machine can check. There's a reason for that rule. A well-known study — Huang et al., "Large Language Models Cannot Self-Correct Reasoning Yet" (arXiv 2310.01798) — found that when models review their own reasoning without external feedback, they fix wrong answers about as often as they break right ones. Every method above that relies on the model evaluating its own thoughts inherits that ceiling. ReasonTree's bet is to route around it: executable states and verifiers wherever the domain allows (Cases 1–3), and where nothing can be executed (Case 4), honest labels instead of borrowed confidence — plus two habits that are rarer than they should be: a hard compute budget on every search, and the "underdetermined" output when facts genuinely don't pin down an answer.

We didn't just assert the difference — we ran a small version of the comparison. We built a one-level Tree-of-Thoughts-style harness (three independent proposals, then one evaluate-and-vote call; our own emulation, not the original authors' code) and pointed it at the same tasks, same model, no tools:

Fairness requires the caveats: this is a tiny probe — two puzzles and one trap question, one model, our own ToT emulation rather than the official implementation, single runs. A proper head-to-head against the published Tree-of-Thoughts and Graph-of-Thoughts codebases, across many tasks and seeds, has not been done — by us or, to our knowledge, by anyone, for the verifier-gated setup. It's the obvious next experiment, and the repository is set up to run it. Until then, the claim we're comfortable making is narrow and matches everything above: structured branching helps a model surface doubts; only an executed check reliably converts doubts into a different answer.

Everything here is reproducible: the frozen puzzle set, the captured thought streams, the verifier code, the comparison harness, and the renderer that generated this very page are all in the open-source repository — github.com/uolkan/reason-tree.