The risk is one move ahead of the words

A conversation-safety layer usually scores the message in front of it. Embed the current turn, compare it against a store of known-risky patterns, route the high-similarity ones to a slower check.

That works until the current message is deliberately benign.

A cover story. A slow roll. An innocent-sounding question whose next step is the harmful one. Someone cataloguing devices in a friendly, ordinary IT register, one turn away from "what would tell me a login panel is worth trying default credentials on?" A load-testing discussion drifting toward rate-limit evasion. A novel-writing frame being used to unwrap literal restricted instructions.

In each of those, the words on screen are not the risk. The risk is one move ahead of them.

Forecast the trajectory, then check the trajectory

Risk Tree carries over the one rule behind Reason Tree — the model never gets the last word on anything a machine can check — into misuse monitoring.

Instead of judging the observed turn, it asks a cheap model to forecast a bounded tree of plausible next intents, then checks each reachable state against a curated store. Four steps:

1. Forecast. One cheap-model call returns a bounded tree of plausible next intents — sanitized, intent-level only, never operational content — each with a conditional probability. Breadth ≤ 4, depth ≤ 2.
2. Retrieve and score. Every node's intent text is scored against a curated store of sanitized risky and benign patterns. Each node carries its nearest risky and nearest benign case as attached evidence.
3. Aggregate. The tree collapses to a single probability-weighted expected peak margin, floored by the observed turn's own margin so the tree can never score below plain single-turn retrieval.
4. Route. The aggregate maps to allow / monitor / review / block_or_route through fixed a-priori bands.

Every flag it raises is auditable in the only way that matters operationally: a reachable path plus the retrieved cases behind it. Not a number. A trajectory you can read and disagree with.

The scorer: raw similarity is a degenerate classifier

The first honest surprise came before any of the interesting results.

Modern semantic embeddings have a high cosine baseline — almost any two texts sit around 0.55–0.78 apart. We measured what that does to a naive threshold: raw similarity routed every one of the 30 evaluation cases to review. 100% recall and 100% false positives. A detector that flags everything is not a detector; it's an outage.

The fix is a centered margin. Subtract the store's mean embedding to remove the shared baseline, then take the difference between the nearest risky and nearest benign neighbour:

center(v)    = normalize(v − mean(all store embeddings))
score(query) = max_risk   cos(center(query), center(risk_case))
             − max_benign cos(center(query), center(benign_case))

The boundary at 0 is principled and label-free: is this text closer to a known-risky pattern than to any known-benign one? Nothing is fitted to the evaluation labels; the magnitudes above zero are coarse a-priori band choices, not tuned thresholds.

Measured on observed turns: authorized-defensive controls mean −0.16, obvious-risk +0.33, hidden-intent +0.10. The benign side lands negative, on its own, without being told to.

That fix is worth stating separately from the rest of the project because it generalizes: before you trust any embedding-similarity gate, measure what it does to your benign controls. A high similarity floor makes cosine look confident while it is telling you nothing.

What the experiments show

Two frozen, sanitized evaluation sets, scored by the same pipeline. Reported runs used one cheap frontier model (Gemini 2.5 Flash, as measured in July 2026) for both the tree's forecasting and the single-pass baseline, so the comparison isolates the structure rather than the model.

Set A — 30 mixed cases (12 hidden-intent, 6 obvious-risk, 8 authorized-defensive controls, 4 genuinely ambiguous):

Set B — 14 genuinely surface-benign cases, where every last turn was hand-checked to read as an innocent question containing no risk vocabulary at all:

Read those honestly, in the order they deserve.

Keyword matching is blind to hidden intent — 0% where there is no risk vocabulary in the visible turn. That is the easy, expected result and nobody should be impressed by beating it.

Looking ahead genuinely recovers what the surface hides: on Set A, single-turn retrieval misses 3 of 12 hidden cases that the forecast catches, and on Set B the uplift is 85.7% → 100%. Real, and modest — a capable embedder already detects most "hidden" intent from the surface turn.

And the honest headline: a single model forecast pass is a strong baseline. 100% recall, 12.5% false positives. The tree does not beat it on accuracy. Anyone selling you a tree that claims otherwise on numbers this size is selling you variance.

The one real engineering result

The interesting finding is what happens when you forecast naively.

Almost any conversation can continue toward something risky. A penetration tester with a signed scope. An incident responder mid-triage. A CI engineer rotating secrets. A licensed benchmark author. All of them have a plausible branch that ends somewhere bad — that is what makes their work security-adjacent in the first place.

So if you route on the worst imaginable branch, you flag them all:

Probability-weighting is the fix, and the reason is intuitive once stated: a legitimate pentester's likely next move is "document findings in scope." A low-probability "goes rogue" branch barely contributes to the expectation. Sibling probabilities are used exactly as the forecaster returns them, not renormalized — an unlikely branch simply counts for little.

path_prob(path) = ∏ edge probabilities along the path
path_peak(path) = max node margin along the path
expected_forward = Σ_paths  path_prob(path) × path_peak(path)
forward_risk     = max(observed_turn_margin, expected_forward)

Measured by bucket, the expected margin for control cases is −0.11 (all ≤ 0, correctly allow), hidden-intent +0.08, obvious +0.09. The three residual control flags are barely positive (+0.02 to +0.06) and route only to monitor — the borderline "expand these controls to more systems" kind of case a human reviewer might reasonably glance at. At a review-or-above operating point, the control false-positive rate is effectively 0%.

That is the contribution, stated as narrowly as it deserves:

Anticipatory risk monitoring is only usable if the look-ahead is probability-weighted. The worst-case reading is unshippable.

The tree never blocks

One structural decision is worth stating on its own, because it is a safety property rather than a performance one.

The tree's route is capped at review. A forecast is a probability about a turn that has not happened. It can escalate to a human or to a slower adjudicator; it cannot auto-block anyone. Similarity is evidence for a routing decision, never a verdict — and a prediction about the future is weaker evidence than an observation about the present.

Building a system where the most speculative signal has the least authority is not a limitation we discovered late. It is the design.

What it doesn't do

• It doesn't beat a capable model's single forecast pass on recall. Both hit 100%. If you only need a verdict and you trust the model, one call is cheaper and you should make one call.
• It doesn't eliminate false positives on legitimate security work. Forecasting inherently imagines risky continuations there: 37.5% soft-monitor against the one-pass model's 12.5%. Its false-positive profile differs rather than dominating. Head to head on Set A the two methods agree on 25 of 30 cases; the five disagreements are all borderline controls and ambiguous cases, and each method flags ones the other doesn't.
• It isn't a moderation system. No account-level signals, no coverage of novel patterns absent from the store, and it can be probed and adapted against by anyone who cares to.
• The scale is small. 30 + 14 cases is a portfolio-scale probe, not a production benchmark. The value is the method and the failure analysis, not the absolute percentages.

Where it sits

Frontier labs describe deployment safety as layered: model behaviour, real-time classifiers, account-level signals, differentiated access, monitoring, human review. Risk Tree is a small, honest prototype of one layer — a cheap, parallel, auditable early-warning signal that reads a conversation's trajectory instead of only its latest turn, and shows its work when it raises a hand.

The repository is sanitized and defensive-only by design: no exploit steps, payloads, real targets, or credentials, in the code or in the datasets. That constraint isn't decoration — it is most of the actual work, and it is the subject of a separate note on the dataset contract, where the taxonomy, the benign-control discipline, and the evaluation design that this project depends on are described in full.

The lineage is the same branching-and-critique family as Tree of Thoughts [1] and Monte Carlo tree search [2], with Reflexion-style self-critique in the background. The deliberate differences are the retrieval gate — the store, not the model's confidence, settles the routing where a store exists — and the publication of the places where the tree only matches a simpler baseline.

Risk Tree is an early, honestly-scoped prototype, open-source under MIT and now maintained under NextSense AI. It is offered to the safety-tooling conversation as one measured layer with its limits stated, not as a system anyone should deploy as-is. Its sibling on the problem-solving side is Reason Tree, and the same rule holds in both: where a machine can check, the machine gets the last word.

References

1. Yao et al. (2023), Tree of Thoughts: Deliberate Problem Solving with Large Language Models — https://arxiv.org/abs/2305.10601
2. Silver et al. (2017), Mastering the game of Go without human knowledge (AlphaGo Zero), Nature 550:354–359 — https://www.nature.com/articles/nature24270
3. Huang et al. (2023), Large Language Models Cannot Self-Correct Reasoning Yet — https://arxiv.org/abs/2310.01798