Every guard needs an exit: liveness in agent guardrails

Nineteen

One of our staleness guards once refused the same call nineteen times in a single session.

Every refusal was correct. The guard was doing precisely what it was designed to do: the state it was protecting had moved on, the call referenced a version that no longer applied, and letting it through would have written a stale value over a fresh one. Nineteen times the model asked. Nineteen times the runtime said no. Nineteen times the model — being a persistent, well-behaved thing that assumes a failure is worth retrying — rephrased slightly and asked again.

This is the most expensive lesson in our engineering history, and it cost us a while to even name it correctly, because from the inside it looks like a model problem. It is not. The model behaved reasonably. The guard was the bug.

Somewhere else in the same class, a real user abandoned a session after a confirm-and-refuse loop hit around ten. They did not file anything. They did not complain. They closed the tab, which is what people do, and the only reason we know about it at all is that we read session traces the way other teams read crash logs.

Safety without liveness

The distributed-systems framing is the one that makes this obvious, and I wish we had reached for it sooner.

Every correctness property splits into two kinds. Safety: nothing bad happens. Liveness: something good eventually happens. They are independent. A system can have perfect safety and zero liveness — a guard that refuses everything is flawlessly safe — and the failure mode of that system is not a crash. It is a hang.

Guardrails, as almost everybody writes them, are pure safety. You sit down to write one in the safety frame: can this proceed? The answer is a boolean. You return the boolean. You write a test that asserts the boolean is false in the dangerous case, it passes, you ship, and you are done.

The thing you have quietly assumed is that the caller can read. A boolean refusal is a fine contract for a program whose control flow you wrote, because when the branch returns false, the code you wrote goes somewhere else. But the caller here is a language model. It does not have a branch. It has an intent, a strong prior that failures are transient, and an unlimited supply of slightly-different phrasings. If the only information a refusal carries is can't do that, a persistent model will try again — and again — because from its position "try again" is the reasonable next move and nothing in your message argued otherwise.

A guard that is perfectly safe can be fatally un-live. And when it loops, it does not merely fail to prevent the bad outcome. It produces one: a session that goes nowhere, a user who watches an agent visibly stuck, a machine that has lost the plot in the most legible possible way. The guard manufactures the very failure it was meant to prevent.

The version of this I find most instructive is the human-facing one. Ten identical confirmations is not "a bit annoying." It is the moment a person concludes the software is broken and that continuing is a waste of their afternoon. The refusal was integrity-correct at every single step, and the aggregate was a lost user. Correctness that only holds per-call is not correctness.

Guardrails as liveness-safe state machines

Here is what we build now. None of it is clever; all of it is the sort of thing that only gets written down after it has cost somebody something.

Every deterministic refusal must name a converging, reachable next move. Not a suggestion, not a hedge — a move the caller can actually make that gets closer to done. "No" is not a complete guard output. "No, and here is the transition that resolves this" is. The test is convergence: if the caller does the named thing, is the guard now satisfiable? If the answer is "maybe, depending," you have written a hint, not an exit.

Identical re-emitted refused calls are detected and escalated, not answered again. A second identical refusal is information: it means the exit you named was not reachable, or not understood, or not the right one. A nineteenth is an outage. The runtime should notice repetition and change strategy — escalate, surface it to the human, take a different path — rather than serenely returning the same boolean forever. The cheapest possible version of this catches the entire class, and we did not have it.

Structurally remove the ambient parallelism that lets a model batch un-serializable calls. A lot of loop pathology is not one call retried; it is several conflicting calls issued at once, each of which is individually refusable and which together have no ordering that satisfies everyone. Constraining what can be attempted concurrently eliminates the situation instead of adjudicating it.

Never refuse silently. Every refusal carries a speakable line. In a voice workstation this is not a nicety: a silent refusal is indistinguishable from a crash. The user said a thing, nothing happened, nothing was said. There is no difference, from where they are sitting, between a guard doing its job perfectly and the machine being dead. If the runtime refuses, the human hears why, in one sentence, along with the way forward.

Design safety and liveness together, in the same review. This is the actual process change, and it is the one that stuck. The review question is no longer is this refusal correct? — that question is too easy and it is the one that let nineteen through. The question is: what happens if the caller does the obvious thing next? Ask that of every guard, out loud, and the un-live ones fall out immediately.

The same law applies to what the agent says

There is a second family of guards in an agentic product, and it obeys the identical law from a different direction. It took us a while to see them as one thing.

An unconstrained agent that can both act and speak will assert things that are not true. Ours has claimed an artifact was on screen before it rendered. It has invented a title. It has quoted a price it was explicitly told to keep silent. It has settled a "test" stage with zero actual runs and announced that the thing was tested and ready.

The instinctive response to each of these is to add a prompt rule. Do not mention prices. Do not say a thing is on screen unless it is. Do not claim testing without evidence. This feels like fixing it. It is not; it is a guard with no exit in a different costume — a rule with no structural change behind it, aimed at a caller that is under no obligation to comply.

Two things make honesty leaks resistant to the rule-writing instinct.

First, they are emergent, and there is frequently no bad string to grep for. The best example we have: a forbidden spoken price turned out to be dutifully read aloud from a mandatory notice that legitimately, correctly contained the number. Nothing in the system was misbehaving. No prompt rule was violated in spirit. The capability to say the forbidden thing had simply been handed to the model through a channel that existed for a good reason, and the model used what it was given. You cannot find that by searching the code, because the leak is behavioral, not textual.

Second, honesty is not copywriting. You cannot prompt your way to a guarantee. A prompt rule shifts a probability. If the property matters — and "this agent does not claim things that did not happen" matters more than almost anything else in a product people are supposed to trust — a probability is not the right instrument.

So the rule we learned, stated as bluntly as we say it internally: when a forbidden utterance appears, remove the surface that fed the model the capability, and pin the absence with an offline check. Not one more prompt rule. That last phrase is a stated engineering value in our codebase, and it has saved us from a great deal of comfortable, ineffective work.

Around it sits the rest of the honesty contract: every claim, stage completion, and spend bound to independent backend evidence recorded this session; model-initiated completions held to a stricter evidence bar than a genuine human tap, because a model has both the motive and the means to declare victory; AI-chosen defaults stamped as inferred and never stored as though the user had said them.

One discipline, two faces

Every integrity guard needs a reachable escape hatch. Every honesty rule needs a removed surface. These look like different problems — one is about refusing correctly, one is about speaking truthfully — and they are the same discipline seen twice.

Both are what happens when you stop treating a behavioral property as if it were a textual one. A guard that only emits text asks the model to behave; a guard that names a reachable transition changes what happens next. A prompt rule about honesty asks the model not to say something; removing the surface changes what can be said at all.

Put the property in the deterministic runtime. Let the model be free to be wrong — that freedom is not a concession, it is the design. And then close every incident the same way: a structural change, plus a new offline check that fails the build the next time the class tries to come back.

The number nineteen is on a wall in my head. It is the cheapest reminder I own that "safe" and "working" are two different words.

---

These are engineering-competence claims from our own shipping history, described at the level of problem classes and disciplines. The products are human-led and agent-assisted: the person decides, the agent does the heavy lifting and narrates what it did. The guarantees we make about generated output are structural — valid, renderable, playable — not that it is flawless.