Research

Engineering

Voice is a command surface, not a companion: what always-on gets wrong about coding agents

Full-duplex voice closed a conversational gap measured in milliseconds; long-running coding agents fail in a gap measured in minutes — which is why an always-on microphone is the wrong default for work that runs for four.

Volkan Turk · Founder & principal architect, NextSense AI

1. What shipped

Voice arrived in coding tools gradually and then all at once.

The gradual part was dictation. OpenAI's Codex added push-to-talk into the composer on 2026-02-12, then spent months on the unglamorous surface around it — clearer microphone and quota errors, dictation cleanup, a configurable dictionary for names, file paths and code symbols. Speech in, text out, one turn at a time.

The all-at-once part was GPT-Live on 2026-07-08, followed on 2026-07-23 by ChatGPT Voice landing in the desktop app and wiring straight into Codex and Work:

"Powered by GPT-Live, ChatGPT Voice lets you talk through work and coordinate tasks in Chat, Work, and Codex… Voice is available with Plus, Pro, Business, Edu, and Enterprise plans in the desktop app and through Remote on iOS."

Or, as the community announcement put it the same day: "Control your computer and direct multiple agents running in ChatGPT Work or Codex, using just your voice."

The model underneath is a real step change, and OpenAI describes what it optimises for with unusual clarity:

"GPT-Live is built on a full-duplex architecture, meaning it can listen and speak at the same time. During conversations, GPT-Live can show it's paying attention with phrases like 'mhmm' or 'yeah', engage in quick back-and-forth, or just stay quiet when you need a moment to think."

Read that list again, because it is a precise statement of the problem being solved. Backchannels. Quick back-and-forth. Staying quiet for a moment. These are turn-taking behaviors, and they operate on the timescale of conversational rhythm — a few hundred milliseconds to a couple of seconds.

What happens when the work is bigger than that:

"For questions that require web search, deeper reasoning, or more complex work, it delegates to our latest frontier model behind the scenes and brings the result back into the conversation when it's ready. While it works, GPT-Live can keep talking with you and maintain the flow of conversation."

Keep talking with you. The remedy for a long task is to keep the conversation going while it runs.

And then, in the same document, the sentence this entire essay is a footnote to:

"Over time, we believe this research will also unlock the ability to use voice for increasingly complex, longer-running, and more agentic work."

That is OpenAI filing longer-running, more agentic voice under future work. They are right to.

2. First, the steelman

Nick Dobos, on X:

"Voice Vibecoding is a massive paradigm shift. New Codex voice mode is a big upgrade! … Previously codex voice had a 10 minute limit, and you had to reopen codex and press send and wait for a transcription every time. Now it just stays on forever, and you don't need to press any buttons or reopen the app."

He is right, and it is not a small thing. Look at what he is actually praising: not intelligence, not voice quality — the removal of ceremony. A hard session limit, gone. A button pressed for every utterance, gone. Reopening the app, gone. Waiting for a transcription before you could continue, gone.

Anyone who has used a push-to-talk coding tool for a week knows how much of the experience is friction tax rather than conversation. Removing all of it at once genuinely feels like a different product, and a critique that starts by pretending otherwise is not worth reading.

So: the ceremony was a real problem, and always-on really does fix it. My argument is that it is the right fix aimed at the wrong gap.

3. Two different gaps

There are two silences in an AI coding session and they are not the same silence.

The conversational gap is the pause between one person finishing and the other starting. Measured in hundreds of milliseconds. When it goes wrong the interaction feels stilted and laggy. Full-duplex architecture with backchannels is an excellent solution to it.

The agentic gap is what happens after you tell a coding agent to do something substantial. It reads, thinks, writes, builds, hits an error, recovers. Measured in minutes — and it has a property the conversational gap does not: the human is supposed to be silent. Not politely waiting for a turn. Actually done talking, with nothing useful to add until there is something to look at.

That is the crux. With a thinking, building agent the interaction shape is not conversation at all. It is: issue an intent, then fall silent. A burst of dense speech, then minutes of nothing, then possibly another burst. Silence is not a gap in the conversation; silence is the normal operating state of the workflow.

Backchannels do not help here. "Mhmm" during a four-minute build is not attentiveness, it is noise. And a microphone held open across those minutes, because the design assumes a conversation is in progress, creates three failure surfaces push-to-talk did not have.

3a. Session lifetime

The clearest evidence is in Codex's own issue tracker, filed by people who stopped talking because the agent was working.

#35103 (2026-07-24): "when you use the new voice feature on codex desktop, if it times out due to inactivity, the thread disappears." Roughly a two-minute inactivity timeout, not configurable. The reporter explicitly describes longer processing sessions, where the app goes inactive precisely because agents are working.

#36618 (2026-08-02): the display sleeps from inactivity, and on wake "the live Voice control is gone; a small Play button occupies the same control location; and speaking is unavailable." Then the detail that makes this the most instructive report in the set: "the underlying Codex task maintains normal operation throughout."

#35474 (2026-07-26): a websocket reset — "stream disconnected before completion… Connection reset by peer" — after fifteen to twenty seconds, while the text task continues.

The pattern is identical across all three and it is not subtle. The agent survives. The voice session does not. The most durable component in the system — the long task, the thing that actually takes minutes — outlives the channel that was supposed to be how you talk to it. Inactivity is being read as evidence the session is over, in a workflow where inactivity is evidence the session is working.

3b. The hot mic as a system liability

An always-on capture path is not free at the OS level, and its side effects leave your app.

#12111 (2026-02-18): WebRTC automatic gain control ratchets the macOS system microphone level down and never restores it, eventually degrading Teams and Meet until the user finds the workaround flag. A voice feature quietly reconfigured a shared system resource on the way past. That is what a hot mic is — not a UI state, a claim on the machine that persists whether or not anyone is speaking.

There is a second concern I want to state carefully, because I think it is real and I do not think it is proven. A microphone held open across long silences is listening to a room the user has mentally left: during a four-minute build people take a call, talk to someone, think out loud. Push-to-talk made that boundary explicit; always-on trades the boundary away for the ceremony.

Honest caveat. I could not find a report stating that hypothesis in that form. What exists publicly is the symptom class — timeouts, disconnects, session death, unwanted capture — reported by people who fell silent while an agent worked; the closest direct account of self-pickup and feedback loops is about ChatGPT voice generally, not Codex. Treat ambient capture as a design risk that follows from the architecture, not a documented epidemic. It is the thinnest leg here and I would rather say so than dress it up. That applies more weakly to the whole section: these issues are symptom-class evidence for the mismatch, not developers writing "the problem is that voice assumes I'm talking when I'm not." I am connecting reports, and I think the connection is strong — but it is an argument, not a citation.

3c. Attention arbitration

The third surface is the most interesting, because nothing is broken — two correct designs are competing for the same human.

#13590: "Dictation stops (and appears to cancel) when command approval prompt appears during a session."

During a long agent turn the system needs your attention at unpredictable moments: approve this command, resolve this ambiguity, confirm this spend. Those interrupts are correct — you want them — but they arrive on the visual channel while the voice channel believes it is mid-conversation. Two subsystems, both behaving properly, both assuming they own the human's attention. The user is the shared resource, and nothing is arbitrating.

Developers watching from a distance land in the same place. On Hacker News, discussing voice in Claude Code: "voice mode != hands-off mode, so you can't DWC (Driving-While-Coding)" — plus roughly 500ms of activation lag, "so the initial words are missed out." The skepticism is not about whether voice works. It is about whether always-on buys the freedom it implies.

4. What we think the shape should be

We build voice-first workstations — you speak, a realtime agent shapes and ships real work, live, in front of you — so this is not a neutral analysis. It is the position we reached after a lot of session forensics, and it starts from nearly the opposite premise to always-on:

Voice earns its place when the workflow is designed around intent bursts and long agent turns, not around continuous conversation.

Four properties follow.

Sessions that expect silence. If quiet is the normal state, quiet cannot be the timeout condition. A session must be able to sit there holding everything for as long as the work takes, because the human going quiet is the system working correctly.

State that survives interruption. Sleep, reload, disconnect, a mid-session top-up: all ordinary, none of them should cost you your place. Resumability is not an edge case in this medium — it is the medium. Naive optimistic flows fail precisely in the moments an interruptible interface invites.

An agent that reports back when there is something worth saying — and otherwise says nothing. The inverse of filling dead air. Our discipline is momentum narration: when something locks, one short sentence for what locked and one for what is next. Between those moments the agent is quiet, and quiet is not awkward, because the canvas carries the wait; the microphone does not have to.

Costs stated at the point of action. Out loud, in the same breath as the action, with free things saying "free." An interface you operate by speaking must never let you discover a charge afterwards.

In practice that comes out as choreography, and the choreography took longer to get right than anything else. A new project opens on an empty canvas with a single microphone in the middle: tell me what you're building. The first sentence you speak — or type — materialises the stages, lights the first one, and collapses the microphone into the composer as the live brief fills in. That transition is doing real work: it says this is a burst moment, speak densely now, and after it the canvas is the centre of attention while the microphone becomes a tool at the edge of it rather than a presence in the room.

Underneath, voice and typed text are the same agent through one door, driving one tool catalog and one state machine — start talking, finish typing, or type into the middle of a live voice turn. The channel was never the commitment, which is what makes falling silent a choice rather than an exit. Barge-in is full-duplex, built on echo cancellation rather than muting the microphone; on this one point we agree completely with the always-on camp, because half-duplex voice is a walkie-talkie and nobody wants to key a radio to change their mind. Every action is stamped with a state version, so the agent confirms a change only when it matches the current state — after a three-minute wait, a stale acknowledgement is worse than none. And the agent runs the room: it never waits to be asked what's next, and it presents options rather than questions, each with a name and a one-line reason. That is what makes intent-burst interaction viable. You get to be silent because someone competent is holding the thread.

What that buys is a session where you talk in short dense bursts and then watch: a complex game built by conversation in Atelier — Games Lab (public test; 2D and 3D from one spec), a full brand package assembled on a live layered canvas in Atelier — Brands Lab (public test), native Apple Liquid Glass interfaces rendering sub-second on-device with SwiftUI handoff in Atelier — Liquid Glass Lab (on the Mac App Store as Atelier Glass Lab). Different outputs, one interaction shape.

None of this makes voice hands-free. It makes voice a command surface: the fastest way to express intent to a machine that then goes and does something long, while the screen — not the microphone — holds the space in between.

5. Nobody has measured this yet

I want to end fairly, because the easy version of this essay is a dunk and the easy version would be wrong.

OpenAI is not claiming to have solved agentic voice. They said, in their own launch post, that longer-running and more agentic voice is what this research will unlock over time — an accurate roadmap, honestly stated, and GPT-Live appears to be very good at the gap it was built for. What is interesting is that the industry shipped the always-on default before the silence problem was solved, and the issue tracker is where the difference between those two timelines is currently being paid.

This is an open problem, not a solved one somebody got wrong. The mature UX literature on voice agents is about conversational agents, where awkward silence is among the better predictors of abandonment and the fixes are filler phrases and streaming speech. In an agentic workflow that finding inverts: silence predicts work. As far as I can tell, no rigorous published UX study of voice against long-running coding agents exists. That absence is why everyone here is designing on intuition — us included. Ours has a lot of session traces behind it, which is not the same as evidence, and I would rather say that plainly.

If you are building in this space, the question I would put at the top of the design doc is not how do we keep the conversation alive? It is: what should this interface do when the person has correctly stopped talking?

Answer that one first. Everything else is downstream of it.

References

Our own claims here are engineering-competence claims from our shipping history, described at the level of interaction design rather than mechanism. The products are human-led and agent-assisted: the person decides, the agent does the heavy lifting and narrates what it did. Atelier — Liquid Glass Lab is publicly shipped on the Mac App Store; Games Lab and Brands Lab are live in public test. Guarantees about generated output are structural — valid, renderable, playable — not that it is flawless. Our production voice runs on a single provider today and we describe the stack vendor-neutrally on purpose; OpenAI, Codex and GPT-Live are named here because they are the public subject of the analysis.

More from research

Research

Three roads to reasoning in language models

Reasoning reaches a language model by three different routes; the industry converged on two of them, and the third still earns its keep wherever a machine can check the answer.

Engineering

Every guard needs an exit: liveness in agent guardrails

A guard that is perfectly safe can be fatally un-live: ours refused the same call nineteen times in a row, each refusal correct, and manufactured the exact failure it was built to prevent.

Engineering

Why your voice agent talks over itself

Over-talking is not a prompting failure — it is a concurrency problem wearing a UX costume, and no instruction to be more concise can fix a race for a single speaking channel.