Should HarnessRouter support persistent harness runtimes instead of spawning a CLI process per turn?

[yunhao-tech]

Summary

HarnessRouter currently uses the harness CLI as the primary execution interface and generally starts a new process for each turn.

This is a pragmatic way to support multiple harnesses behind one protocol: the CLI is often the lowest common interface, it already encapsulates the complete agent behavior, and a subprocess provides a useful isolation boundary.

However, using a one-process-per-turn execution model introduces several limitations:

• process startup overhead on every turn;
• session continuity depends on each vendor CLI's resume implementation;
• interactive capabilities are reduced to what a one-shot CLI command can expose;
• the UHP session lifecycle is not backed by a persistent harness runtime;
• harness-specific features such as mid-turn input, interactive tool approval, pause/resume, and runtime-local state become difficult to preserve.

Would it make sense to support persistent, capability-aware harness adapters while retaining CLI execution as a fallback?

Why CLI-first makes sense

There are good reasons for the current design.

1. The CLI is the lowest common interface

Not every supported harness exposes a stable SDK or service protocol. A process with stdin/stdout is therefore the broadest integration surface and makes it possible to support more harnesses without requiring a vendor-specific in-process integration for each one.

2. The CLI already contains the full harness behavior

Using a model SDK directly would require HarnessRouter to reimplement substantial parts of each harness, including:

• the agent loop;
• tool execution;
• MCP integration;
• project instruction and skill discovery;
• context management and compaction;
• provider configuration;
• permission handling;
• persistence and recovery.

The CLI allows HarnessRouter to reuse the actual harness rather than reproduce it.

3. Subprocesses provide isolation

A subprocess boundary makes cancellation, timeout handling, environment isolation, working-directory control, and process-group termination relatively straightforward.

For these reasons, CLI support should probably remain an important compatibility path.

The problem with one process per turn

The concern is not primarily that a CLI is used. The concern is that the CLI invocation also defines the runtime lifecycle.

Today, a logical HarnessRouter session does not necessarily correspond to a live harness session. Each turn starts a new process and attempts to reconstruct continuity through vendor-specific state.

For example, continuity may depend on mechanisms such as:

• Claude Code --resume;
• Codex resume --last;
• Pi session identifiers;
• Hermes state stored in SQLite;
• DeepSeek resume-or-create behavior.

These mechanisms are not equivalent. Some resume a transcript, some recover vendor state, and some only approximate continuity. As a result, the UHP session abstraction can be stronger than the execution guarantees provided by the underlying adapter.

This creates several concrete problems.

Startup overhead

Every turn pays for process initialization, configuration discovery, authentication setup, MCP initialization, workspace scanning, and potentially other harness-specific startup work.

For a multi-turn interactive session, this work is repeated even though the logical session remains active.

Vendor-dependent session semantics

Session continuity is only as reliable as the CLI's resume functionality. A change in vendor behavior, session file format, or CLI flags can affect HarnessRouter's session guarantees.

A workspace checkpoint plus a vendor session ID is useful for recovery, but it is not equivalent to a persistent runtime.

Limited interactivity

A one-shot command model makes it difficult to support capabilities such as:

• appending user input while a turn is running;
• interactive tool approval;
• pausing and resuming a turn;
• changing execution parameters during a turn;
• maintaining runtime-local caches and context;
• coordinating multiple active agents.

These are runtime operations rather than ordinary request/response operations.

Lowest-common-denominator behavior

If all adapters are forced into the same one-shot execution model, harnesses with richer interfaces cannot expose their full capabilities.

For example, a persistent Codex app-server integration should not need to behave as if it were another codex exec invocation.

Proposed direction

Could HarnessRouter separate the protocol-level session lifecycle from the process lifecycle and introduce capability-aware harness adapters?

A possible interface could look like this:

create_session
send_turn
append_input
approve_tool
cancel_turn
checkpoint
restore
close_session
capabilities

Each adapter would declare the operations it actually supports:

{
  "persistent_runtime": true,
  "resume": true,
  "append_input": true,
  "interactive_approval": false,
  "checkpoint": true
}

This would allow HarnessRouter to use the strongest available integration for each harness without pretending that all harnesses have identical capabilities.

Possible adapter strategies could include:

• Codex: a persistent app-server adapter;
• Claude Code: an Agent SDK or another managed persistent-runtime adapter;
• DeepSeek Harness: an SDK-backed persistent JSON-RPC runtime;
• other harnesses: a CLI adapter when no stable persistent interface exists.
  The CLI adapter would remain valuable as a compatibility fallback rather than being the lifecycle model imposed on every backend.

Runtime lifecycle

Persistent runtimes would not need to stay alive indefinitely.

HarnessRouter could manage them with an idle policy such as:

1. create or restore a runtime when a session becomes active;
2. keep the runtime alive across turns;
3. checkpoint it after an idle timeout;
4. terminate the process;
5. restore it from the checkpoint when the next turn arrives.

This preserves most of the isolation and resource-control benefits of subprocess execution while avoiding startup on every turn.

Session ownership

A useful architectural principle may be:

HarnessRouter owns the logical session lifecycle; vendor resume mechanisms are recovery tools, not the definition of the session.

Under this model:

• UHP defines the durable session and response objects;
• the adapter manages the live harness runtime;
• workspace checkpoints provide portable recovery state;
• vendor session IDs are adapter-specific metadata;
• CLI resume is used when necessary but does not determine the protocol semantics.

Questions

1. Was the one-process-per-turn model chosen mainly for implementation simplicity and backend compatibility, or is it an intentional long-term isolation boundary?
2. Are persistent harness runtimes already part of the roadmap?
3. Would a capability negotiation model fit the current UHP abstraction?
4. Could the existing Codex app-server support become the first persistent adapter rather than being started again for every turn?
5. Would the maintainers accept an adapter lifecycle that supports both persistent runtimes and one-shot CLI fallbacks?

I think the current CLI-first approach is a reasonable bootstrap strategy. The main concern is making it the permanent execution model for every harness, including those that expose richer and more persistent interfaces.

[richard-epsilla]

Thanks for writing this up so carefully. It argues the other side better than we had argued it internally, so let me answer directly.

Was one process per turn deliberate? It started as compatibility and grew into a boundary we lean on. Two things rest on it now: the session write wall in 0.8.2, where each session's agent runs under its own uid, so isolation is the kernel refusing rather than us being careful; and cancellation, which is just killing a process group and asks nothing of the harness. None of that means the boundary cannot move. It means whatever replaces it has to carry the same weight.

Is it on the roadmap? Not written down as a commitment, but the door is open and this issue has moved it up rather than settled it. The seam already exists: the codex app server is in the tree behind a flag, started fresh every turn, which is the wasteful version of your point.

Would capability negotiation fit UHP? Yes, and I would do this first whatever we decide about persistence. We already publish discovery at /v1/uhp. Declaring what each backend can actually do is a far smaller change than any runtime work, and it earns its keep even if no persistent adapter ever ships, because it stops the protocol claiming a uniformity it does not have. Two asks: capabilities as data rather than branches in code, since a declared field that still needs a matching if backend == "codex" has bought nothing; and honest refusals, so "we have not built it" and "this backend never offered it" fail differently.

First persistent adapter? The codex app server, for the reasons you give. Would we take a lifecycle supporting both? Yes. What I would want: defaults unchanged for anyone who did not ask, the workspace checkpoint stays authoritative, cancellation does not get weaker, and declared capabilities reflect what has actually been run rather than what a vendor's docs promise.

On latency. I measured before replying and got my own probe wrong first. I named the harness in a body field that does not exist, so nothing bound and every backend quietly ran the default. Five tidy sets of numbers, all secretly the same model, every turn returning completed. Bound properly, trivial turns, first then follow up in the same session:

pi      (gpt-5.4-mini)      3796 ms / 4888 ms
dsh     (deepseek-v4-flash) 5186 ms / 5017 ms
claude  (claude-sonnet-5)   6281 ms / 6121 ms
codex   (gpt-5.4)           7642 ms / 6646 ms
hermes  (qwen3.7-max)      23285 ms / 24869 ms

Your premise holds, the second turn is never cheaper. But the spread follows the model rather than the process, and these include inference so they do not isolate startup. I would not lead with latency: it is a small term inside a number dominated by something else, and going persistent does not touch the dominant term. Your other arguments are stronger without it.

Where you changed my mind. My first reaction was that interactive approval is a permission gate, and we replaced those with sandboxing on purpose. Too narrow. Mid turn input is not mainly about approving actions, it is about alignment: asking which of two readings you meant before spending twenty minutes producing the wrong thing. That is correctness, not security, and sandboxing does nothing for it. We watched it happen last week, a long task going wrong early with no way to reach in and say so. In a strict request and response turn an unsure agent can only guess or give up, and both are worse than asking. I had not been pricing that.

Where I still hesitate. A question needs somewhere to go, but I am not convinced it needs the process alive while it waits. A turn could end in a waiting state, workspace checkpointed and question recorded, and the answer opens the next turn with context restored. That survives a restart or a closed laptop, which a suspended process does not, and it reaches backends that will never expose a persistent interface. One detail from having built this elsewhere: make the terminal state mandatory, so a run ending without one counts as a failure. Otherwise blocked and crashed look identical, and "waiting for input" becomes a synonym for lost.

I offer that as a complement rather than a substitute. Ideally the waiting state is the floor everyone gets, and persistent adapters are the better experience where a backend can support it.

Two warnings from that same build, since they cost us. Cancellation quietly stopped working once turns ran inside a long lived runtime, because nothing held a handle the control plane could reach, while the button stayed in the UI. And a persistent environment is not a persistent process: a durable workspace plus a resumable session id gives you most of the continuity you are asking for while still running one process at a time.

So: capability declaration first, then a waiting state, then one optional persistent adapter. That order is about each step being useful on its own, not about keeping the rest at arm's length. The fuller model you describe is where I expect this to land.

Happy to review a PR on the capability descriptor and the discovery document, or to talk the adapter interface through here first if that is easier.

[Foxfire1st]

I am in a state where I would absolutely love to have backend adapter that would give me one unified interface accross the common harnesses. My issue is that I need orchestration that I can bend around my workflow. My MCP is a control plane having a dual revision epistimic ledger. Meaning my code and my memory live in seperate repos that need syncing. Without going to deep into specifics what I need is a multi agent orchestration that I can use as a dependency.

At first I tried to preserve the original TUI/CLI because I really really badly wanted to avoid the pain. So I had eventually my browser dashboard via websocket connected to the CLIs. So I could basically plugin whatever I had installed. The subscriptions were no issues. But that's where the problems started. I needed orchestration, therefore agents accross Harnesses need to talk to each other. The only way I found to make that work is via "paste" which is disgusting. Making me trip up with my own inputs. It got even worse with inter harness states. There is no way to infer wether an agent is working, turn-ended, waiting for a answer to a quiz. I had implemented a mail inbox system which one way or the other you need anyway. But it is 100% reliant on agents actually remembering to write a message to the manager or orchestrator. Another issue is capabilities. You guys had a good idea mentioned that I did not arrive on myself to write a minimal adapter just to fish for the available models, efforts etc. But even if I had it, the issues with orchestrations falling asleep forced my hand.

So I implemented individual harness adapters for Claude Code, Codex, and PI. I haven't even gotten around to make one for Hermes. I also build the chat frontend in React since my dashboard runs in the browser. So far I can now configure in a setting files roles that encode for each agent role the harness, model, and effort level and there is a freeform field so I can activate ultracode in Claude Code.
With that my Orchestrator can spin up other agents with the configured settings. And I can also start chats myself via the dashboard.
The system when I click on a selector does that quick request to pull the available models, and effort levels.

So I can sit in chat and converse with the agent, it also lets me interrupt to send a new message, and queue messages. Buggie but it kinda works. When the agent pulls up a quiz I can click on that. For codex and cc I also have subagents which you can inspect. What I don't have is slash commands. I can't compact or anything. That's not implemented. PI I implemented but that doesn't work really without additional attention. Right now I can't put more attention towards orchestration and chat interface as it distracts me from my actual project goals. But I need those. And so far I haven't found a system that I can use as dependency that gives me that amount of control I look for. Many other sollutions like Herdr basically do exactly what I tried earlier running the TUI.

You also cannot run the same session through 2 different channels where you run the TUI for the interface, and then have another connection to pull out agent state for external communication to other harnesses. At least Codex and Claude doesn't allow that.

ACP is getting better by the day but when I started my implementation it didn't have what I was looking for. However thinking again maybe I could make it work with a slim adapter for capability inference, while doing everything else via ACP.

But yeah that's my 2cents.

[richard-epsilla]

@Foxfire1st thank you for the account. It is the most useful kind: someone who tried to keep the TUIs, was forced off them, and built the thing anyway. I want to say back what I learned from it, then be precise about what exists here today and what does not, because the gap between those is our roadmap.

What I take from your build

1. Agent state has to be a fact the runtime reports, not a courtesy the agent performs. Your mail inbox is the tell: it works exactly as often as the agent remembers to write, and an orchestrator cannot schedule on that. "Working, turn ended, waiting for an answer to a quiz" are runtime states. If the runtime does not expose the third one, every orchestrator ends up polling a transcript and guessing.

2. "Paste" is what happens when the unit of composition is the terminal instead of the session. The moment the session is the unit (a durable workspace plus a session id, addressable by an API), agents talking to each other is an orchestrator sending turns, not a human ferrying text. You arrived at that by building adapters; it is the same conclusion this project started from.

3. Roles are a configuration object: harness, model, effort, a freeform flag or two, chosen at spawn. And a selector in a dashboard is only honest if it pulls what the backend actually offers rather than what its docs say. Capability discovery is not a nicety, it is what makes the role object true.

What you can lean on today, with the limits stated

• Sessions and turns are the unit. GET /v1/sessions/{id}/turns returns each turn with its status, and a turn always ends in a terminal state (done, failed, cancelled, or incomplete). When the record knows why a turn stopped early it says so (step limit, timeout, interrupted); when it does not know, it claims nothing. That is deliberate: blocked and crashed must not look the same, and "we are not sure" must not be dressed up as either.
• Models per harness come from GET /v1/harnesses/{id}/models, and /v1/uhp is the discovery document. That is the "quick request" your selector makes, as protocol.
• A role is a harness object here: POST /v1/harnesses with a base harness, a default model, instructions, skills, and MCP servers. Your orchestrator can mint those and address them by id.
• The reference implementation streams every session's events for a harness on one server-sent topic (/v1/harnesses/{id}/events). That is how the console shows "working" without the agent writing anything to anyone. It is implementation, not yet protocol, which is one of the things this thread should change.
• One Responses-compatible endpoint. Your orchestrator is a client of it, not a host of TUIs, and the open source image is the dependency you run.

And the honest gaps, each of which your account makes concrete rather than theoretical:

• No waiting state yet. A quiz today is a turn that either ends or stalls; there is no state that says "this turn is paused on a question, here it is" that a machine can route. That is the primitive your inbox was standing in for.
• No appended or queued input on a running turn. You built interrupt and queue; the protocol has neither.
• Effort is not a declared capability. The reference implementation currently strips reasoning fields it cannot honor everywhere rather than passing them through unevenly. A declared per-model field beats a silently ignored one, and your selector needs it as data.
• Slash commands and compaction stay vendor bound. I would rather expose compaction as a capability where a backend has it than fake it where it does not.

Where this puts the roadmap

The order from earlier in this thread stands, and your experience sharpens each step:

1. Capabilities as data first, per backend and per model, including the ones you pull today: models, efforts, whether interrupt and queued input work, whether a question can be surfaced, whether subagents are inspectable. Honest refusals with it: "this backend never offered it" and "we have not built it" fail differently.
2. The waiting state next: a turn ends in waiting with the question recorded and the workspace checkpointed, and the answer opens the next turn with context restored. It survives a restart and a closed laptop, which is also the answer to orchestrations falling asleep, and it reaches every backend, including ones that will never expose a persistent interface. Your quiz clicks become an API call an orchestrator can make.
3. One optional persistent adapter after that, for the backends that can carry richer semantics. ACP is a credible wire for that adapter where a backend speaks it (there is a Devin integration proposed in this repository over ACP), and your instinct of a slim adapter for capability inference with everything else over ACP is compatible with this split rather than opposed to it. What I would not do is make ACP the floor; the waiting state has to work for backends that will never speak it.

Two things I would like from you, if you have the time

1. The capability fields your selector and orchestrator actually consume, per harness, as a plain list. That list is the descriptor we would standardize, and I would rather start from what a working system needed than from what a vendor advertises.
2. Which harness's interrupt and quiz behavior held up best under orchestration, and where each one lied to you: turn ended when it was really waiting, or the reverse. The waiting state's design depends on knowing those failure modes, and you have seen them.

A PR against the capability descriptor and the discovery document is welcome, from you or from us with your list in hand. Either way this thread is now the specification's history for those three steps, so thank you for adding to it.

[kuanzema]

Great discussion! Based on the discussion here, I think we have a pretty clear next-step sequence for HarnessRouter/UHP — open to feedback:

1. Capabilities as data first.
   We’ll define a capability descriptor at the backend/model level and expose it through discovery, including things like supported models, effort levels, interrupt/queued input, waiting-for-input support, subagent visibility, compaction, and persistent-runtime support where applicable.

   The goal is for clients and orchestrators to discover what a harness can actually do, rather than relying on backend-specific branches or vendor documentation.

2. Add a first-class waiting state.
   A turn should be able to explicitly say that it is waiting for user/orchestrator input, record the question/state, checkpoint the workspace, and resume cleanly when an answer arrives.

   This gives every backend a reliable human-in-the-loop primitive without requiring the underlying process to remain alive.

3. Then experiment with an optional persistent adapter.
   For backends that expose richer runtime interfaces, we can support persistent adapters without making persistence a requirement for UHP. Codex app-server looks like a natural first candidate.

CLI execution will remain an important compatibility path. The broader direction is to separate the UHP session/runtime semantics from the specific execution mechanism underneath them.

For the immediate next step, we’ll focus on the capability descriptor and /v1/uhp discovery surface.

If anyone here has already built orchestration on top of Codex, Claude Code, Pi, or other harnesses, especially @Foxfire1st, a concrete list of the capability fields your system actually consumes would be very useful as input to that schema.