The harness around the model

A model can propose an action. An agent harness decides what that proposal means, whether it may run, what state to save, and when to ask the model again.

Suppose a developer asks: “Fix the parser bug and run its tests.” The model might request a file read. Your program executes the read and sends the result back. The model then requests an edit and a test command. Those requests are data until the harness dispatches them to real code.

The small loop is easy to recognize. Most of the engineering lives around it.

One task, several model requests

Scroll sideways to explore the diagram →

This is a teaching abstraction, not a claim that every implementation uses this exact control structure. Codex organizes work around session tasks and sampling steps; pi exposes a reusable loop with lifecycle hooks; Goose composes ordered state-machine operations; Sidekick coordinates a native loop or external runtime through events, durable turns, and workflow workers. Their chapters show the actual implementations.

Use these terms throughout the guide:

“Turn” is deliberately not a universal unit here. Each project uses it differently. Compare lifecycle boundaries before comparing metrics such as turns per task.

Walk through a repair

This trace is illustrative. It was not executed as a benchmark.

The harness does not know that the fix is correct just because the model says it is finished. The test result is evidence; the model's conclusion is an interpretation of that evidence. Evaluation of task quality is separate from the runtime's ability to execute a valid loop.

Five boundaries worth learning

Proposed action to executed action

A schema describes what arguments are valid. It does not grant permission. An approval decision authorizes an action under a policy. A sandbox limits what the process can actually access. These are three separate mechanisms.

For example, a shell command may be valid JSON, approved by a user, and still fail because the operating system denies access to a path. Conversely, a custom extension can bypass an application-level check if it holds direct host access. Always locate the enforcement point.

Live events to saved state

A UI can display a tool result before the persistence layer has committed it. A process crash in that gap may leave the UI and resumed conversation disagreeing. Saving a conversation also does not prove that an interrupted external action is safe to repeat.

For an email send, the process might crash after delivery but before recording success. Replaying the tool may send a second email. Idempotency means that repeating an operation with the same identity does not repeat its effect; it needs support at the effect boundary, not just a transcript file.

Full history to model context

A session can contain more information than fits in the next request. Compaction replaces some history with a shorter representation; truncation simply removes or bounds content. Both influence what the model knows. They must preserve enough tool-call/result structure for the provider to accept the request.

The saved original history, the current model context, and the user-visible conversation can therefore be different views. Ask which one a resume, fork, or compaction operation updates.

Concurrent work to ordered history

Two reads can finish in the opposite order from their calls. A UI may want to display each result immediately, while the transcript may need a deterministic order. Two edits to the same file introduce another problem: they may invalidate one another's assumptions.

Concurrency is a scheduling decision. It requires reasoning about effects, not just wrapping tool calls in Promise.all or its equivalent.

One run to many participants

Steering adds new input while work is active. Follow-up queues defer input until a later boundary. Cancellation requests that active work stop. Multi-agent delegation creates additional agent runs. Multi-player collaboration lets multiple humans or clients participate in shared work. These solve different problems.

A server accepting two connections is a useful primitive for multi-player support. It does not establish user identity, ownership of approvals, conflict resolution, or a shared editing experience. The advanced chapter evaluates those separately.

Read the implementations in a useful order

Start with pi to study a clearly exposed loop and its hooks. Then read Goose to see ordered operations and effects express control flow. Read Codex for task lifecycles, tool scheduling, policy enforcement, and client coordination. Read Sidekick to see event-driven orchestration across a shared service, workers, and external runtimes. This is an educational reading order, not a product ranking.

Then use the comparison and advanced topics to follow one concern through all four. The builder's lab turns the comparison into design exercises.

Four ways to organize the same feedback loop

All four harnesses turn model output into tool execution and feed the results back. The useful comparison is where each places decisions about state, authority, concurrency, and continuation. Sidekick also wraps external harnesses; its native execution path is the comparison target here.

The short descriptions below are an interpretation of the inspected architecture, not a claim about which product is best. See the methodology for pinned revisions and scope. The website's comparison explorer exposes all 15 dimensions with per-cell qualifications and direct source links.

The architectural center

Sources: Codex turn preparation, pi loop, Goose state machine, Sidekick native loop and attempt ownership. Sidekick's repository is private; its pinned source links require repository access.

These are different decompositions, not mutually exclusive strategies. Pi's coding application still has persistence. Codex still has a repeated sampling loop. Goose still needs a provider adapter and executable tools. Sidekick still iterates within its conversation workflow.

Compare behavior, not feature counts

Each row compresses important details. Follow the evidence in Codex, pi, Goose, and Sidekick, or expand the corresponding website cells. A “core” or “optional” label is a scope marker, not a quality score.

Follow one problem across the designs

A tool takes longer than its neighbor

The model requests reads A and B; B finishes first. Codex and pi can report progress while retaining call order in the final transcript. Goose records observed results with IDs. Sidekick submits separate tool workflows, collects results by correlation ID, and returns the batch in call order. These approaches can represent the exchange correctly. The tradeoff is between a predictable source-order transcript and immediate append of the observed outcome; client code must understand which contract it consumes. Codex ordered drain, pi scheduling, Goose execution, Sidekick collection.

Now replace read B with “test the file after edit A.” These are dependent actions. None of these scheduling descriptions proves the harness infers that dependency from natural-language tool arguments. Put dependent work in separate model steps or explicitly sequence it in a supported orchestration program.

The user changes their mind during execution

“Steer” can mean “queue this for the next safe boundary,” not “stop the current tool.” Pi explicitly finishes the current batch before consuming steering. Codex has active-turn admission and feature-dependent preemption paths. Goose consumes queued input through a state-machine operation after the relevant model/tool phase. A client should distinguish queued, admitted, executed, and cancelled states. Pi loop, Codex turn processing, Goose steering.

Sidekick makes the queue itself durable. Its main bound-agent loop places steers before rebuilding context, commits them, then broadcasts their placement. Follow-up admission occurs after an answered settlement. Those are separate boundaries from cancellation and from child or panel execution. Sidekick iteration boundary, next-turn admission.

A conversation is too large

Codex, pi, and Goose can continue from summary-based representations, but they store and adopt them differently. Codex has multiple compaction/rollover paths. Pi projects an append-only session tree using compaction checkpoints. Goose uses conversation visibility plus inserted summary records. Sidekick's inspected native pipeline instead establishes optional trimming of older, large tool outputs; this is not evidence of equivalent full-history summary compaction. Sidekick context factory. “Compaction supported” therefore says little about retained raw history, replay, adoption races, or context reconstruction. The advanced chapter compares those contracts.

A worker disappears while a tool is running

Sidekick separates the user's turn from the worker attempt processing it. A replacement attempt can take ownership, then inspect recorded tool intents before another model request. A settled intent supplies its saved result. An uncertain running call may rerun only when stored and current replay policy allow it; an unsafe call receives an interrupted-result explanation instead. Ownership prevents a superseded worker from settling the turn, but cannot undo an external effect. Attempt ownership, intent recovery.

This illustrates event-driven orchestration: admission, completion, and recovery change durable state and determine which work may execute next. Panel completion provides another example: the last completed or failed grant in a batch makes a moderator wake eligible. The native model/tool continuation remains an ordinary loop. The session reducer is not a complete event-sourced history, and this comparison does not establish Sidekick as the only event-driven harness. Panel completion, native iteration, dispatch and storage boundaries.

The event-driven execution topic follows Sidekick's orchestration boundaries and compares their costs with the other designs.

What to borrow first

For a small custom agent, study pi's explicit loop and hooks before adopting its surrounding application. For a runtime reconstructed from stored state, study Goose's operation/effect boundary. For a long-lived coding system with changing tools and multiple clients, study Codex's step snapshots, execution scheduling, and persistence barriers. For a shared service where work can outlive a worker, study Sidekick's turn/attempt distinction and tool-intent recovery.

These are educational suggestions grounded in the architectures, not deployment recommendations or measured rankings. Copy the invariant you need before copying the surrounding machinery. A script with one trusted tool may need only a loop and cancellation. A shared service with remote effects needs identities, policies, durable state, and recovery semantics even if its model loop stays small.

Event-driven execution

An agent can spend seconds generating a response, minutes waiting for a tool, and hours waiting for a human. If its work must survive a worker disappearing, a browser disconnecting, or several participants acting at once, execution needs an identity and lifecycle beyond one function call.

Event-driven execution means that incoming facts or commands cause state transitions that make further work eligible. Sidekick is the main example here: its orchestration uses durable turns, worker attempts, message reducers, and tool workflows. Its native worker still runs an iterative model/tool loop. The useful distinction is where continuation and ownership are decided, not whether the implementation contains a loop. Iteration boundary, attempt ownership.

Events, commands, and effects

These concepts describe different responsibilities, even when a protocol transports them in similar envelopes:

Sidekick's session reducer accepts messages, commands, and signals and returns new state plus effects. A separate executor handles persistence and broadcasting. That makes the decision testable apart from database and network operations, but it does not make every effect durable or atomic. Reducer, effect executor.

Event sourcing is a stronger storage choice: an authoritative sequence of events reconstructs state. Merely publishing events does not establish it. Sidekick also relies on turn, attempt, and tool-intent rows; its session stream is not a complete authoritative execution history. Likewise, token streaming is observability unless the events also drive execution decisions. Dispatch, settlement.

Two levels of continuation in Sidekick

Scroll sideways to explore the diagram →

At the inner level, the native loop rebuilds context, streams a model response, dispatches tools, records their results, and repeats. At the outer level, admission and ownership determine which worker may advance the turn; waiting, recovery, and settlement determine what happens after an attempt ends. The reducer translates messages into session effects. These mechanisms cooperate rather than replace one another. Native iteration, tool dispatch, attempt ownership.

The steering and cancellation topic examines when new user input becomes eligible for the next request. Observability covers what clients can infer from lifecycle events.

Scope matters: Sidekick can also wrap an external harness through an Outpost. That path delegates model execution and has different retry behavior. The examples below concern the native cluster path. Sidekick's repository is private; its pinned citations require access. The Sidekick chapter provides the full reviewed scope. Execution routing.

Trace a wait and a worker failure

Consider a request to investigate an issue, ask the user which remedy to apply, and then perform the selected action. This is an illustrative trace, not a recorded benchmark.

1. Admit and claim. The request receives a pending turn. A worker claims an attempt on it. Conversation admission and current-attempt checks coordinate competing producers. Admission, claiming.
2. Reach a human wait. A sole native AskUserQuestion call suspends the turn and ends the worker run. The turn remains awaiting input rather than completed. The answer returns the same turn to pending for a later attempt; one process need not wait for the human's entire absence. Question suspension, resume admission.
3. Record tool intent. After resumption, a model requests an action. The loop commits the assistant request and tool intent before dispatch. The tool executor must claim that intent before acting. Intent commit, execution gate.
4. Lose the worker. Suppose an action starts but its result never reaches the transcript. A replacement attempt may take ownership of the turn. Ownership does not prove the action failed or stop a remote effect already underway. Attempt ownership.
5. Recover from evidence. A stored outcome is redelivered. An unclaimed intent can execute. A running call waits for an existing result, then reruns only if both stored and current replay policy permit it; otherwise recovery reports an interrupted, uncertain outcome. The next model request receives that result. Recovery, policy intersection.

This separates retrying the worker from repeating the action. A replay-safe operation may execute more than once. An uncertain mutation needs reconciliation, an externally enforced idempotency key, or an explicit unknown result—not an assumption that silence means nothing happened. See persistence and recovery.

Events can also create new work without a crash: Sidekick panel completion updates durable grants, and the last completed or failed statement can make a moderator wake eligible. The important feature is that completion changes scheduling eligibility, not simply that a notification was emitted. Panel completion.

Compare the four decompositions

These designs overlap. The table compares their inspected architectural centers, not exclusive categories or measured rankings.

The tradeoff for a builder follows from those boundaries. Pi's reusable loop gives a compact place to understand model/tool ordering, while the application must choose its durable recovery contract. Codex's captured sampling context makes a step's settings explicit, while tracing a complete request requires following its session and execution layers. Goose makes the next action depend on loaded state, while adding an operation requires understanding its position in the ordered scan. Sidekick makes cross-worker ownership explicit, while adding storage, dispatch, and recovery protocols around the loop. These are design implications of the cited structures, not measured complexity or performance scores.

The useful question is not “Which one has events?” Ask which facts determine the next action, where those facts live, and what a replacement process can safely infer.

Costs that move into the harness

The following are design implications, not measured performance results.

• Ordering: independent producers can finish out of order. Correlation IDs associate results with requests; sequence allocation orders presentation. Neither establishes that dependent effects ran in the required order. Sidekick returns a tool batch in original call order after collecting correlated results. Tool collection.
• Duplicate delivery: a workflow retry can encounter work already claimed or completed. Stable identities and conditional claims must distinguish redelivery from a new request. A boolean “busy” flag is insufficient when its owner can die. Attempt claiming, intent gate.
• Fencing: the current attempt identifies the producer permitted to change durable state. Rejecting an obsolete worker's settlement does not revoke authority already exercised against an outside service. Settlement.
• Consistency: database commits and broadcasts have different boundaries. Sidekick's ordinary dispatch can broadcast after a flush but before the caller commits; deferred acknowledgements and settled-turn projections use stronger boundaries. Clients need a recovery path to authoritative state. Dispatch, completion projection.
• Latency: workflow dispatch, coordination, and additional commits introduce waits. They can enable overlap and release a run during human input, but do not inherently make a single model/tool exchange faster.
• Operations: workers, database state, result channels, timeouts, and recovery policies create more failure modes to monitor. Post-commit hooks can still fail after settlement; the inspected sweeper explicitly logs lost completion work. Post-commit limits.

When this model earns its complexity

Choose durable orchestration when accepted work must survive process loss, waits outlive individual runs, several workers share responsibility, or completion must unlock another participant's work. Make admission, ownership, and retry policy explicit before distributing execution.

For a small agent owned by one process, start with an understandable loop, tool-call identities, and cancellation. A pure reducer can improve testability without adding a broker. Add durable attempts when you can name a failure they must survive; add cross-worker scheduling when it solves an actual execution or coordination need.

Builder exercise: expose the crash windows

Build a tiny simulated runner with a scripted model, a read tool, and a mutation tool that increments an external counter. Give the turn, attempt, and tool intent distinct IDs. No paid model or real external service is needed.

Inject failure before tool claim, after the counter changes, after its result is stored, and before completion is broadcast. Redeliver the same work twice. Then let an obsolete attempt finish after a replacement has claimed ownership.

Assert separate properties: one current owner, no stale settlement, stored results redelivered without another effect, uncertain mutations surfaced honestly, and clients able to recover completion from durable state. Also suspend for a human answer and resume the same turn under a new attempt. Sidekick's inspected crash-window tests provide a concrete reference; they were reviewed as source, not executed for this guide. Recovery tests.

Tools: catalogs, eligibility, and discovery

An agent may have a hundred tools installed while the next model request advertises only six. That can be deliberate: some tools are unavailable to this agent, some require discovery, and others are callable only through an orchestration interface. “Registered,” “visible,” and “authorized” describe different states.

Suppose a user asks to find an invoice and email a summary. The model needs to discover the invoice reader and understand its arguments. Finding a mail tool does not authorize sending that email. A good harness reduces irrelevant choices while preserving a separate execution-time authority check.

Separate the catalog from permission

Think of tool preparation as several decisions, without assuming every harness implements them as separate modules:

Hiding a schema is useful prompt construction, but is not sufficient access control. A guessed name or nested invocation must still reach the appropriate execution checks. Conversely, deferring a tool does not mean denying it: the model may be expected to discover it. See tool safety classification for the separate authorization question.

How the four catalogs are built

These are inspected implementation paths, not guarantees about every provider or deployment. Sidekick's source is private and its links require access. Its native path is distinct from an external harness hosted through an Outpost.

Capability negotiation: what can this run actually offer?

Capability negotiation means resolving which features are usable for the selected model, provider, configuration, and execution context. It need not involve a network handshake: the inspected paths often use model metadata, compatibility flags, and local policy. Technical support and permission are separate questions. A provider may accept a tool schema while the agent lacks authority to execute its operation.

For example, imagine switching models after discovering an invoice tool. The new model might support ordinary function calls but not the old model's tool-search protocol. The harness must choose a supported representation for the next request, preserve call/result identity, and retain execution-time permission checks. Silently dropping a required constraint would change the contract.

Treat these as examples of resolution at different layers, not one standardized negotiation mechanism. For provider serialization, streaming, and history conversion, continue to provider adaptation.

For a harness you build, resolve presentation before constructing the request and check authority again before execution. Record why a capability was selected, adapted, or refused. Test a model switch, a revoked grant, a missing executor, and a required schema feature unsupported by the provider. A fallback is useful only when it preserves the contract your caller requires; otherwise return an explicit incompatibility. Changes to the advertised surface also belong in your prompt-cache strategy.

Discovery can change the next request

Pi provides a clear trace. Initially, a deferred invoice tool is registered but absent from the active model tool list. The model calls tool_search with an invoice-related query. The search ranks eligible inactive candidates using BM25, a lexical matching method, appends matches to the active list, and returns a short description. The next model request then declares the tools. Activation is recorded as a tool change. The search tool's own description remains stable as registrations change. Pi search and activation

That is dynamic hydration: loading the detailed callable surface when needed. It saves upfront schema space at the cost of a discovery step and the possibility of a poor match. A successful metadata search says a tool looks relevant; it does not prove that the tool will return useful results.

Codex also separates deferred search from direct exposure. Its search path produces loadable specifications, and its code-mode path applies the exact step's nested-tool policy before limiting ranked matches. Search availability itself depends on model support and tool policy. The runtime therefore cannot assume that every provider presents every discovery mechanism identically. Codex search eligibility, ranked results

Goose's extension-management surface is a different level of discovery: it identifies extensions available to enable, then manages their activation. An extension may contribute several tools. Treating this as identical to searching an already registered individual-tool catalog would conceal the connection and lifecycle work involved. Goose extension management

Goose's optional code mode separately offers list_functions and get_function_details for the callable catalog in its lease snapshot. Its catalog excludes app-only tools and other unsupported entries; execution checks names against that catalog. Listing details for a nested call is different from promoting a function into the model's direct tool list. Code-mode discovery

Stable wrappers are another design

Sidekick's attached skills illustrate a different tradeoff. When an agent has resolvable skills, context construction includes Skill, ReadSkillReference, and UseSkillTool. Loading a skill appends instruction material as a tool result rather than adding new function definitions to the model's tools array. This preserves a stable calling surface for that mechanism. Stable skill surface

UseSkillTool resolves the attached binding, checks that the requested tool is declared by the skill, and dispatches through the normal system-tool path. At this pin it supports built-in system tools, not arbitrary configured CLI or user Python tools. It does not grant authority merely because instructions mention an operation. Skill dispatcher

This is not a claim that Sidekick's entire catalog is immutable. Agent configuration, role, delegation depth, and other context determine which definitions are assembled. For governed tools, granted actions shape descriptions; a tool with no allowed actions can be omitted. Panel peers and hidden deliberation passes lose interaction/delegation tools inappropriate to their execution role. Action shaping, role restrictions

Refresh catalogs without changing a pending call

Suppose lookup is discovered, then its server refreshes before execution. Does the admitted call still address the implementation and schema the model saw? Catalog refresh is a correctness boundary as well as a cache concern.

Codex captures MCP binding and output-policy information for the admitted call. Goose caches public extension tools by version and scope, while normalizing advertised names and metadata. Pi's discovery distinguishes registered tools from the active set. These mechanisms address different lifetimes; none should be reduced to “cache tools forever.” Codex call snapshot, Goose scoped catalog, pi activation

Stable ordering also matters independently of eligibility. Goose explicitly sorts the final inference list to support prompt caching, then adapts tool presentation for its provider configuration. A stable order does not mean a stable membership set, and provider adaptation does not substitute for permission enforcement. Goose inference preparation

Builder exercise: explain every missing tool

Create a registry with a reader, a sender, and a tool available only to a specialist agent. Start with only the reader visible. Make discovery return the sender's contract without granting send permission. Then change the specialist's role and refresh the catalog while an earlier call is pending.

For every tool, expose a diagnostic reason: absent implementation, excluded by policy, deferred, incompatible presentation, or awaiting runtime authorization. Check that direct calls and code-mode calls obey the same intended authority boundary. Record which catalog revision or binding an admitted call uses.

Keep these responsibilities independent. Registration supplies execution; discovery supplies information; activation supplies model visibility; authorization supplies permission for the operation. This separation makes a missing tool understandable and a newly discovered tool safe to evaluate.

Provider adaptation and protocol correctness

A provider adapter translates between the harness's conversation model and a model service's request and response format. Its hardest job is preserving meaning: which tool was requested, whether its arguments are complete, which result answers it, and which metadata must accompany the next request.

Suppose two tool calls stream simultaneously. One receives the fragment {"path":"src/; the other receives {"query":"parser"}. A single shared argument buffer can mix their inputs. Even separate buffers are insufficient if the connection closes before the first call finishes. A UI can display partial progress, but execution needs a stricter completion contract.

This page examines selected paths in the four pinned snapshots. It does not certify every provider integration. Sidekick's links point to private source and require repository access.

Three representations, three responsibilities

The display transcript is another view. A readable reasoning summary is not interchangeable with an opaque reasoning signature: a provider may require the latter for replay even though the UI has nothing useful to show. Similarly, a failed tool call still needs a coherent representation in the next request.

Four adaptation boundaries

These boundaries describe different scopes. Goose's provider interface also permits integrations that own their context; Sidekick's external Outpost runtimes have their own model protocol handling. Native adapter behavior should not automatically be attributed to those wrapped runtimes.

Completing a tool call is more than parsing partial JSON

Codex's inspected Responses parser forwards output_item.done as a complete typed item. It deliberately ignores function-argument delta events rather than reconstructing a function call from those fragments; custom-tool input deltas have a separate event. Turn handling consumes the completed item. The SSE reader reports an error if the stream ends before a completion event, and handles incomplete responses according to their reason. An interrupted response can become a completion with end_turn: false. Event mapping, completion handling, stream termination

This is not an assertion that Codex rejects every malformed frame. The SSE reader logs and skips JSON it cannot deserialize, and item conversion can ignore an unrecognized payload. The completion requirement is a specific invariant, not universal strictness.

pi's Responses adapter instead keeps slots keyed by output index, appends argument fragments to each slot, and parses partial JSON for progress events. At item completion it finalizes arguments and removes temporary buffers. It rejects a missing terminal response event; when the final stop reason is toolUse, it also rejects calls whose unfinished buffers remain. This makes “looks parseable so far” different from “the provider finished this call.” Assembly and finalization

Goose's OpenAI-compatible path accumulates arguments by tool index. Metadata-only frames are skipped without prematurely ending accumulation. On finalization, output-token exhaustion produces an error-bearing tool request; non-object JSON and unparseable arguments also become errors. Its argument parser can remove wrappers or repair some non-truncated malformation, but refuses inputs that look structurally truncated. That is a deliberate compatibility policy rather than permission to invent missing arguments. Accumulation, tool errors, argument parser

Sidekick accumulates native tool fragments by index and parses their accumulated argument strings after streaming ends. Invalid JSON is retained as a distinguished raw-input value. The agent loop recognizes that value, refuses execution, and supplies a correlated error result the model can correct. Partial field extraction used for UI updates does not authorize execution of the repaired preview. The translator handles exceptions and explicit provider errors, but does not itself require a terminal marker before building assistant content after normal iterator exhaustion. Its completion contract therefore differs from the inspected pi and Codex paths. Translator, refusal boundary

Keep the next request valid after failure

Consider this illustrative trace:

Codex's history normalization inserts missing call outputs, removes orphaned outputs unless they name external tool events, and strips unsupported image/audio content. This is a model-context invariant, not evidence that an interrupted external action was rolled back. History normalization

pi's transformation pass supplies missing results as errors saying no result was provided. It holds intervening system messages until the tool exchange closes, skips errored or aborted assistant messages, and remaps result IDs when call IDs change for the destination provider. Conversation transformation

Goose solves a particularly instructive failure: an unparseable tool request still needs an assistant-side call when replayed. Its OpenAI formatter emits a placeholder call named unparseable_tool_call with the original ID, allowing the following error result to remain paired. This placeholder describes failed history; it is not a new operation to execute. Failed-call serialization

Reasoning metadata belongs to its protocol

Codex's typed reasoning item contains summary, optional content, and encrypted content separately. Collapsing that item into its visible summary would discard fields the protocol can retain. Reasoning representation

pi makes replay compatibility explicit: reasoning survives only when provider, API, and model match. Signed or redacted blocks can survive even with empty visible text; foreign reasoning is dropped, and foreign tool thought signatures are removed. Its Responses adapter also backfills encrypted reasoning from the terminal response when an Azure response omitted it from the earlier completed item. Compatibility transformation, signature backfill

Goose's OpenAI formatter conditionally preserves thinking context, carries it into tool-call messages, and can translate it into configured content formats. Its common stream contract does not make every reasoning representation universally portable. Thinking serialization

Sidekick's lower OpenRouter adapter exposes structured reasoning_details, but the inspected unified-message translator accumulates textual reasoning_delta and creates a text thinking block without carrying those structured details into the assistant message. This review therefore does not establish lossless replay of opaque reasoning through the native loop. Displaying thinking and preserving provider replay state are different capabilities. Adapter details, unified output

Make capability fallbacks visible

An adapter can reject an unsupported feature, omit it, transform it, or delegate it elsewhere. Those choices change behavior. Codex declares provider capabilities as an upper bound that model and turn settings can restrict. pi replaces images with explicit placeholders for models without image input. Goose's interface exposes ownership of context, so local context management can defer to an external provider. Codex capabilities, pi modality conversion, Goose ownership

Sidekick illustrates another limitation: when its native adapter supplies a provider-only routing restriction, it sets require_parameters to false. Constructing a request parameter therefore does not establish that every routed endpoint will honor it. See Tools for capability negotiation and prompt caching for the separate cost of changing request structure. Routing configuration

Builder exercise: replay an adversarial stream

Build a fake provider that interleaves two tool calls, inserts a metadata frame, and closes before one call completes. Then repeat with valid completion, malformed arguments, a missing result, and a provider switch carrying signed reasoning.

Assert that fragments never cross call identities, unfinished arguments never become executable merely because they parse, and failures produce valid subsequent requests. Preserve original evidence separately from normalized context; identify synthesized errors as such. Test destination-specific reasoning and modality rules instead of assuming a universal transcript format.

Finally, record item completion separately from response completion. Some harnesses start completed tools before the entire model response finishes. A later stream failure can therefore coexist with already-started work; protocol repair alone cannot make replay safe. Continue with retries and failure semantics, testing and evaluation, and persistence and recovery to make those boundaries testable.

Large tool results and usefulness

A tool returns 30,000 log lines. Only three explain the failure, but the model cannot know which three without inspecting the evidence. A harness must decide what to retain, what to put in the next request, and how the agent can recover omitted material. Cutting the output to fit is one operation; judging whether it answers the task is another.

This chapter distinguishes verified output-handling mechanisms from proposed quality checks. The inspected paths below do not establish a general semantic usefulness classifier across these harnesses. Tool-specific validation, hooks, and the model's subsequent reasoning can add behavior, but should not be silently counted as a built-in result-quality judge.

Four different limits

These budgets are independent. A file path can recover omitted response text only if the file exists and an allowed tool can read it. A long-lived transcript can preserve evidence that is absent from the next model request. A recovery payload may be smaller than the original result.

What the four harnesses do

Sidekick's findings concern its native runtime. An external harness wrapped through an Outpost may handle output differently. The design comparison links to the reviewed harness versions and execution scopes.

Preserve a way to inspect the evidence

Pi's shared truncation defaults are 2,000 lines and 50 KiB, whichever is reached first. Its read tool keeps a bounded head of the selected content and reports the next offset. Bash instead exposes the tail and a path to fuller output when truncation occurs. That makes the response actionable: the agent can search the artifact or read a narrower region instead of repeating the original operation. Pi limits, read continuation, bash result

Goose's general large-response handler defaults to 200,000 characters per text block, configurable through its response-size setting. It writes the block to a temporary file and returns a notice pointing to it. Because file-write failure preserves the text with a warning, this handler is not a hard upper bound on total response size. Several smaller blocks or non-text content also require separate accounting. Goose large-response handler

The design lesson is to specify both the preview and retrieval contracts. An artifact reference should identify what was saved, how to read it, and its lifetime. A preview should say whether it is a head, tail, sampled excerpt, or summary. Otherwise an agent can mistake “not shown” for “not present.” These are builder recommendations, not claims that every implementation provides all these fields.

Keep model context separate from raw output

Codex's MCP output adapter gives model history a truncated response while preserving the underlying result for code mode. A script can inspect that result and emit selected evidence. Unified process output separately accounts for bytes omitted during collection and applies model-facing limits; increasing the final response budget cannot restore bytes already discarded upstream. MCP response projection, process omission markers

Code mode can reduce a large result to relevant records before it enters the model conversation. That shifts selection into a program, rather than proving the selection is correct. For example, filtering logs only for “error” may miss a failed assertion reported as a warning. Retain the original evidence or a reproducible retrieval path, and make the filter explicit.

Sidekick's optional context processor uses message age and approximate size. Its defaults target tool results at least ten messages behind the tail and roughly 1,000 tokens or larger. It replaces eligible contents with a placeholder while preserving tool-call identity in the request projection. The current factory enables this only through configured context preferences. Old content is not necessarily irrelevant: age is a heuristic for space management. Processor, activation

Sidekick also bounds the payload retained in a durable tool-intent record. It can drop inline image data and trim bulky result or metadata fields, recording truncation markers. That protects recovery storage but means a redelivered payload need not contain the complete original evidence. Re-running a mutation merely to obtain missing output would be a separate authorization and replay-safety decision. Bounded recovery payload, persistence and recovery.

Size, validity, success, and usefulness differ

Consider four responses from an invoice tool:

• Valid JSON containing the wrong customer's invoice is structurally valid but irrelevant.
• An empty result can be a correct answer or a failed query, depending on the contract.
• A successful HTTP response containing a login page is not necessarily a successful retrieval.
• A truncated log may contain the decisive error, or omit it entirely.

Schema checks assess shape. Exit codes and error flags report execution outcomes. Truncation markers describe omitted content. None alone establishes task relevance, factual correctness, or sufficient evidence. The inspected generic handlers use budgets, content types, age, and status; this review did not locate an always-on semantic usefulness classifier in those paths. This is a scoped finding, not proof that no tool or optional extension can perform quality evaluation. Codex budgeting, pi status handling, Goose content handling, Sidekick age/size logic

Builder exercise: keep useful evidence discoverable

Create fixture results containing a decisive line at the beginning, middle, and end, plus valid-but-irrelevant JSON and an explicit tool error. Apply your output policy and inspect what the next model request receives.

Check that truncation is visible, call/result identifiers survive, artifact retrieval works, and retries do not repeat side effects just to recover missing text. Record original size, retained size, representation type, and retrieval reference separately from execution status.

If you add a semantic evaluator, give it a concrete task contract and preserve its uncertainty. Measure missed evidence and false rejection of useful results, not just token reduction. Keep that evaluator distinct from size handling, tool authorization, and conversation compaction; each changes a different part of the agent's reasoning environment.

Prompt caching: preserve what the next request can reuse

An agent repeatedly sends much of the same context: instructions, tool definitions, and earlier conversation. Prompt caching lets a model provider reuse eligible computation from an earlier request. The harness controls the request it constructs; the provider controls whether reuse occurs. A cache key, retention preference, or breakpoint is a request to participate in that mechanism, not proof of a hit.

Keep this separate from application caches such as tool-discovery indexes, stored token counts, credentials, or loaded skill files. Also distinguish provider caching from a transport optimization that sends only new messages: reducing bytes sent over a WebSocket does not by itself prove model computation was reused.

This page compares the pinned request-building paths. It does not assert universal provider invalidation rules, current service guarantees, or measured savings. Sidekick citations require access to its private repository; its native OpenRouter path is separate from any externally wrapped harness.

A stable prefix is an architectural choice

Imagine an effective prompt containing stable instructions and tools, then conversation history, then newly changing information. Appending a tool result can leave the earlier prefix unchanged. Editing an early instruction or tool declaration changes the request before that history. A prefix-based cache may then reuse less of the request. Actual eligibility also depends on the provider, model, request options, routing, and retention behavior.

Scroll sideways to explore the diagram →

The diagram is conceptual. It does not specify every provider's wire ordering, tokenization, cache hierarchy, or hit boundary. The implementations below demonstrate different ways to preserve reusable input.

Four approaches in the inspected code

What changes when configuration changes?

“Invalidated” is often too vague. Record which input changed and which reuse mechanism stopped applying.

Codex: request identity, continuation, and effort

Codex derives its normal prompt cache key from the session identity, with explicit overrides and parent-scoped internal-session cases. It also gives rebuilt base-instruction and Responses Lite tool items deterministic identities derived from their visible contents within the thread. These reduce accidental request differences; a changed instruction still produces different content and identity. Cache identity, request construction.

Its WebSocket continuation path checks that request properties match and that input extends the previous request plus returned output. Model, top-level tools, reasoning, cache key, and other property changes can reject that incremental path. It then needs a non-incremental request; this is not evidence that every provider-side prefix was lost. Compatibility checks, input comparison.

There is a deliberate exception to “effort changes rewrite request configuration.” On the feature/model-supported path, Codex pins request-level effort for the context window and appends trusted configuration updates for changed effort. Successful compaction retires that baseline; unsupported models use the selected request-level effort. This preserves the instruction to use a new effort while avoiding unnecessary changes to the older request representation. It does not guarantee a cache hit. Effort implementation.

pi: preserve changes in the transcript when the API supports it

Pi compares structured system-prompt sections and produces an update for the next request. An API supporting mid-conversation system messages can retain those updates in place. Otherwise conversion collapses them into the current leading system message, changing the earlier prompt when necessary. A forced system prompt from an extension can also replace the request's leading prompt. Correct instructions take precedence over prefix preservation. Prompt preparation, conversion fallback.

Tool updates are similarly conditional. The inspected Anthropic adapter can keep initial request-level tools fixed and encode later additions/removals in messages, including a deferred placeholder from the initial request. This path requires support for both mid-conversation system messages and tool changes, plus a nonempty initial tool list. The OpenAI-style transcript helper anchors additions only when supported and when there are no removals or redefinitions; otherwise it sends the current catalog. These are compatibility-gated implementations, not universal API capabilities. Anthropic conversion, tool fallback.

Effort updates can use message/configuration items on supported paths; the OpenAI adapter excludes its configuration-update path for ChatGPT sign-in. Cache retention is also translated per adapter: pi exposes none, short, and long, while emitted fields depend on capabilities and authentication. Treat these as requested policies, not identical lifetimes across providers. Effort and retention options, request fields.

Pi additionally contains configurable cache warming: it may replay the prior request with a small output budget when its eligibility and estimated-economics checks permit. It stops when the saved request is no longer current and skips unsupported replay cases. This makes extra provider calls; its cost estimate is not a measured saving for a particular workload. Warmer eligibility, refresh and cost decision.

Goose: deterministic assembly and provider-specific breakpoints

Goose sorts model-visible tools by name and rounds its generated prompt timestamp to an hour. Both avoid changes unrelated to the task; the timestamp can still change at an hour boundary. When caching is enabled, its native Anthropic formatter marks the system prompt, final tool declaration, and last two user messages. Its OpenRouter-style chat helper instead marks eligible content near the tail and an earlier anchor, plus system/tools. Different envelopes require different placement strategies. Ordering, time, Anthropic format, chat breakpoints.

The OpenAI provider can attach the session ID as prompt_cache_key. Goose also accepts a configured cache TTL and re-derives it on session resume rather than treating a saved value as permanent session state. These controls do not preserve reuse after arbitrary model, reasoning, or prompt changes. No general mechanism deferring those changes solely to protect caching was established in this review. OpenAI key, retention configuration.

Sidekick: stable catalog, changing tail

Sidekick marks the first non-vanishing system message and last non-vanishing message as cache breakpoints. Its wire formatter consistently uses content arrays, avoiding a representation change just because a message stops being the active breakpoint. Configured vanishing context—such as current contextual information—is appended after history and excluded from breakpoint selection. This is a concrete separation between stable and frequently changing input. Wire formatting, tail assembly.

Its skill catalog is sorted and includes resolved version metadata in the system prompt. Loading instructions or references returns tool-result content; the fixed Skill, ReadSkillReference, and UseSkillTool interface avoids adding each skill's tools to the advertised array. However, editing attachments, descriptions, or resolved versions can change the catalog, and the catalog is rebuilt during context assembly. “Stable” means deterministic for the same inputs, not frozen forever. Native reasoning options are passed to OpenRouter; no equivalent effort-baseline preservation was established here. Catalog assembly, catalog rendering, skill interface, reasoning parameters.

Trace a skill change and design for it

Suppose request A advertises a review skill. Request B loads its body through a tool and appends the result. That can preserve A's earlier prefix. Before C, someone edits the advertised description and publishes a new version. If the harness rebuilds that early catalog, C changes before the appended result—even though the visible conversation looks unchanged. A supported appended instruction update offers a different representation; simply retaining the same cache key does not undo the change.

For a custom harness, make request assembly deterministic, place volatile data deliberately, and distinguish catalog edits from on-demand loading. Snapshot or version configuration at a documented boundary; defer optional refreshes only when doing so preserves correct behavior. Never delay revoked permissions or required instruction updates for cache savings.

Measure provider-reported cache reads/writes alongside model, request settings, and a digest of the effective prompt components. Track transport continuation separately. Test a repeated request, appended tool result, reordered catalog, changed schema, changed effort, skill load, skill edit, and compaction. These experiments reveal your actual contracts; code comments about caching do not substitute for observed provider usage.

Compaction is a state transition

Suppose a model has just read a large log. The next request exceeds its context window. Deleting arbitrary old messages may discard the user's correction or leave a tool result without its call. Summarization can retain intent, but it changes the representation from which the agent continues.

Ask four questions: what triggers compaction, what survives, when the result is adopted, and what remains available for replay?

Sources: Codex context checks and background adoption, pi compaction and session projection, Goose compaction operation and storage update.

Design implication: a background summary is computed against an older history. If the user changes the task while it is running, blindly replacing current context can lose that correction. Capture the input boundary, check compatibility when adopting, and preserve subsequent input. Do not confuse a summary successfully generated with a summary safely installed.

For an experienced builder, test compaction alongside tool-call/result pairing, model switches, user corrections, forks, cancellation, failed summaries, and overflow retries. A good summary of a static transcript does not test these transitions.

Sidekick: trimming is not summary compaction

Sidekick’s native loop rebuilds request context from stored messages each iteration. Its optional pipeline strategy replaces old, large tool outputs with placeholders while preserving tool-result identifiers. That changes the request projection, not the saved history. The inspected factory does not establish full-history summary checkpoints or automatic summary-compaction recovery. Configuration fields naming a summarization model do not establish those behaviors. An external Outpost runtime may manage its own context. Pipeline, trimming strategy (private source).

Steering, follow-up, and cancellation are separate controls

Imagine a model requests three tools and the user types “Stop editing; just explain.” There are at least three possible contracts: cancel active work, let the current call finish but skip the rest, or finish the batch and apply the instruction to the next request. A control named “interrupt” does not tell you which one is implemented.

For input arriving during a response/tool batch, pi's core consumes steering after that batch completes; it also polls initially and after next-turn preparation when needed. Follow-up waits until work would otherwise stop. Codex admits input to an active turn with expected-turn guards and has feature-controlled paths that can preempt sampling/retry waits. Goose drains a session's steering queue between model/tool phases; its ACP interface can check an expected active run ID. Pi scheduling boundaries, Codex start/steer, Goose steering.

Cancellation propagates to cooperating work. It does not undo a successful file write or remote action. Nor does a queued steering message prove the model has seen it. A client should expose the distinction between receiving input and admitting it into the next model context.

Design implication: specify the admission boundary and test it under a slow model, slow tool, parallel batch, pending approval, and run completion. A stale run ID should not accidentally steer a later task.

Sidekick: input placement is durable

Sidekick records queued input with its author and requested mode. At the next native main-loop iteration it checks cancellation, places steers, commits their message rows, and rebuilds context. Child, panel-peer, and deliberation loops do not drain that main-thread queue. After an answered turn, settlement can admit waiting follow-ups and unconsumed steers as a new turn; an unanswered settlement leaves them queued. Outpost adapters without steering support downgrade busy steering to follow-up. The arrival of an event and the admission of its input are separate steps. Iteration boundary, next-turn admission, adapter capabilities (private source).

Runaway agents: detecting repetition and bounding work

An agent can keep working without making progress. It might repeat a failing command, alternate between two unsuccessful approaches, or keep revising an already adequate answer because a hook refuses to let it stop. Preventing this requires several controls. A repetition detector recognizes a pattern; a budget limits expenditure whether or not the pattern is recognizable.

Consider a task to fix a test. The agent runs the same test three times, receives the same missing-dependency error, and changes nothing between runs. A repeated-call detector might intervene early. A twenty-step budget eventually stops the task even if the agent changes the command spelling on each attempt. Neither control establishes that the task succeeded. The final state must distinguish an answer from exhausted or interrupted work.

This page compares the pinned Codex, pi, Goose, and Sidekick implementations. Sidekick source links require repository access. Findings about missing detectors are limited to the inspected paths; no behavioral benchmark was run.

What each limit actually measures

These controls need separate counters. Compaction makes another model request fit; it does not replenish a task's spending allowance. Transport retry repeats a failed request; successful but useless tool calls need different feedback. A model's repetition-penalty sampling parameter is also different from a harness inspecting repeated actions.

The four implementations

Codex: continuation hooks and separate usage accounting

Codex's run_turn can continue for tools, queued input, compaction, or a blocking Stop hook. A hook that supplies a continuation prompt causes that prompt to enter history and the loop to continue. Subsequent hook calls receive stop_hook_active; the inspected branch does not implement a numeric repeated-hook cap. An empty blocking response produces a warning instead of an empty continuation. This is a concrete extension boundary where a host must understand how its hook eventually releases the task. Stop continuation Hook request

Configured rollout accounting is another boundary: record_rollout_budget_usage returns SessionBudgetExceeded when its shared budget reports exhaustion. This is usage enforcement, not a semantic detector of repeated commands, and its presence should not be read as a universal default step cap. Budget enforcement

Transport recovery has its own policy. handle_response_stream_error normally uses a retry count, but an optional feature permits continued retries for eligible connection failures. A transport fallback can reset the bounded counter. The UI receives reconnecting feedback. A claim that every Codex run is bounded by the same retry count would therefore be inaccurate. No general identical-tool-call detector was established in this review. Retry policy

pi: an explicit continuation decision, with retries outside the core loop

In pi's reusable runLoop, finishTurn can return end or continue. Without an explicit decision, tool continuation, steering, and queued follow-ups determine whether another request runs. The inspected core loop has no fixed iteration counter or repeated-command detector. Applications can implement a bounded policy at this hook; extensions that continually return continue can also create unproductive work. Core continuation

The coding application separately limits retryable failures through _prepareRetry, using configured maxRetries, exponential backoff, an abortable wait, and retry lifecycle events. It retains the failed attempt in raw history while omitting it from the next model projection. These retries do not count successful, repetitive actions as failures. They also demonstrate why a core agent_end event alone is insufficient to infer that the complete application has stopped trying. Retry preparation

Goose: turn limits, hook limits, and an inactive repetition configuration

Goose's MaxTurnsOperation counts assistant blocks since the current kickoff and yields a message asking whether the user wants to continue once the configured allowance is reached. Adjacent assistant messages form one block, so this is not simply a count of tool calls or emitted events. The model receives budget information after half the allowance has been used. Session configuration overrides global GOOSE_MAX_TURNS; the inspected agent fallback is 1,000. Counting Enforcement and feedback Configuration

Goose also contains a RepetitionInspector that compares a tool's name and argument value with the previous call. However, ordinary agent construction registers it with None, which disables the repetition threshold. Its inspection method checks temporary copies; the existence of this class does not establish an active, accumulating default repetition detector. Inspector implementation Registration

StopHookOperation has a distinct block cap, defaulting to eight. It counts recorded denial markers in the current kickoff window and eventually yields an explicit warning. Recipe retries have another maximum and can replace the attempted conversation suffix while retaining their attempt counter on the kickoff message. Goal and grind continuations also insert prompts. These mechanisms explain why inspecting only the inference function misses important reasons an agent keeps going. Hook cap Defaults Recipe and goal continuation

Sidekick: explicit iteration exhaustion within an event-driven system

Sidekick's native run_agent_loop defaults to 25 iterations and retries empty responses at most twice within that loop. The bounded loop checks cancellation at iteration boundaries. Exhaustion emits a typed error result and returns a failed outcome with reason iteration_limit; it is not silently reported as an answer. No general semantic or identical-call loop detector was established in this path. Defaults Loop boundary Empty-response handling Exhaustion

The scope of the counter matters. Each hidden deliberation pass invokes its own loop and gets its own allowance, separate from the eventual synthesis. Workflow retries or resumed attempts are also distinct from one invocation's iteration counter. A worker execution timeout is not necessarily an overall deadline: the workflow heartbeat refreshes it. Thus “25 iterations” must not be presented as a total conversation, multi-agent, or wall-clock budget. Deliberation scopes Heartbeat

Model-based supervision and progress assessment

A progress supervisor is a separate model invocation that evaluates the worker's recent trajectory against the task objective. It can decide whether to continue, redirect the worker, or stop. A repeated-command detector asks whether actions look alike; a supervisor can ask whether different actions are advancing the task. Its judgment is still fallible and needs evidence.

No built-in periodic model-based progress supervisor was established in the inspected paths of these four snapshots. There are related mechanisms and useful integration points:

The distinction is between reviewing the answer, deciding whether completion is acceptable, and supervising progress during execution. All can produce useful guidance, but they observe different evidence at different times. See multi-agent delegation for ownership, communication, cancellation, and workspace boundaries when a supervisor coordinates a worker.

Pattern: a stronger supervisor over a bounded worker

The following is a proposed architecture, not a claim that these snapshots ship it. Use a lower-cost worker for routine execution and a stronger model for selected supervisory checks. “Stronger” should mean better performance on your progress-assessment evaluations; a larger or more expensive model is not automatically a better judge.

1. Run a bounded segment. Let the worker complete a small allowance of turns, or trigger review after repeated failures, conflicting results, or an attempted completion. Check at a defined execution boundary; do not silently replay active mutations.
2. Provide evidence. Send the objective and acceptance criteria, recent calls and results, actual file or artifact changes, prior guidance, and remaining budget. Include a compact longer-term record so an alternating cycle cannot hide outside the recent window. The worker's claim that it made progress is only one input.
3. Request a structured decision. Require a decision, supporting evidence, and a concrete next action or stopping reason. Keep the supervisor's review context separate from the worker's execution context; tool output remains evidence rather than authority to rewrite the task.
4. Apply the decision through the harness. Associate it with the reviewed run and context revision. If new user input or completed work makes it stale, reconcile or review again. Inject guidance at the next steering boundary, or record an explicit terminal outcome.

For the missing-dependency example, useful guidance is: “The last three test runs failed before any tests executed. Inspect dependency installation before editing the parser again. Resume testing once collection succeeds.” This gives the worker a diagnosis, an alternative action, and an observable checkpoint. “Try harder” supplies none of them.

A stronger supervisor can examine less evidence less often while the worker handles routine steps, but the added calls cost tokens and latency. Measure whether interventions improve task outcomes enough to justify that cost. Review too often and the worker may be interrupted before an approach can succeed; review too rarely and the allowance may be spent before redirection helps. A supervisor can also share the worker's mistaken assumptions or miss facts omitted from its summary.

Keep hard budgets outside both models. Charge worker, supervisor, child-agent, and retry usage to the task; cap reviews and repeated redirects. A supervisor cannot renew its own allowance or bypass tool authorization. Define an explicit outcome for judge timeout, malformed decisions, and unavailable evidence rather than treating those as unlimited permission to continue. Stopping future work does not undo an operation already performed.

Test the supervisor's decisions

Start with controlled traces: an identical failure repeated, an alternating two-strategy cycle, legitimate polling, slow but measurable progress, and a worker claiming success despite a failing acceptance check. Also test stale guidance after user steering and repeated disagreement between supervisor and worker. Assert that redirection reaches the correct run and exhausted budgets still stop it.

Then compare the same task set with and without supervision, holding worker configuration and total allowance constant. Record success, unnecessary stops, useful redirects, cost, and elapsed time, including supervisor usage. This evaluates the stronger-model pattern instead of assuming it works. See testing and evaluation for the distinction between runtime contracts and real-model task outcomes.

A bounded continuation design

For a new harness, first define a durable task budget, then allocate child and retry allowances from it. Record consumption across process restarts. Keep a separate short-window progress detector: normalize tool arguments, include relevant result/error signatures, and distinguish deliberate polling from repeated failed mutations. Treat its verdict as a reason to pause or change strategy, not proof that a task is impossible.

For the failing-test example, feedback might say: “The last three runs produced the same missing-dependency error and no file change. Inspect the environment before running this test again.” If the global allowance expires anyway, report unfinished work, consumed limits, and any uncertain side effects. Allow an explicit continuation decision without pretending that it erases prior spending.

Finally, cancellation must reach model requests, queued tools, active executors, and retry waits. A useful regression trace covers a repeating tool, an alternating two-step cycle, a permanently blocking stop hook, a child exhausting its allocation, and a restart near the limit. Test the counters and terminal outcomes rather than relying on the model to obey a reminder. See steering and cancellation and persistence and recovery for the surrounding contracts.

Multi-player is more than multi-agent

Here multi-player means multiple humans or clients participating in shared work. Multi-agent means several model-driven workers. They can coexist, but adding workers does not solve shared-human authority.

Consider two clients attached to one session. Alice starts a change; Bob submits a conflicting instruction while Alice is approving a command. Which run receives Bob's input? Who may approve? Will both clients see the same event history? What happens if the approving client disconnects?

Sources: Codex subscriptions and turn checks, pi RPC and durable views, Goose shared registry and notification forwarding.

“Not established” is a scope limit, not an absence claim. Those three sets of code paths do not prove per-human identity, room membership, role-based approval, presence, collaborative file editing, or conflict resolution. A connection ID is not automatically a human identity.

Design implication: separate the session log from its audiences and writers. Define who can append input, control a run, grant permission, and receive tool output. Reconnection needs an ordering contract between the historical snapshot and live events; otherwise updates can be missed or duplicated.

Sidekick: shared humans are an application boundary

Sidekick establishes a concrete shared-conversation surface. Access checks distinguish view, edit, and manage rights, including direct and group sharing. Sending requires edit access; submissions retain author identity and client-message IDs. Conversation locking decides whether input is placed or queued. These are application-level collaboration contracts beyond accepting multiple connections. Access checks, send admission (private source).

Concurrent message writers obtain sequence numbers atomically. Reconnecting clients receive a state cursor and a queue high-water mark to reconcile snapshots with later updates. Persisted messages, transient streaming events, and authoritative turn rows still have different lifetimes. This supports shared observation and controlled writing; it does not establish collaborative file editing or one transactional event log. Sequence allocation, queue reconciliation, state cursor.

Multi-agent delegation introduces ownership

Codex exposes child-thread lifecycle controls, including spawning, messaging, waiting, and interruption. Pi's coding-agent supplies an optional example that launches child processes; its separate durable package demonstrates owned child conversations. Goose runs foreground subagents in separate persisted child sessions and returns their results and approval requests through the parent workflow. Codex controls, pi example and durable example, Goose subagents.

Separate conversation state does not prove separate filesystem access. Two agents can have independent histories and still overwrite the same file. The delegation contract should identify task ownership, workspace isolation, cancellation propagation, approval routing, budgets, and what the parent receives.

Design implication: delegate independent work with a bounded result contract. For shared mutations, provide explicit ownership or conflict handling rather than assuming a coordinator can repair every race afterward.

A parent can also supervise a worker's progress instead of dividing a task into independent subtasks. The proposed model-based supervision pattern uses a separate, potentially stronger model to assess evidence and return bounded continuation, redirection, or stopping decisions. Ordinary delegation alone does not establish that this assessment happens.

Sidekick: owned child turns and peer panels

Native delegation admits an owned child turn only while its parent attempt still owns the running turn. The parent tool intent records the child identity in the same transaction. The child then has its own attempt, scoped context, intent recovery, cancellation checks, and settlement. A retry can reason about that existing child instead of blindly spawning another worker. Child linkage, child execution (private source).

Panels use a different topology: peers receive persisted permission to speak, called floor grants. When the last statement in a locked batch completes or fails, finalization produces a directive to wake the moderator. This is event-driven coordination with durable state determining which work becomes eligible next. It coexists with an explicit model/tool loop inside each native worker. Panel completion.

MCP separates capability providers from the loop

MCP provides a standard way to discover tools and call them through a server boundary. The harness still decides which tools become visible, when schemas refresh, how calls are authorized, how long they run, and what to do if a connection fails after an effect may have occurred.

Codex captures tool bindings and output-policy state for admitted MCP calls and can expose deferred tools through lexical search. Pi's built-in MCP integration handles configured connections and tool-list changes, adapting tools into the agent pipeline. Goose routes names through session-scoped extension leases. Codex MCP handler and search, pi MCP runtime, Goose lease.

A tool catalog can change while work runs. If a model requested tool lookup under catalog A, refreshing to catalog B should not silently change what that already-admitted call means. Catalog lifetime and binding ownership are therefore correctness concerns, not only caching details.

Read-only annotations can guide scheduling or retry policy. They are metadata supplied by an integration, not proof that an arbitrary remote server has no effects. Similarly, an MCP connection standard does not imply that all servers are inside the same sandbox.

Sidekick: hosting an agent does not establish native MCP

The inspected native executor routes registered system tools, configured CLI tools, and stored Python tools. No native MCP discovery/client path was established in the backend and Outpost review. The ACP adapter supplies an empty MCP-server list when creating or loading a session. ACP carries an agent session; MCP exposes capabilities such as tools. An independently configured wrapped harness may use MCP, but that is not evidence of a Sidekick-owned MCP client. Native routing, ACP configuration (private source).

Code mode for MCP changes the orchestration boundary

With direct tool calling, the model may need a request for “search,” another for “fetch each result,” and another to combine them. With code mode, the model writes a small program that performs those calls and returns the useful subset.

This conceptual example is pseudocode; each harness exposes different names and result shapes:

matches = await tools.search(query)
details = await concurrently(fetch(match) for match in matches)
emit only the fields needed for the answer

This can reduce model round trips and intermediate context. It also moves control flow into model-written code. You now need execution limits, cancellation, observable nested calls, explicit result/error shapes, and a clear policy boundary around the capabilities exposed to that code.

Sources: Codex runtime and nested dispatch, pi nested execution, Goose code execution and registration.

The Goose distinction is an architectural scope finding, not a claim that nested tools have no checks anywhere. Its catalog restriction, lease routing, server-side behavior, cancellation, and outer policy still matter. Conversely, routing through a hook in Codex or pi only enforces the policy actually configured there.

Result shapes also differ. Goose's nested callback returns assistant-visible text, parsed as JSON where possible; it excludes arbitrary structured content, images, and hidden text from that adapter. Pi can expose declared MCP structured output to scripts. A program that assumes every tool returns the same JSON structure can fail even when both harnesses support the same server. Goose result adapter, pi MCP tool adapter.

Design implication: test a denied tool both directly and inside code mode. Do not assume an interpreter with no filesystem or network APIs is harmless: its injected tools can carry filesystem, network, and credential authority. Review the capability boundary as carefully as the JavaScript runtime.

Sidekick: configured Python tools are a different mechanism

Sidekick can execute configured Python tool implementations and dispatch a skill’s declared built-in tool through UseSkillTool. Neither inspected mechanism is a model-authored program that discovers and composes MCP tools. Native MCP code mode was not established in this review; a wrapped external harness may bring its own facilities. Distinguish execution of an installed implementation from generation of an orchestration program during a run. Configured Python execution, skill dispatch (private source).

Tool safety classification: risk, authorization, and execution

A model requests a Bash command. Before executing it, the harness can ask a second model to assess its effects and whether the user authorized them. This is model-based tool review. It can reduce approval interruptions, but “auto mode” is not a standardized contract: one implementation may invoke a reviewer, another may simply permit execution.

Suppose the user asks for a parser explanation and the agent proposes touch /tmp/unrelated-note.txt. Creating that empty file is low risk, but unrelated to the request. Conversely, deleting a production database may be explicitly requested and still prohibited by policy. “Safe,” “authorized,” and “allowed” answer different questions.

Four decisions that should stay separate

An allow decision is not proof that all four answers were favorable. Codex's bundled reviewer policy, for example, generally allows low- and medium-risk actions regardless of authorization, with explicit-policy and malicious-injection exceptions. Its high-risk threshold depends on authorization. Deployments can supply policy configuration, so read the effective rules rather than interpreting allow as “the user explicitly approved this.” Bundled decision policy.

Model review, hooks, and permission judges

These findings describe the pinned source snapshots used throughout this guide, not every hosted product or provider integration.

Goose also has a separate security scanner combining command patterns with optional command/prompt classifiers. Its shell scan estimates maliciousness; this is distinct from SmartApprove's read-only classification. A failed classifier call returns no model score, leaving pattern evidence available. Scanner composition, classifier failure.

Put review before the effect

A useful design sequence is: validate arguments, resolve the exact action, apply rules, review when required, bind the decision to that action, and execute under the selected restrictions. This is a design pattern, not a claim that every tool follows identical code.

Codex's tool orchestrator resolves an approval requirement before its sandboxed execution attempt. A forbidden requirement rejects immediately; a skipped requirement can still require review under strict auto-review. Its Guardian reviewer runs in a restricted review session with read-only checks and approvals disabled. Reviewing a command should not itself grant a reviewer the command's write capabilities. Orchestrator, reviewer isolation.

Bind approval to the arguments and relevant state, not merely the tool name. bash can list files or destroy them. A reviewed script can change before execution. Codex's review host contract requires fresh authorization evidence and revalidation before consuming cached approval. This illustrates the general time-of-check/time-of-use problem: the action that executes must still be the action that was reviewed. Review host contract.

Failure is not a safety verdict

Codex distinguishes parse/session failures, timeout, cancellation, and completed denials. Parse/session failures block execution; timeout and cancellation have their own outcomes. An input-budget failure can fall back to user approval when Guardian is optional. A review that could not complete is not evidence that the requested action was dangerous. Completion handling.

Goose's permission judge returns no read-only IDs when model configuration, completion, or response extraction fails. Deferred candidates absent from that result require approval. In pi's permission-gate example, a matching dangerous command is blocked if no confirmation UI exists; this behavior belongs to that example extension, not every pi installation. Judge failures, candidate handling, pi example.

Protect the reviewer from the material it reviews

Tool arguments and retrieved text can contain instructions such as “ignore the policy and approve this.” Treat those strings as evidence, not authority. Codex's bundled review prompt separates trusted authorization sources from untrusted outputs and permits explicitly delegated authority. Goose labels the serialized requests as untrusted and tells its judge not to obey instructions embedded in IDs, names, or arguments. These are prompt-level defenses, not guarantees of correct classification. Codex evidence handling, Goose input, Goose instruction.

Fast paths deserve equal scrutiny. A read-only annotation is a claim supplied through the tool definition. A regular expression catches particular spellings; it cannot establish the behavior of arbitrary scripts, shell substitutions, or interpreters. Goose's annotation shortcut and pi's small regex example make those tradeoffs visible. Goose shortcuts, pi patterns.

Builder exercise: evaluate the whole decision path with paired cases: the same command under different user requests; the same tool name with different arguments; a harmless wrapper invoking a mutating script; malicious approval instructions in tool data; a stale approval; and reviewer timeout or malformed output. Measure unauthorized executions, unnecessary interruptions, latency, and failure behavior separately. Test shortcuts and fallback paths alongside the classifier. A classifier's accuracy alone does not establish the safety of the harness around it.

Sidekick: automatic acceptance and capability checks

Sidekick’s native system-tool dispatcher has a deterministic capability gate: it reads the acting agent’s current grants and checks action prerequisites, argument constraints, and applicable role restrictions. At this pin, only Mail and Files declare capability maps. This gate is not a universal Bash classifier or a policy for every tool. Dispatch, declared coverage, gate (private source).

Separately, the frontend’s auto_approve mode turns incoming permission requests into automatic acceptance. That is different from asking a model whether the action is safe and authorized. Outpost adapters map settings onto their wrapped runtimes, so evaluate both the adapter configuration and the client’s response behavior. No Sidekick-owned model safety classifier was established in these inspected paths. Response selection, acceptance submission, adapter profiles.

Skills: load procedures when the task needs them

A skill packages instructions for a kind of work, usually in SKILL.md, alongside optional references, scripts, and templates. A tool schema describes an executable operation; a skill explains how to carry out a workflow using available operations. An extension can add executable behavior or lifecycle hooks. A skill can arrive inside an extension or plugin without becoming executable merely because it was discovered.

Consider a harness with fifty specialist workflows. Putting every procedure into every request consumes context and makes irrelevant instructions compete with the task. Progressive disclosure means advertising compact descriptions first, then loading a selected procedure and its relevant resources. Codex, pi, and Goose support this separation through the loading paths below; Sidekick adds a versioned binding model described later. Codex catalog guidance, pi prompt formatting, Goose skill operation.

Follow one skill through the loop

Suppose the user asks: “Extract the invoice totals from these PDFs.” A teaching trace is:

1. Discover: the harness finds invoice-extraction, its description, and its source identity. It advertises this metadata to the model.
2. Select: the model recognizes a matching description, or the user explicitly invokes the skill. These are separate entry points; an implicit match is not guaranteed.
3. Load: the harness injects the procedure, or the model requests its contents through a reader. The procedure directs it to a format reference and an extraction script.
4. Resolve resources: the agent loads that reference relative to the skill's location. A remote package needs its provider's reader, not an invented local path.
5. Act: the agent requests a command to execute the script. Tool authorization and sandbox enforcement apply at this execution boundary.
6. Continue: results return to the ordinary agent loop. Loading a skill did not create a separate autonomous worker.

This trace is a design abstraction. The pinned implementations below do not all produce the same message roles or tool calls.

File and provider catalog loading

Sources: Codex catalog instructions and explicit injection; pi catalog filtering and command expansion; Goose loading, command effects, and command precedence. These describe the pinned paths, not every UI or deployment configuration.

Identity and precedence are behavioral contracts

Two packages can both contain a skill named review. Preserve the selected source identity, expose collisions, and make resolution predictable. Display names alone are insufficient for reproducibility.

Codex illustrates why subsystem scope matters: its host selection helper requires an unambiguous plain name, while the extension catalog selector takes the first enabled plain-name match and deduplicates by authority plus package. Both support explicit source selection. Do not describe these as one universal collision rule. Host selection, extension selection.

Pi keeps the first discovered name and emits a collision diagnostic containing winner and loser paths; it also deduplicates canonical file paths. Goose's discovery visits roots in order and keeps the first parsed name. Its project roots precede global roots, but duplicate winners within one root are unspecified because traversal is unsorted. Pi loader, Goose root order and discovery contract.

A loaded procedure is not an execution grant

Distinguish discovery precedence—which file wins—from instruction authority—which instructions the agent must obey. Finding a project skill does not establish who authored it or authorize every effect it requests. A procedure saying “publish the report” still needs the user's applicable authorization and the harness's execution policy.

Keep tool permissions in an enforced policy layer. Treat skill metadata as declarations unless that layer explicitly validates and interprets them. In pi's inspected loader, disable-model-invocation affects catalog visibility; that mechanism is not a command sandbox. A bundled script remains code that executes through a tool. Pi loader and formatter.

File discovery and safe file access are also distinct. Goose's discovery contract explicitly says traversal can follow symlinks outside a search root. Its supporting-file reader separately rejects parent-directory paths and uses platform-specific confined opens; the native load_skill path also runs pre/post tool hooks. A successful discovery scan is therefore not evidence that every later read is safe. Discovery boundary, supporting-file access, tool lifecycle.

Budget context and define refresh behavior

Progressive disclosure reduces initial prompt size; loaded procedures and references still consume context. Codex budgets catalog metadata, can shorten descriptions or omit entries, and warns when an explicitly injected main prompt exceeds its limit. A shortened routing description may change selection quality, so record truncation as observable behavior. Metadata allocation, main-prompt limit.

Refresh metadata and content deliberately. Pi's explicit command reads the current file using the already-loaded catalog entry; /reload refreshes resources. Goose discovers skills when building its prompt contribution and again when handling a load. A file changed between discovery and execution can therefore differ from the version originally described. For reproducible runs, record the selected identity and content digest, and define what happens after compaction removes earlier instructions. Pi expansion and reload, Goose operation.

Builder checks

• Install two identically named skills and verify the selected source and collision report.
• Hide a skill from automatic selection and verify explicit invocation separately.
• Change or delete a file after discovery; verify refresh and missing-file behavior.
• Test oversized bodies, paginated reads, relative resources, and symlink escapes.
• Request an action explicitly forbidden by execution policy and verify the skill cannot override that denial.
• Compact after loading a procedure and verify the agent can recover required instructions without silently substituting a newer version.

Sidekick: versioned bindings and a stable tool interface

Sidekick stores skills as drafts and published versions, with saved agent bindings resolving a version or channel. A metadata catalog enters the system prompt; Skill and ReadSkillReference load instructions and references on demand as tool results. This makes saved bindings, rather than filesystem discovery order, the runtime selection boundary. Draft access follows the saved binding rather than the conversation user’s skill-edit permissions. Binding resolution, catalog assembly (private source).

UseSkillTool checks that a skill is attached and that the requested tool is declared, then invokes the normal system dispatcher with the acting agent’s context. This preserves its applicable capability checks. Only built-in system tools are supported through that path at the pin. “Load the skill first” is model guidance; the dispatcher checks the binding on invocation rather than requiring proof of an earlier load event. A skill’s instructions, its binding, and its executable capabilities therefore remain distinct contracts. Invocation boundary.

Next user prompt suggestions

A next user prompt suggestion is a proposed message the user might send after an agent finishes. After “The parser fix passes its tests,” the interface might offer “Explain why the old parser failed.” The user can accept, edit, or ignore it. Generating that text is a separate inference task from answering the original request.

The interesting engineering problem is ownership: the model may predict an instruction, but only the user can adopt it. A suggestion must not silently enter the conversation as something the user said.

Separate prediction from nearby features

These features can share an editor or event system, but their contracts differ. See steering and cancellation for input that has already been submitted.

What the pinned sources establish

Codex's request carries thread_id and source_turn_id. Its notification repeats those identifiers and distinguishes Completed { text: Option<String> } from Failed. That contract can represent both a successful prediction and a successful decision to suggest nothing. The handler's explicit rejection prevents treating the types alone as a working feature. Prediction types, request handler.

Codex also supports experimentalPredictionMode on thread forks. It requires an ephemeral fork of a loaded parent, inherits parent state, disables automatic compaction, and enables reasoning-effort override. The protocol comment identifies prompt-cache reuse as the purpose of inheriting configuration, tools, environments, and history. This is useful supporting infrastructure, but it does not establish the prediction prompt, trigger, filtering rules, or acceptance UI. Fork contract, fork implementation.

pi's built-in completion catalog and optional GitHub issue example have concrete data sources; the latter fetches open issues and filters them locally. Goose's desktop adapter obtains commands and mentions through its ACP client. These are useful extension points, not evidence that either harness predicts a new user message. Pi catalog, issue example, Goose adapter.

Search boundary: this review checked Codex core, app-server, protocol, and TUI sources; pi agent, coding-agent, TUI, web-UI sources and extension examples; and Goose agent/server and desktop sources. Searches covered prompt suggestions, predictions, autocomplete, and follow-up paths at the registered commits. Codex rendering fixtures contain references to next_prompt_suggestion.rs; those fixture strings do not establish an implementation. The conclusions concern these source snapshots, not every client, external extension, or subsequent release.

A design for an optional suggestion worker

The following is a proposed design, not a claim that the compared harnesses implement it.

1. Wait for the run to settle. Start after the final response and after queued work is exhausted. A low-level completion event may be too early: pi's notification example uses agent_settled because agent_end can precede retry, compaction, or queued follow-up work. Pi settled event example.
2. Capture a bounded context snapshot. Include recent user intent, the final answer, and relevant outcomes. Attach the session, completed-turn identifier, and composer revision. Exclude pending tool calls and unrelated private material.
3. Request text or abstention. Use a separate instruction such as “Suggest one concise message this user is likely to send next; return no suggestion when uncertain.” Require a small validated output, with no tool execution. Keep this synthetic instruction outside normal conversation history.
4. Reject stale results. Give inference a short time budget. Cancel when typing, a new turn, a branch switch, or a session change invalidates the snapshot. Recheck identifiers when the result arrives: cancellation alone may lose a race.
5. Present an editable draft. Acceptance can fill the composer; submission follows the ordinary user-message path. A displayed or accepted draft grants no permission to execute its contents.

For example, turn 12 ends and prediction starts. The user immediately types “Actually, revert that fix.” A late prediction saying “Commit the fix” belongs to turn 12's old composer state and must be discarded. It must never overwrite the user's correction or queue a commit.

pi's autocomplete editor demonstrates a reusable part of this design: it passes an abort signal, snapshots text and cursor position, and checks request identity plus the unchanged editor state before applying a result. This is evidence for handling asynchronous editor results, not a prediction-model implementation. Editor freshness checks.

Cost, privacy, and evaluation

Design implications: prediction adds model calls even when nobody accepts the result. Budget tokens and latency separately from the main agent, permit abstention, and avoid delaying the final response. Reusing full conversation context can improve relevance while exposing more data to the selected provider; use deliberate context selection and make any provider change explicit.

Evaluate acceptance together with edits, dismissals, stale-result rejection, and whether suggestions remain within the user's intent. Acceptance alone can reward intrusive suggestions. Include cases with a completed task, unanswered clarification, sensitive tool output, a newly typed correction, and multiple clients. The essential invariant is that prediction produces optional text; execution still requires a submitted user instruction and the harness's normal policy checks.

Sidekick: clarification is not prediction

No dedicated next-user-message prediction path was located in the inspected backend, frontend conversation state, or chat composer. The native AskUserQuestion tool asks clarifying questions and suspends execution until answers resume it. That is an assistant requesting missing input, not a model proposing the user’s next message. This is a bounded finding about the reviewed sources, not a claim that an external client or wrapped agent cannot provide suggestions. Question contract (private source).

Persistence, recovery, and replay

A saved conversation is evidence of what was recorded. It does not guarantee that an action was never executed twice or that every event a user saw reached storage.

Codex's ThreadStore distinguishes standard persistence from contexts allowed to enqueue background writes, with explicit flush/shutdown barriers. Pi's coding-agent emits message events before persistence; the separate experimental durable runtime makes stronger tool-intent and replay-policy guarantees. Goose's machine persists effects and reloads session state; live streaming may precede persistence, and pending approval execution depends on retaining a valid extension lease. Codex storage contract, pi events and durable recovery, Goose effect application and tool execution.

Design implication: model recovery around uncertainty. If a tool charged a card and the process crashed before recording success, blindly rerunning it is wrong. Use an externally enforced idempotency key, reconcile status, or surface an interrupted result. Avoid claiming exactly-once effects from the existence of a database or append-only log.

Sidekick: recover the turn, not just the worker

Sidekick's native path separates a durable turn from the worker attempt advancing it. A heartbeat and ownership checks govern replacement attempts. A superseded attempt cannot write a valid final settlement, although this does not stop an external operation it already started. Outpost dispatch attempts, which represent separately hosted execution, are not automatically taken over and resent. Attempt ownership. Sidekick's source is private; these pinned links require repository access.

The tool boundary is equally explicit. The loop commits tool intents beside the assistant request before dispatching them. Suppose a tool changes a remote system and its worker disappears before the result enters the transcript. Recovery first inspects the intent: a stored result can be redelivered; an unclaimed call can be claimed; a running call has an uncertain outcome. After waiting for an existing result, the native recovery path reruns a running call only when both stored and current replay policy allow it. Otherwise it returns an interrupted-result explanation. Replay permission is separate from user authorization. Intent commit, recovery decisions, policy intersection.

This is a stronger contract than replaying the last transcript entry, but it is not exactly-once execution. A replay-safe operation may run more than once. Post-commit hooks and completion projections also remain separate work: the sweeper logs failures rather than promising durable retries for every downstream notification. The Sidekick chapter traces these failure windows. Post-commit limits.

Retries and failure semantics

An agent loses its connection while creating an issue. Should the harness retry? That depends on what failed: the model request, the tool connection, the local worker, or only delivery of the result. A missing response does not establish that the issue was never created.

Retry scope is the unit of work repeated. Idempotency means repeated execution has the same intended effect as one execution. Reconciliation means checking the external system to determine what actually happened. These concepts matter more than the choice of exponential backoff.

This page compares the pinned Codex, pi, Goose, and Sidekick sources. Sidekick citations require private repository access. Findings describe inspected paths, not a universal guarantee for every provider, extension, or external service.

Trace the ambiguous failure

Consider this illustrative execution:

1. The model requests create_issue with call ID call-42.
2. The harness records the request, then invokes the tool.
3. The remote service creates issue 123.
4. The connection or worker dies before a result is durably recorded.
5. Recovery finds a request without a result.

Repeating step two can create issue 124. Repeating the model request can also produce a new call ID for the same intended action. A tool-call ID correlates protocol messages; it only prevents duplicate effects if the destination enforces it as a stable idempotency key. Restoring conversation history cannot undo issue 123.

The recovery decision needs evidence: not started, completed with stored result, or started with unknown outcome. An unknown mutation should be reconciled or reported as uncertain. A retry-safe read may be rerun, but its new result can differ because the world changed.

Compare the retry boundaries

Codex: request recovery and tool escalation are separate

Codex's sampling retry loop rebuilds subsequent request input from current session history rather than always replaying the initial input unchanged. Context-window and usage-limit failures leave this path immediately. The retry handler classifies other errors and respects server retry timing; a WebSocket-to-HTTPS fallback can reset its bounded counter. An optional feature permits unbounded retries for eligible connection failures, with additional restrictions on session source and provider. Sampling loop, retry decisions.

The caller wraps the retry handler in cancellation and preemption handling, so the backoff wait can be interrupted. These are request-recovery mechanics, not a blanket instruction to execute every earlier tool again. Likewise, a reconnect count is not a total task budget. Cancellation boundary.

Tool escalation has its own policy. Following an eligible sandbox failure, the orchestrator checks whether escalation is allowed and whether another approval is required. An automatic review for the original, narrower action does not automatically authorize the broader retry. This illustrates two independent questions: “May this attempt run?” and “Could repeating it duplicate an effect?” The inspected escalation path answers the first; it is not a generic exactly-once protocol for arbitrary commands. Escalation and approval.

pi: application retries versus durable tool recovery

Pi's reusable loop ends on an errored or aborted assistant message before dispatching that message's tool calls. The coding application may then prepare another request: it checks retry settings, increments a bounded counter, emits lifecycle events, and waits with an abort signal. The failed assistant attempt remains in raw history but is omitted from the next model projection. Context overflow is routed to compaction instead of ordinary transient retry. Core ordering, retry preparation.

The separate experimental durable runtime addresses a different failure. Before execution it commits the final tool arguments and a replay policy, defaulting to unsafe. Recovery reruns only when both the saved policy and the currently resolved tool say safe. Otherwise it settles an interrupted result explaining that the tool may have partially run. Changing a tool's policy after a crash cannot unilaterally grant replay permission. This is evidence-based recovery, not proof of exactly-once external execution. Intent and recovery.

Goose: multiple budgets and a first-item boundary

Goose's shared provider helper uses configurable bounded retries, jittered backoff, and provider-supplied rate-limit delays. Its credential refresh allowance is separate from transient-error retries. Classification also matters: the general default can retry some request failures, while permanent thinking-signature/configuration errors are excluded. Provider policy.

The agent's stream wrapper chooses a stricter transient_only policy and retries only before the first successful stream item. Once an item arrives, it forwards the stream; this wrapper does not restart generation after arbitrary partial output. Providers that manage their own context bypass this outer retry window. Provider-internal retries can still exist, so a single counter is not the complete request budget. Stream boundary.

A recipe retry is larger in scope. After a completed response fails configured success checks, Goose can run an on_failure command and reset conversation messages to the kickoff. The attempt count survives on that kickoff message. This allows another attempt at the task, but previously executed tools and the failure command can have persistent effects. Builders must design cleanup and idempotency explicitly; a transcript reset is not a transaction rollback. Recipe checks and reset.

Sidekick: replace an attempt without blindly repeating actions

Sidekick's conversation workflow declares three retries. On an ordinary native worker exception, it interrupts the attempt and returns the turn to pending; exhausted retries reach a terminal failure handler. Those workflow retries are distinct from native model-loop iterations and from tool replay. Outpost execution follows a different ownership path. Workflow configuration, attempt interruption and terminal handling.

Native intent recovery inspects what is known before another model call. A settled intent redelivers its stored payload. An unclaimed intent can execute through a conditional claim. A running intent first waits for an existing result; rerunning then requires both stored and current replay policy to permit it. Otherwise it records an interrupted outcome. Delegation receives special treatment rather than blindly spawning another child. Recovery decisions.

Cancellation is also a recovery input: pending or running intents can become cancelled, while an already settled result is still redelivered. This preserves known outcomes instead of hiding completed work behind a cancellation label. It does not cancel the original external executor or reverse its effects. Recovery contract.

Builder exercise: test uncertain outcomes

Use a scripted model and a fake issue service. Give the service an optional idempotency key and a lookup endpoint. Inject failures before dispatch, after issue creation, after result storage, and during retry backoff. No real model or paid service is needed.

Assert that stored results are redelivered without another mutation; uncertain mutations trigger lookup or an explicit unknown result; and retries reuse the external idempotency key. Repeat with a newly generated model call ID to expose deduplication that only works within one response. Cancel during backoff and after the service commits, verifying that cancellation prevents future attempts without claiming the issue disappeared.

Track model attempts, tool attempts, and workflow attempts separately under an overall deadline or spending allowance. Record the retry reason, scope, delay, policy, and final outcome. See testing and evaluation for deterministic fault injection, runaway agents for budget boundaries, and persistence and recovery for the storage contract.

Conversation forking: a new future from selected history

A conversation fork gives selected history a new continuation. A developer might preserve a working implementation, fork from the design discussion, and explore a different approach. The harness must decide exactly what the new conversation inherits: messages, tool results, summaries, settings, permissions, and possibly a provider's own session identity.

The important limit is that a conversation fork is not a rollback of the world. Suppose the original agent edited a file and sent a message. Forking from before those actions can remove them from the new model's context, but the file remains edited and the message remains sent. A separate worktree, filesystem snapshot, or compensating operation is needed if the experiment requires different external state.

This page compares the pinned Codex, pi, Goose, and Sidekick implementations. Sidekick links require access to its private repository. Source inspection establishes the cited mechanisms, not a guarantee about every frontend or external provider.

Four operations that look similar

A child can inherit context without being a user-facing fork. It usually adds ownership, cancellation, and result-delivery obligations. A fork can also exist without running anything: creating its history does not necessarily submit the next prompt. Keep those lifecycle decisions separate in APIs and UI labels.

Codex: explicit snapshot semantics

Codex exposes two useful ForkSnapshot choices. TruncateBeforeNthUserMessage selects a prefix before a user-message boundary; when its index is beyond the available history, the active unfinished turn is dropped rather than accidentally copied as complete. Interrupted instead retains the current persisted history and, when it ends mid-turn, appends an interruption boundary. This makes the cut policy explicit rather than treating an arbitrary message slice as a valid completed conversation. Snapshot contract Snapshot construction

The new thread has its own identity and records source lineage. There are copied-history and reference-backed paths. A prepared referenced fork supplies frozen source history and model context, while the manager supplies fork-specific startup instructions and configuration. “Same conversation so far” therefore does not mean “reuse the parent's live callbacks and runtime object.” Fork startup Store preparation

Tool history introduces another invariant: a model-facing call needs a corresponding output, and an output must have a matching call unless it names an external tool event. Codex's history normalization fills missing call outputs and removes orphaned outputs. An interrupted fork should preserve uncertainty, not manufacture evidence that an unfinished tool succeeded. The referenced fork's model-context boundary also matters after compaction: visible history and the context used for the next request are related views, not necessarily identical lists. Normalization

pi: a tree leaf and a new session file are different choices

pi's session manager stores an append-only tree. branch moves the active leaf; future entries become children of that node without deleting the abandoned path. branchWithSummary additionally inserts a summary of that path. createBranchedSession performs a different operation: it extracts the root-to-leaf path into a new session, records parent-session provenance, recreates labels, and rechains retained entries so label removal does not orphan descendants. It also repairs affected compaction boundary references. Branch and extraction

The runtime's fork defaults to a position before a selected user entry and returns that user's text for editing. An explicit at position retains the selected entry. It tears down the current runtime and creates the replacement around the selected session manager, which is important because extensions and other session-bound objects must belong to the new session. Runtime replacement

Context reconstruction follows the chosen path and applies its latest compaction checkpoint. Model and thinking settings are recovered from entries on that path. Tree navigation can summarize abandoned work, refuses navigation during active streaming or compaction, refreshes the finalized projection, and restores transcript tools. This is richer than copying chat bubbles: context, metadata, and the tool surface must agree on the selected branch. It does not restore files changed by those earlier tools. Context projection Settings Tree navigation

Goose: copy, optional truncation, and provider identity

Goose's ACP handle_fork_session validates the effective working directory before copying. It creates another session, optionally truncates conversation history using the conversationBefore timestamp, activates the new session with the requested directory and MCP servers, and prepares its agent. Session copying retains settings including provider/model configuration, extensions, recipe values, system-prompt settings, and container configuration. A new session identifier therefore does not imply a fresh environment. ACP fork Copy settings

The timestamp path deletes messages at or after the cutoff; a separate message-ID truncation method can distinguish messages sharing a timestamp. These boundaries deserve care around tool request/result pairs. This review did not establish that the ACP timestamp cut itself repairs every possible pair or is equivalent to a user-turn boundary. A caller should choose coherent cut points rather than rely on a timestamp alone to express conversation semantics. Truncation methods

There is an additional identity boundary: after forking, Goose calls resume_saved_provider_session, which finds the latest saved session identifier for the selected provider and invokes provider.resume. A copied Goose session may therefore ask an external provider to resume an existing provider session. Whether that provider independently forks, shares state, or rejects the request is provider-specific and was not audited here. New local identity is not proof of new remote identity. Provider resume

Sidekick: scoped branches without assuming a general fork API

The inspected Sidekick implementation establishes branches used for delegation and deliberation, not an arbitrary user-facing “fork this conversation at message X” API. ContextBuilderService fetches main-thread messages when parent_tool_use_id is absent, or messages scoped by a parent identifier when it is present. Deliberation uses a synthetic branch identifier inside the same conversation for draft and review passes. These are execution and context scopes, not evidence of a cloned top-level conversation. Scoped context Deliberation ownership

When synthesis incorporates a deliberation branch, _fold_deliberation_branch includes draft text as the agent's prior assistant content and review text as attributed user content. Tool traffic does not cross that boundary. This is a deliberate projection that avoids merging unrelated tool-call protocols into the main conversation. It is also different from a full transcript fork or a compaction summary of an abandoned user branch. Branch folding

Outpost can resume a wrapped agent's own session, but that external runtime's fork facilities should not automatically be credited to Sidekick's native harness. A current general-purpose conversation-fork contract remains unestablished by this review.

A fork contract worth specifying

For a new harness, return a fork record containing the new identity, source identity and revision, cut boundary, inherited settings, context-checkpoint reference, and execution-environment choice. Freeze the source revision before assembling the copy so simultaneous appends cannot produce a mixed snapshot. Decide how incomplete tool calls appear, and preserve their uncertain outcomes.

A concrete trace should verify: original history contains a file edit; the fork cuts before that edit; the fork's prompt omits it; both sessions still see the edited file when sharing a workspace. Then repeat with an isolated workspace and verify that only the filesystem snapshot changes that result. Include compaction, duplicate timestamps, cancelled calls, and external-provider resume metadata in the trace.

Summaries deserve provenance too: identify the branch they summarize and keep the original history available for inspection. Finally, do not silently inherit stale approvals merely because a message was copied. Re-evaluate execution policy for the new environment and make the fork's next action explicit. See compaction, multi-agent delegation, and persistence and recovery for adjacent state boundaries.

Observability should describe these boundaries

Useful traces distinguish model latency, admission wait, tool execution, approval wait, compaction, persistence, and user-input admission. A single run-duration counter cannot explain which layer was slow or whether “done” means model completion, tool completion, session settlement, or committed storage.

Pi provides a concrete example: an ordinary tool's duration measures tool.execute, excluding preflight and post-tool hooks. Its code-mode nested-call timer surrounds ctx.executeTool, which includes the nested pipeline. The numbers describe different intervals even within one harness. Compare like intervals before attributing a difference to performance. Pi tool timing, pi nested-call timing.

Sidekick: visible does not always mean committed

Sidekick separates ephemeral text/progress events, persisted conversation messages, and durable turn outcomes. Its session reducer describes effects; the executor writes message rows into the caller's database transaction and can then broadcast them before that transaction commits. A user may therefore see an update that a later rollback removes from durable history. Atomic sequence numbers coordinate concurrent writers, but receiving a number is not proof of commit. Explicit deferred-broadcast paths provide a stronger acknowledgement boundary. Effect execution, dispatch. Sidekick's source is private; these pinned links require repository access.

Native turn completion has a different contract: a completion event projects an already committed settlement. It identifies the turn and outcome; delegated-child completion also carries parent identity, so it does not stand in for the parent's completion. Clients can reconstruct the settlement metadata from durable state if the live notification is missed. This contract belongs to Sidekick's orchestration layer; a wrapped external harness's own stream events should not automatically receive the same meaning. Completion projection.

For example, a worker can emit text, fail, and be replaced while the same turn remains active. Monitoring should correlate the conversation, turn, worker attempt, and tool call separately. Record whether an observation means “streamed,” “committed,” or “settled”; a single “agent finished” timestamp would hide the recovery boundary. Attempt replacement.

The codebase chapters identify concrete events and tests for each implementation. The builder's lab supplies deterministic scenarios for these boundaries. Together they provide a more useful starting point than a checklist of feature names.

Testing and evaluating a harness

A model can produce a convincing answer while the harness loses a queued message, executes a tool twice, or records an interrupted operation as completed. Testing the final prose alone misses these failures. Start by checking the machinery with scripted model responses, then evaluate whether real models use that machinery effectively.

Suppose a model requests two writes. An approval hook cancels during preparation of the second call. Did either write execute? Does every call receive a terminal result? Does the next request contain a valid conversation? Those are harness contracts: properties that should hold regardless of how capable the model is.

This page examines tests and evaluation infrastructure in the pinned Codex, pi, Goose, and Sidekick snapshots. Sidekick source links are private and require repository access. The tests were inspected, not executed; their assertions describe intended contracts, not a passing-suite or production-reliability claim.

Three different questions

A fake provider can prove that steering reaches the next request. It cannot prove that a model follows that steering. A benchmark can reveal a lower success rate without identifying whether the cause is the prompt, tools, provider adapter, or execution environment. These layers complement each other; neither test count nor one aggregate score measures harness quality by itself.

What the four codebases demonstrate

Codex: control the stream and inspect the next request

Codex's test helpers serve scripted server-sent events, capture outgoing requests, and expose request fields such as instructions, tools, and call outputs. A separate streaming server gates individual chunks with explicit signals. Tests can therefore pause at a known boundary, inject an action, and inspect what the harness subsequently sends. This is more precise than hoping a live provider responds slowly enough to reproduce a race. Request capture Scripted sequence Gated delivery

The cancellation tests distinguish admission from completion. One holds the execution gate and cancels before admission, asserting timing reports no execution start. Another allows a handler to finish, pauses its lifecycle notification, then cancels: the successful output and completed lifecycle must survive. Cancellation should not rewrite an already completed action into a fictional failure. Cancellation contracts

Tool-search tests also exercise catalog changes: reuse for unchanged handlers, rebuilding for changed registry inputs, and removal from the deferred catalog when exposure becomes direct. This is a useful pattern for testing tool negotiation: assert the actual resulting catalog, not merely that an update method ran. These are catalog-cache tests, not measurements of provider prompt-cache hits. Catalog tests

pi: script the provider while exercising the session

pi's coding-agent test harness registers its faux provider, constructs an actual AgentSession, and allows controlled tools, extensions, settings, and session storage. Faux responses can be functions of the supplied context, making it possible to fail a test when the next request lacks expected input. Session fixture

The two-write example is a real regression test: the second preflight aborts, neither tool executes, result hooks do not run, and both calls receive matching terminal error events and aborted results. Another test injects a custom message during tool execution and checks ordering in agent state, session entries, emitted events, and model history. A steering test pauses a tool, queues user input, then verifies the next faux response sees it. Preflight regression Message-order regression Steering boundary

“Faux” does not mean every byte is deterministic. Its stream can split content into randomized chunks, and its token/cache accounting is an estimate built from text length and common prefixes. Builders should fix or normalize randomness when asserting exact traces, and avoid treating simulated usage as provider economics. Synthetic usage and chunks

pi also has separate real-model documentation evaluations. Its documented protocol pairs runs with and without documentation in isolated containers using the same staged packages. The report code rejects incomplete or unscored pairs and withholds headline pass rates when pairs are blocked. This separates infrastructure failures from low task scores instead of silently dropping inconvenient runs. Evaluation protocol Pair validation Headline calculation

Goose: reconstruct between state-machine steps

Goose's pipeline tests script a dummy provider and rebuild the pipeline after each applied step. A tool-turn test asserts two inference calls, the call/result exchange, the final reply, and successful continuation after reconstruction. Other assertions check restored model, mode, recipe, tools, and isolation between sessions. Reconstruction tests

The fixture exposes a crucial limitation: reconstruction shares the session manager, dummy API, scheduler, and retained extension lease. It tests reconstruction from stored session state within that controlled environment; it does not simulate every consequence of losing the whole process or remote connection. Read the fixture as carefully as the test name. Reconstruction implementation

Goose separately includes Harbor benchmark tooling. Its adapter uploads a selected local binary and configuration into a task environment, creates a recipe, and captures streaming JSON output. That makes the executable and extension configuration explicit experimental inputs. The presence of benchmark tooling establishes a measurement mechanism, not a comparable score for these four pinned snapshots. Benchmark adapter

Sidekick: inject failure around durable effects

Sidekick's intent-contract integration tests use the real workflow task body, native loop, and PostgreSQL, while faking the model stream, Redis transport, and selected executors. They count simulated tool effects and inject worker-death exceptions at specific boundaries. These are stronger persistence checks than an in-memory transcript assertion, but are not literal process-kill tests of a deployed cluster. Fixture boundaries

One scenario crashes after both a replay-safe search and an unsafe send have executed, before their outcomes settle. On redispatch, assertions expect the search count to reach two, the send count to remain one, and the unsafe result to explain its uncertain outcome. A second scenario fails before execution; its intended record permits the previously unstarted call to run. This tests the distinction between “never started” and “may have completed.” Crash-window assertions

Ownership tests separately check refusal when a heartbeat is fresh, takeover when the cluster attempt is stale, and rejection of heartbeats from superseded attempts. These contracts matter in event-driven execution, where a valid conversation history alone does not establish which worker may act. Ownership assertions

Build a failure matrix before a leaderboard

The following is a proposed exercise, not a claim that every repository tests every row. Use a scripted provider and a counted, reversible tool; retain requests, events, durable records, and execution counts.

Use explicit barriers to trigger each transition rather than arbitrary sleeps. Then deliberately break one invariant and confirm the test catches it. Recording a trace is useful only if assertions inspect the contract it is supposed to protect. Replaying a recorded provider response tests a known response shape; it cannot establish that the live provider still behaves the same way.

Finally, add a small real-model task set. Grade observable outcomes, such as a passing parser fixture or the correct structured artifact, independently of the model's claim of success. Pin the harness commit, model identifier, effort, prompt, tools, environment, and budgets. Change one factor per comparison, repeat trials, retain failures and timeouts, and report success alongside cost and duration. Keep missing telemetry distinct from zero. Use any discovered protocol failure to create a deterministic regression; use genuine strategy failures to improve the behavioral evaluation set.

Continue with provider adaptation and retries and failure semantics for the boundaries to test, then steering and cancellation, persistence and recovery, and observability for their surrounding contracts.

Build a harness by making its contracts explicit

The goal of this lab is a small runtime whose behavior you can explain. The pseudocode below is a proposed design, not an extracted implementation or production-ready library. The codebase chapters supply concrete implementations of the tradeoffs.

Start with the smallest observable loop

run(input, session, cancellation):
    record input
    repeat while run budget remains:
        stop if cancelled
        context = prepare_context(session)
        response = request_model(context, tool_schemas, cancellation)
        record completed response

        if response failed:
            return a classified failure

        if response contains tool calls:
            for call in response.tool_calls:
                resolve name and validate arguments
                decision = authorize(call, current_policy)
                result = execute_or_record_denial(call, decision, cancellation)
                record result with call.id

        input = take_steering_at_defined_boundary()
        if input exists:
            record input
            continue

        if response has tool calls:
            continue

        input = take_follow_up()
        if input exists:
            record input
            continue

        return completed

    return budget_exhausted

Use a fake model and fake tools first. A fixed sequence of model responses is enough to test the runtime. It isolates loop bugs from provider behavior and costs no inference tokens.

Define the run budget independently of whether the model appears to be making progress. The runaway agents topic compares hard limits with repetition detection; tools explains how to choose the advertised catalog before making a request.

Exercise 1: maintain call/result identity

Have the fake model request two tools, one with an unknown name. Record the successful result and the unknown-tool error against their original IDs. The next request should contain a valid representation of both outcomes.

Completion criterion: no orphan tool calls; a tool error reaches the model as a tool outcome; malformed requests cannot dispatch an unintended function.

The provider adaptation topic explains how streaming and provider-specific history formats complicate this contract. Include incomplete streams and a provider switch in the exercise.

Further reading: the tool validation and execution sections in pi, Codex, and Goose.

Add policies before expanding the tool surface

Expose a read-only in-memory file tool first. Add file writes only after defining which paths are permitted. Add process execution after defining environment, working directory, output limits, timeout, and cancellation behavior.

Exercise 2: deny a valid request

Send a structurally valid write request for a disallowed path. The tool executor must never run. Feed the denial back as a result the model can respond to.

Completion criterion: validation and authorization have separate tests; a denied action is observable; a tool's “read-only” label is not accepted as operating-system enforcement.

The necessary part is an actual boundary around effects. A classifier, remembered approvals, or a policy language is optional complexity until your application needs it.

Define interruption before adding parallelism

Cancellation requests that active work stop and must reach cooperating tools. It does not undo completed effects. Steering supplies new instructions. They should not share an ambiguous “interrupt” operation.

Exercise 3: steer during a slow tool

Run a tool that waits for a test-controlled signal. Queue “Use the other file instead.” Verify whether the current tool completes, whether other calls in its batch start, and when the next model request sees the new message.

Completion criterion: tests assert the chosen boundary; the UI describes that behavior accurately; an explicit cancel reaches the tool independently of steering.

Exercise 4: reverse tool completion order

Let the model request reads A and B. Complete B first. Decide whether events and persisted results use completion order or request order, then assert both independently. Repeat with two writes to the same file.

Completion criterion: results retain IDs; transcript order is intentional; dependent mutations are serialized or rejected. A faster happy path is not enough to validate a scheduler.

Make context preparation a visible operation

Exercise 5: compact around a tool boundary

Create a transcript close to your context budget. Include a tool request, its result, and a later instruction correcting the task. Produce a summary and retain a recent tail.

Completion criterion: the next provider request is structurally valid; the correction survives; the original history remains inspectable according to your storage policy; failure to summarize has an explicit outcome. Test a summary that grows larger than expected.

Treat context estimates as estimates. Reserve space for the model's response and tool results. When the provider reports overflow, make recovery bounded; repeatedly summarizing an already tiny context can otherwise loop forever.

Also test large tool results before they enter the request and prompt caching when instructions or tool schemas change. Keeping a request small and keeping its prefix stable are separate goals.

Separate resume from safe replay

Exercise 6: crash after an external effect

Use a fake counter service. Execute an increment, then simulate a crash before saving its result. Resume the session.

Completion criterion: the runtime either knows the operation is safe to retry, reconciles its external status, or reports an interrupted/uncertain outcome. It must not claim exactly-once behavior merely because the transcript is durable.

Record operation IDs, statuses, and replay policy before needing recovery. For an application with external side effects, this is more consequential than adding another model provider.

Use retries and failure semantics to distinguish retrying a model request, re-delivering a saved result, and re-executing a tool whose outcome is uncertain.

When adding conversation forking, define which history and configuration the new branch inherits. A branch in conversation history does not restore files or undo those external effects.

Extend the harness without bypassing its rules

Exercise 7: put the same tool behind MCP and code mode

Model Context Protocol (MCP) lets a server advertise tools and execute calls over a defined interface. Start with one fake server exposing a read and a write. Then expose a code execution tool whose program calls those tools.

Completion criterion: direct calls and code-generated calls pass through the same policy checks, tracing, cancellation, and result handling. A denied write stays denied when wrapped in a script. Tool discovery is scoped to the current configuration and access rights.

Test errors at three layers: the script fails, an MCP call fails, and the connection fails after the server may have acted. These are not interchangeable retry cases.

Add another participant deliberately

Exercise 8: two clients, one active run

Connect clients A and B. A starts a run; B submits input while A's tool is active. Have both try to approve the same pending action. Disconnect A, then reconnect with an event cursor.

Completion criterion: the system defines who can steer and approve, detects stale run IDs, avoids duplicate decisions, and reconciles history with live events. Presence indicators and shared cursors are optional UI; authority and consistency are runtime concerns.

Exercise 9: a child agent with a bounded task

Delegate one independent read-only investigation. Give the child its own context and an explicit result contract. Cancel the parent while the child is active.

Completion criterion: you know whether cancellation propagates, whether the child can mutate shared files, how its tools are authorized, and how much of its output returns to the parent. More agents do not remove the need for ownership of shared state.

Evaluate behavior, then task quality

Keep two test collections. Runtime tests check invariants using fake providers, clocks, tools, and storage failures. Task evaluations measure whether a real model solves representative problems under the chosen harness configuration.

The testing and evaluation topic compares actual testing infrastructure in the four source snapshots and provides a controlled failure scenario to adapt for your harness.

For fair task comparisons, record model and version, prompts, tool schemas, permissions, initial files, context limits, retries, budgets, and result criteria. Repeat runs and report variability. This resource performs source analysis; it does not provide task-quality, cost, or speed benchmarks.

A useful first milestone is not “the agent solved one task.” It is “the runtime reliably represents completion, denial, failure, cancellation, and uncertain recovery, and I can inspect why it chose each state.”

Evidence and scope

This resource compares commit-pinned source checkouts. Codex, pi, and Goose were inspected on 18 October 2026; Sidekick was added on 19 October 2026. It explains implementation choices; it does not rank products or measure task-solving quality.

Sidekick was reviewed from the supplied local checkout. Its source links require repository access; public readers may receive a GitHub 404. This guide publishes explanations and citations, not a copy of its private source. Its native model/tool runtime and its Outpost adapters for external agents are assessed separately. Hosting another harness does not establish that harness's capabilities as native Sidekick behavior.

These are source snapshots, not an assertion about the latest published release. A feature present in a checkout may be unreleased, optional, experimental, or used only by a particular frontend.

How the research was done

Research agents investigated the repositories using local source, documentation, and test definitions. Sidekick's review was divided across execution, coordination, and tool/provider boundaries. The coordinating agent synthesized a common vocabulary and compared the findings. Each chapter contains an evidence matrix and a source reading path. Structured feature records power the website's comparison table.

Implementation claims link to immutable GitHub commit URLs, usually with line anchors and named symbols. Repository documentation explains intended behavior; implementation code establishes the paths inspected; test code provides evidence of intended invariants. Reading a test is not the same as running it.

The example repair traces and interactive walkthrough are educational simulations. They do not call the harnesses, execute tools, or report observed performance. The builder's exercises describe a proposed runtime, not a single design shared by all four projects.

How to read support labels

These labels are descriptions, not scores. A cell can span several layers, so its text and citations take precedence over its compact label. For example, a library can support replicated state without its terminal application providing a collaborative workspace.

Fair comparison rules

Compare the same boundary: model request with model request, persisted session with persisted session, and application-level policy with application-level policy. Do not equate permission prompts with operating-system isolation, MCP support with arbitrary MCP server safety, or saved history with exactly-once execution.

Separate the reusable runtime from its application shell. Codex's public repository does not establish every proprietary desktop or hosted-service behavior. Pi's newer packages do not automatically describe the coding-agent's default runtime. Goose's shared state machine and application-specific operations are different layers.

Negative findings are bounded to the inspected code paths. “No full collaborative room established” means the research found specific sharing primitives without establishing the complete product; it does not claim such a product cannot be built.

Limits

The research did not run the harnesses against live providers or execute their full test suites. It did not audit every tool implementation, platform-specific sandbox, extension, or deployment. It provides no latency, cost, reliability, or model-quality rankings. Website checks are separate from harness validation and are recorded in the project's verification report.

Source behavior can depend on flags, policy, model capabilities, frontend, and environment. For decisions that depend on one setting or deployment, follow the linked code and add a focused runtime test in that environment.

Revisit with new evidence

Keep old snapshots intact. Add a new commit-pinned chapter and fact set, check changes along the same comparison axes, and document whether a difference is a real behavior change, a newly discovered fact, or a correction to this guide. See the maintenance workflow.

Add a harness or revisit a release

The content is the project. The website is a generated reading surface over that content. There are no harness-specific branches in the comparison renderer.

Files and ownership

Add a harness

1. Identify the repository, commit, release tag if known, and precise runtime/application scope. Do not use a moving branch URL as evidence.
2. Create a new directory under snapshots/<harness-id>/<short-sha>/. Copy the chapter template and write chapter.md plus facts.json there.
3. Research the same axes as the existing harnesses. Use unknown with a bounded explanation when evidence is missing. Do not copy a competitor's behavior into a blank cell.
4. Add a harness record to registry.json, set current to its snapshot ID, and add the snapshot's commit, review date, label, chapter path, and facts path. Keep IDs lowercase and stable.
5. Review claims and source links. Update the guide where the new harness changes a comparison or teaches a distinct design.
6. Run validation and render the site. The new harness appears in navigation, snapshot selectors, and comparison columns from the registry.

A fact record has this shape:

{
  "context": {
    "summary": "A short description of the observed behavior.",
    "detail": "Trigger, state transition, scope, and important qualification.",
    "availability": "core",
    "evidence": [
      {
        "label": "Source symbol and path",
        "url": "https://github.com/OWNER/REPO/blob/FULL_COMMIT/path/to/file#L10-L40"
      }
    ]
  }
}

Include every axis from the registry. Allowed availability labels are core, optional, library, example, partial, and unknown. Established claims need evidence. An unknown finding may have an empty evidence array only when its text explains the bounded inspection scope. Cite the source path that owns the behavior, not just a marketing description.

Revisit a new release

Create a new snapshot; preserve the previous files and metadata. Resolve the release tag to a full commit and record both. A review date is not a release date.

Start with a diff between the old and new commit for the source paths in the evidence records. Then read the changed owning functions and their callers in full. A path that did not change can still behave differently because a caller, default, feature gate, or provider adapter changed.

For each axis, answer:

• Did runtime behavior change, or only an implementation detail?
• Did the default or feature gate change?
• Did responsibility move between the core, application, extension, and provider?
• Do cancellation, context, approval, and persistence still compose the same way?
• Is a formerly missing capability now established, or was the old comparison incomplete?

Write changes.md beside the new snapshot with four categories: behavior changes, newly established evidence, documentation corrections, and still unresolved. Include old/new source references. Add the new entry to snapshots and update current only after review. The website keeps older snapshots selectable; its release comparison shows record changes for a selected harness. Text differences are a review aid, not proof of a semantic behavior change.

Keep the comparison fair

Do not give one project credit for an optional SDK while evaluating another only by its default terminal application. Use the detail field to identify layers. A source checkout with a feature is not proof that a released binary enables it.

Avoid boolean cells for nuanced behavior. “MCP supported” is less useful than the discovery path, transport, tool catalog lifetime, nested-call policy boundary, and error handling. “Multi-player” should separately evaluate client attachment, authorship, active-run arbitration, event distribution, and approval authority.

Add a guide topic

Give each topic a Markdown file and a page record in registry.json. Set parent to the overview page's ID to place it in that page's collapsible submenu. Keep child pages immediately after their overview in registry order so the complete guide follows the same reading order. Use a short title for navigation and a descriptive heading in the Markdown file.

The overview uses layout: "topic-overview"; its top-level Markdown list renders as linked topic cards. Add the new topic's link and description there. Routes, downloads, and submenu entries are generated from the registry. A topic route automatically expands its parent's submenu; the parent's title opens the overview independently of the disclosure button.

Verify and regenerate

From the comparison project directory:

cd website
npm install --ignore-scripts
cd ..
node scripts/validate.mjs
node scripts/validate.mjs --source-root /absolute/path/to/parent-of-checkouts
node --test scripts/validate.test.mjs website/render.test.mjs
node website/render.mjs
python3 -m http.server 4173 --directory website/dist --bind 127.0.0.1

The second validator command assumes checkout directories match harness IDs (codex, pi, goose, sidekick). When repositories live under different parents, create a temporary directory with one symlink per harness ID pointing to its checkout, and pass that directory as --source-root. It uses git show against the pinned commit, so a changed checkout branch does not silently change the evidence being checked. Missing repositories or objects fail source validation; obtain the intended source explicitly before retrying. Private source citations require an authorized local checkout; a public GitHub 404 does not establish that a private citation is invalid.

The generated site opens directly from dist/index.html as well as through a static server. It uses no live provider calls or runtime server state. Check navigation, matrix filters, snapshot selection, the learning walkthrough, deep links, and a narrow viewport after UI changes. Re-render after content changes so the guide and website agree.

Keep package versions exact and refresh the website's own lockfile with dependency changes. Do not install or build the compared harnesses merely to regenerate this resource. The host repository's required checks remain applicable to code edits.

Suggested research delegation

Assign one researcher per harness, each owning its new snapshot directory. Ask a second researcher to challenge claims involving authority, replay, concurrency, and default settings. The coordinator owns cross-project vocabulary and synthesis. Verify citations mechanically and review semantics manually: a valid line number alone does not establish a claim.

Codex: a turn engine with explicit execution boundaries

Snapshot: 2026-10-18, local repository https://github.com/openai/codex.git, commit 8c076942cb70fba8e5c4732c23f1faabc4aed484. This is a source-based account of that revision, not a claim about every Codex release or the proprietary desktop/cloud implementation. The checkout was clean when inspected. No builds, tests, model calls, or example commands were executed for this research.

Codex is useful to study when a small “call model, execute tools, repeat” loop has grown into a system that must survive interruption, permissions, changing tool catalogs, long histories, and multiple clients. Its distinctive lesson is how much state belongs around that loop: a coherent request snapshot, execution policy, event delivery, and persistence checkpoints.

First distinguish the layers and clocks

The Rust core implements conversation and execution behavior. Frontends present that behavior; the app-server accepts thread/turn requests and distributes events. At this revision even the noninteractive codex exec path starts an InProcessAppServerClient. That is a concrete integration seam, not evidence that every frontend uses an identical transport. C1

Three units of work matter:

• A thread supplies the continuing conversation identity and resumable history.
• A user turn is the running task that handles input and emits lifecycle events. RegularTask::run can invoke run_turn again if input remains pending.
• A sampling request asks the model for one response stream. run_turn can make many such requests as tools return, steering arrives, hooks continue work, or context changes. C2

Consequently, counting API requests is not counting turns. A single visible “fix this bug” turn might contain reading, editing, testing, another edit, and a final answer.

Scroll sideways to explore the diagram →

The diagram is a reading map, not a complete dependency graph. Runtime policy differs by tool: command and patch execution use approval/sandbox orchestration; remote MCP execution is not implied to share the local process sandbox.

The turn lifecycle

run_turn prepares context before the first request: it handles pending asynchronous hook output, compaction, relevant MCP readiness, skills/plugins, context changes, and accepted input. Each sampling step captures a StepContext, so tools execute against the tools, model settings, and environment associated with that request. C3

The following pseudocode omits lifecycle bookkeeping but preserves the important loop boundaries:

RegularTask:
  emit TurnStarted; run startup contributors
  repeat:
    run_turn:
      prepare/compact context; record accepted input
      repeat:
        admit pending input at an allowed boundary
        capture coherent step; construct model prompt
        stream one model response, with transport retries
        dispatch completed tool calls as they arrive
        collect tool results into history in call order
        process asynchronous hook output
        if continuation is needed: compact/roll context if required
        otherwise run Stop hooks
        if Stop hooks request more work: continue
        otherwise finish this run_turn
    stop on terminal error
  until no pending input remains

“No more tool calls” is insufficient as a stop rule. Pending input can require continuation; a response can explicitly request a follow-up; Stop hooks can contribute new work. Conversely, an error can terminate the task without replaying queued input. The model stream must reach its completed event; a stream ending early is an error. C2

An illustrative read → edit → test → finish trace

This example was not executed. Tool availability, approvals, names, and command choice depend on configuration.

The harness supplies execution and feedback; it does not mechanically insert a test after every patch. That sequence is the model's chosen workflow in this illustration. Patch verification and the freeform specification are explicit code paths. C5

Tools: exposure, execution, and ordering

ToolRouter distinguishes tools registered internally from tools actually exposed in the current step. Exposure can be direct, deferred through search, or available to Code Mode. It normalizes function, custom/freeform, and search response items into invocations. Unknown tools can produce model-visible errors rather than crashing the whole turn; incompatible internal payloads can be fatal. C6

The registry provides hooks around execution. A pre-tool hook can block or rewrite a request. Post-tool hooks run after successful execution and may reject the result. They cannot reverse an external side effect that already happened. Client-provided dynamic tools form another extension boundary: the handler sends a request outward and waits for the client's result. C7

Concurrency is deliberately selective. ToolCallRuntime waits for readiness before taking an execution lock. Parallel-capable tools acquire a shared read lock; other tools acquire the exclusive write lock. Tasks can start while the model stream continues. MCP tools opt into parallel execution through an explicit flag or their readOnlyHint; this is metadata-based admission, not a proof of purity. C8

Execution completion order and conversation order are different. Sampling collects futures in FuturesOrdered, so history receives results in call order even when later calls finish earlier. Readiness notification can happen earlier than that ordered collection. This design preserves a predictable transcript while permitting useful overlap, at the cost of waiting behind an earlier slow call before later results enter ordered history. C10

Approval is different from a sandbox

An approval policy decides whether an action may proceed or must be reviewed. An OS sandbox constrains what the resulting process can actually access. The tool orchestrator evaluates approval requirements, constructs sandbox attempts, and handles certain sandbox-denial retries. It does not simply rerun every failed command with greater authority. Escalation depends on policy and approval conditions; a Guardian-reviewed authority scope may require a fresh review for escalation. C11

Platform enforcement differs. The core documentation describes macOS Seatbelt, Linux sandbox selection, and Windows support constraints. Unsupported policy combinations can fail closed. These are implementation-specific capabilities, so a comparison should preserve the distinction between requested policy and supported enforcement rather than publish one universal “sandbox: yes” cell. C12

For a harness builder, the essential separation is authorization before execution, bounded execution, then explicit interpretation of failure. Tool descriptions alone do not enforce any of those boundaries.

Long conversations: compaction has several meanings

Codex does not have only one summarization switch. The context-window code considers both an automatic-compaction threshold and the full model context limit; a threshold measured after a reusable prefix does not remove the full-context cap. Model changes can also require compaction, including a move to a smaller window or a changed compaction compatibility hash. C13

There are three important paths:

1. Client-managed summarization. compact.rs makes a model request with a compacting prompt. “Local” here means client-managed orchestration, not offline summarization. On context overflow it can remove older items and retry. The replacement retains selected recent user text plus a summary and reconstructs canonical context; it does not preserve every old item verbatim. C14
2. Provider-supported remote compaction. run_auto_compact selects the remote path when the provider advertises RemoteCompactionSupport::V2; otherwise it can use the client-managed path. Provider support is explicit capability information. C15
3. Token-budget context rollover. The token-budget path installs a fresh context window without running model/server summarization. Calling every window transition “summarization” would misdescribe this behavior. C16

Background compaction is another dimension, not a fourth summary format. It requires its feature and remote-provider capability and is disabled for token-budget mode. A background attempt captures history/model/auth compatibility information. Ready results are candidates adopted at defined checkpoints; they are not allowed to mutate arbitrary current history whenever the request finishes. Shutdown cancels pending work and waits for accounting. C17

These mechanisms protect different invariants: request size, coherent initial instructions, compatible model state, and continuity across history replacement. They also create a learning tradeoff: the model continues from a deliberately lossy representation, so a robust harness must identify what information survives and what must be recoverable from tools or storage.

Steering and multiple clients

Steering means adding direction to an active task. At the app-server boundary, turn/start calls start_or_steer_turn: the result may be a newly started turn or input admitted to the existing turn. The explicit turn/steer route requires a nonempty expected turn ID and can reject stale or incompatible state. Within the loop, pending input is admitted at controlled boundaries; the initial request is protected from being overtaken by later queued input. Some feature-controlled paths can preempt sampling/retry waits to process newly available input. C18

That means “steering supported” should not be read as “every tool stops instantly.” Cancellation, pending-input admission, model-stream preemption, and a completed tool's external effects are different events.

Multiple clients are supported as an app-server primitive. ThreadStateManager maintains sets of subscribed connection IDs per thread and reverse connection/thread mappings. One disconnected client can be removed without deleting the other subscriptions. A thread listener serializes resume-response/history work with subscription so attaching a client does not introduce a gap between the history snapshot and subsequent notifications. C19

These are useful foundations for shared sessions, but a connection is not necessarily a distinct human. In the inspected thread/turn processors, a complete multi-human product—identity attribution, role-based collaboration policy, presence, edit conflict resolution, or ownership transfer—was not established. expectedTurnId is a stale-input guard, not a collaborative document merge algorithm. Multi-agent child threads also have a separate ownership rule: direct app-server user input is disallowed for the relevant spawned-subagent surface. C20

MCP, lazy discovery, and Code Mode

MCP supplies tool catalogs and remote tool execution. Codex translates MCP schemas into its own tool specifications, waits for server readiness, and captures the executed binding and output-policy snapshot so a refreshed catalog cannot silently reinterpret an already admitted call. Search text includes tool descriptions and useful schema/namespace metadata. C9

Lazy discovery reduces how much of that catalog must be exposed initially. ToolSearchHandler indexes deferred entries using BM25, a lexical ranking method. Search returns loadable tool specifications, with namespace coalescing. Its cache is tied to runtime identity and dynamic metadata; it can rebuild when the registry changes. The Code Mode search path filters candidates against that exact step's allowed nested-tool catalog before applying the result limit. C21

Code Mode lets a model express orchestration as JavaScript: call several tools, combine/filter their results, and emit selected output. It is distinct from running a shell script or using a persistent Node REPL. The inspected runtime creates a fresh V8 isolate/context for a cell, rejects static and dynamic imports, and exposes controlled globals such as tools, ALL_TOOLS, text, store, load, and yield/notification helpers. Cross-cell state is explicit through storage rather than arbitrary surviving JavaScript variables. Node filesystem/import APIs were not present in this inspected global surface. C22

For example, conceptually, a cell could await Promise.all(...) for two independent MCP lookups, then emit only a comparison. This reduces model round trips and intermediate text. It does not bypass normal tool handling: nested calls are translated back into typed tool payloads and routed through ToolCallRuntime with the originating StepContext. The delegate retains that snapshot across a yielded cell; recursive invocation of the execution tool is rejected. Long-running cells can yield, be waited on, or be terminated. C23

This architecture separates two costs: catalog selection determines what the model sees, while programmatic orchestration determines how many model turns are needed to combine tool work. Either can be useful independently; feature/configuration gates determine availability.

Persistence, cancellation, and recovery

ThreadStore is an explicit storage-neutral boundary. An append is not automatically a universal durability guarantee. Standard persistence and flush_thread must make queued items durable/readable before returning. TurnStart, SteeredUserInput, and SubagentSpawn permit background persistence subject to later barriers; acknowledging a spawned child requires the documented standard barrier. shutdown_thread flushes; discard_thread may drop pending in-memory writes. C24

Resume also has an ordering contract: the store returns authoritative replay context after acquiring the writer, including writes committed before acquisition. Core reconstruction finds the latest usable compaction checkpoint and replays its surviving suffix, respecting rollback and restoring context/window metadata. A saved UI transcript and the next model prompt are therefore not interchangeable. This review does not establish implementation-level fsync, crash atomicity, or exactly-once external side effects. C25

Tool cancellation preserves already completed lifecycle outcomes. Runtime behavior can wait for handlers designated to finish on cancellation; otherwise it can abort the task and synthesize an aborted tool result. That is bookkeeping for interrupted work, not rollback of filesystem or remote effects. C26

Model transport recovery is separate. Response-stream errors use provider retry limits, retry delays, and supported fallback behavior. WebSocket-to-HTTPS fallback can reset the retry counter. An optional feature permits unbounded connection-failure retries for eligible paths, so “bounded retries” is not universally true across configurations. Context-window and usage-limit failures are handled differently from retryable transport failures. C27

Providers, extensions, and agent teams

The provider interface covers metadata, authentication/recovery, model catalog/routing, and capabilities. The factory supports configured providers and a dedicated Bedrock path; capability declarations decide features such as remote compaction. This is an abstraction boundary with capability differences, not a promise that every provider implements every behavior. C28

Extensibility operates at several levels: Rust tool runtimes, dynamic client tools, MCP tools, skills/plugins contributing instructions or tools, and lifecycle/tool hooks. These extension points have different trust and execution boundaries. Adding a skill is not the same operation as registering a callable runtime. C7

The multi-agent v2 tool surface includes spawn, send-message, follow-up, list, interrupt, and wait. Spawning can inherit history or start without it; spawn cleanup uses a guard to shut down/discard a provisional child on failure. The control graph enforces distinctions such as not interrupting the root/self through the child-control path. These are agent-thread orchestration primitives. They do not, by themselves, prove separate filesystem/worktree isolation for each child. C30

What to test, and what the code already observes

The execution runtime records dispatch wait, handler duration, and total time, with care around nested Code Mode accounting. Registration telemetry distinguishes exposed and hidden/deferred tools. This makes an execution queue delay distinguishable from a slow handler. C31

Inspected tests cover cancellation before admission, preserving a completed lifecycle during cancellation, metadata-sensitive search-cache rebuilding, and coalescing mixed search results. They are useful executable descriptions of contracts; they were read, not run, for this chapter. C32

Three design lessons follow:

1. Capture request state deliberately. A tool must execute under the same relevant settings and catalog used to generate its request. Snapshots and bindings add machinery but prevent time-of-check/time-of-use confusion when input or catalogs change.
2. Separate parallel work from ordered history. Shared/exclusive execution admission plus ordered result insertion provides concurrency without an unpredictable replay transcript. The price is potential waiting behind earlier calls.
3. Name your barriers. Authorization, sandbox enforcement, turn completion, model completion, compaction adoption, and durable persistence are separate boundaries. Explicit contracts make recovery possible; collapsing them into a single “done” flag hides failure modes.

Evidence matrix

All links below are immutable links to the inspected commit. IDs are stable within this research snapshot and can be reused by the comparison matrix.

Suggested source-reading path and limits

Start with tasks/regular.rs, then session/turn.rs. Follow one tool through tools/router.rs, tools/registry.rs, tools/parallel.rs, and its handler. Then read the orchestrator, compaction modules, and thread-store/src/store.rs. Finally trace app-server steering/subscriptions and Code Mode's nested dispatcher. This order keeps the basic feedback loop visible before introducing concurrency and recovery.

This is a focused source review, not a security audit or performance benchmark. Defaults, feature gates, provider capabilities, and frontend configuration affect which paths run. The chapter does not infer proprietary product behavior, remote MCP-server isolation, full multi-human collaboration, or crash-level durability from these interfaces. Revisiting a release should recheck each C-ID against the new pinned commit, especially loop entry points, persistence contracts, tool-exposure rules, and feature-gated compaction/Code Mode behavior.

Pi: a small agent loop inside a configurable coding application

Pi is useful for learning because its core loop is separate from the application around it. A model proposes tool calls; ordinary program code validates and executes them; results become messages; the model runs again. The terminal interface, session history, compaction, and extensions sit around that loop. The important boundary is between deciding what to do, which the model does, and making it happen, which the harness does.

This chapter describes the local earendil-works/pi checkout at commit 482c1381a00d8d00bc81697e4a980c593db6803e, inspected on October 18, 2026. The package manifests identify version 1.3.0. All source links below are pinned to that commit. This is source analysis, not a performance benchmark or a claim about every installed Pi version. The inspected checkout includes built-in MCP integration and parallel tool execution.

The layers and their responsibilities

Scroll sideways to explore the diagram →

Simplified coding-agent reading map. The separate experimental pi-durable runtime is not part of this loop.

The coding application's main composition point is createAgentSession() in sdk.ts. It restores session messages, selects the model and thinking level, constructs Agent, supplies its streaming function, and wraps it in AgentSession. [P1]

Do not merge the two harnesses into one architecture diagram: the coding-agent SDK instantiates Agent, while pi-durable builds on pi-ai and Chord. The latter is explicitly experimental. Its capabilities do not automatically describe the coding CLI's persistence guarantees. [P1, P20]

Follow one request through the loop

Consider an illustrative request: “Fix the empty-input case in src/parser.ts, run the focused parser test, and explain the change.” The following is a possible model trajectory, not a recorded run and not a hardcoded workflow.

1. AgentSession.prompt() processes extension commands and input hooks, expands skills/templates, checks model/auth availability, and starts a run. Agent.prompt() rejects overlapping prompts; an already-running session requires an explicit steering or follow-up choice. [P2]
2. The first provider response requests read({path: "src/parser.ts"}). The loop resolves the tool, prepares and validates its arguments, runs the pre-execution hook, then calls its implementation. The returned file text becomes a toolResult associated with the original call ID. [P3, P4]
3. Another provider request sees that result. The model proposes edit({path: "src/parser.ts", edits: [{oldText: "...", newText: "..."}]}). The edit implementation reads the file, applies replacements against its original content, writes the result, and returns success text plus a diff for the UI. [P6]
4. The model requests bash({command: "node --test test/parser.test.ts"}) in this hypothetical repository. Bash streams progress and returns the exit status/output. A nonzero exit is an error tool result, so the next model response can investigate and try another edit. [P7]
5. Once the model responds without tool calls, the loop checks queued work and continuation hooks. If none request more work, it emits agent_end; the session may still perform retry or settlement processing before agent_settled. [P3, P12]

The loop does not know what a parser fix is, and a final answer does not itself prove that the tests passed. Correctness comes from the actual tool evidence and the model's interpretation of it.

append prompt and changed tool declarations
emit agent_start, turn_start, prompt message events
repeat:
    before subsequent turns: refresh context/model/tools, possibly compact
    append pending steering or follow-up messages
    prepare request
    transform application messages -> provider-compatible messages
    stream assistant response and emit message events
    if response is error or aborted: finish turn and end this core run
    if output hit length limit: fail its tool calls without executing them
    else: validate, authorize through hooks, execute, finalize tool batch
    append tool-result messages
    apply finishTurn decision and emit turn_end
    if finishTurn says end: stop
    if tool continuation or steering exists: repeat
    if follow-up exists: repeat
    if finishTurn requested continuation: repeat once with current context
    otherwise stop
emit agent_end

This condenses runLoop() rather than reproducing every error branch. One core “turn” is one assistant response and its tools; a user request can require many such turns. Tool declarations and prompt changes are recorded in system messages. declareToolChanges() reconciles the executable tool set with the transcript before requests. [P3, P8]

Tools: schemas, concurrency, and useful failure results

A tool has a name, description, parameter schema, and executable function. Tool implementations can also provide display details, structured output for programmatic callers, progress updates, argument preparation, and execution-mode constraints. The loop's provider boundary converts application messages only when it requests another response. UI-only or custom information need not be sent to the model. [P4, P9]

prepareToolCall() performs lookup, optional argument repair, schema validation, then beforeToolCall. Unknown names, invalid arguments, blocked calls, and thrown tool errors become error results the model can observe. Validation can coerce values; it is not merely checking that the original JSON already has the correct types. Truncated assistant output is handled more conservatively: all its tool calls fail without execution because even parseable arguments may be incomplete. [P4, P10]

Parallel execution is the default in this checkout. The runtime first prepares calls sequentially, then starts permitted executions concurrently. Completion events can arrive in completion order, but final transcript results are appended in the assistant's original call order. A globally sequential setting, or any tool marked executionMode: "sequential" in that batch, serializes the whole batch. [P5]

For example, two independent reads can overlap while the conversation remains deterministic about their order. Two edits to the same file require another safeguard: withFileMutationQueue() serializes mutations by resolved file identity, while different files can proceed independently. This is a process-local coordination mechanism, not a lock against unrelated editors or processes. A bash command touching the same file does not automatically participate in the edit queue. [P6]

The default coding tool selection is read, bash, edit, and write. Other built-in tools and extension tools exist, and the SDK supports allowlists and denylists. Read output is bounded and supplies continuation offsets; shell output is bounded and points to a temporary file when truncated. This is context management at the tool boundary, before conversation summarization is necessary. [P1, P7]

Steering, follow-up, and cancellation are different operations

steer() means “include this before the next model response after the current assistant turn's tools finish.” It does not cancel the running request or skip the rest of its tool batch. followUp() means “wait until the agent would otherwise stop.” Both queues default to one message at a time and support draining all queued messages instead. [P11]

Suppose the model already requested two edits when the user types “Only change the parser.” Steering will influence the next model request; those already-issued tools still complete. If the user needs execution to stop, cancellation is the relevant operation. Agent.abort() signals the active AbortController; tool implementations must cooperate. Bash kills its process tree on cancellation or timeout. File edits check cancellation after awaited operations and retain their mutation lock until those operations settle. Cancellation does not undo a write that has already happened. [P6, P7, P11]

AgentSession distinguishes a core run ending from the broader activity settling. Its post-run path can retry, compact and continue, or process extension-generated work. This matters for clients deciding when to enable another input or declare a task finished. [P12]

Approval is policy; sandboxing is containment

The inspected default coding tool path executes local filesystem and shell operations. An always-on per-call approval policy or OS sandbox was not found in that path. The core supplies blocking hooks, and the repository includes a permission-gate example that asks about selected shell patterns. That is a policy decision before execution, not a guarantee about what an approved process can access. [P4, P7, P13]

A separate sandbox example replaces bash operations with @anthropic-ai/sandbox-runtime, applying filesystem/network restrictions to shell execution. Its scope is the overridden bash path; it should not be presented as confinement of every local file tool. Project trust is another boundary: it governs loading project configuration/resources, rather than making individual tools safe. [P13, P14]

Code mode introduces a third boundary. pi-codemode runs JavaScript inside a QuickJS/WebAssembly VM with injected tool functions. The script lacks direct host facilities such as process, modules, and fetch; an injected bash tool still has whatever authority its host implementation gives it. Restricted orchestration code and restricted operating-system commands solve different problems. [P15]

Context and persistence: preserve history, project a smaller request

Long sessions must fit inside a model's context window. Pi estimates current usage from provider token accounting plus newer messages, with message-size estimates when necessary. Automatic compaction triggers when usage exceeds the context window minus reserved tokens; the baseline defaults reserve 16,384 tokens and aim to keep roughly 20,000 recent tokens. Settings and model-specific choices can alter effective behavior. [P16]

Compaction selects a retained tail without cutting at a tool result, summarizes older material, and can separately summarize the prefix of a turn that crosses the cut. It carries file-operation information and rejects summaries that hit the output length limit. The session appends a compaction entry containing the summary, the first retained entry ID, and the current system/tool state. The raw historical entries remain available. [P16, P17]

SessionManager stores an append-only JSONL tree: every entry has an ID and parent ID, and a current leaf chooses the active path. buildSessionProjection() turns that path into the effective messages, applying compaction and later context edits. A context edit can omit an earlier failed response from future model input without erasing the raw failure record. This separates “what happened” from “what the next request should see.” [P17]

Resuming opens a session and reconstructs its context. Branching moves the leaf so new entries grow from an earlier point; creating a branched session copies a selected path into another session file. Optional tree-navigation summaries preserve useful information from the abandoned path. These are conversation operations; the inspected session code does not restore filesystem contents to their historical state. [P18]

The coding session persists completed messages at message_end, after extension/public event dispatch. It is therefore inappropriate to describe its UI stream as a transactional crash-recovery log. The first user/assistant message causes initial file creation, and subsequent records append synchronously. [P12, P18]

Providers, retries, and extensions

The agent accepts a StreamFn; the provider must produce a common event stream and encode request failures as final assistant messages with error or aborted stop reasons. pi-ai's Provider interface owns authentication, model enumeration, and streaming; Models.streamSimple() normalizes the transcript, resolves authentication, and dispatches to the provider. This keeps provider-specific request formation outside the loop. [P9, P19]

Transient provider failures receive bounded exponential-backoff retries at the coding-session layer. The failed attempt stays in raw history but is omitted from subsequent model context. Context overflow takes a separate compact-and-retry path, with a guard against repeated recovery; it is not treated as ordinary throttling. Tool failures generally return to the model rather than automatically rerunning potentially side-effecting operations. [P10, P12]

Extensions can register tools and commands, intercept input/tool calls/results, transform context, participate in turn/settlement boundaries, and supply compaction. Built-in extensions include MCP, tool search, code mode, and llama.cpp integration. MCP uses a standalone pi-mcp client with stdio and Streamable HTTP transports, tool discovery, resources, and authentication handling. MCP tool definitions adapt server schemas and results into the same tool pipeline. [P14, P21]

Subagents require precise labeling. Coding-agent includes an example extension, installed separately, which launches child pi --mode json -p --no-session processes. It supports single, parallel, and chained work, with its own concurrency limits. This is not a default subagent tool in the four-tool coding set. Separately, pi-durable provides owned conversations and tasks, with foreground/background subagent examples. [P22, P23]

MCP through code mode

Code mode lets the model write a short program that searches for tools, calls several, and sends only useful results back to the conversation. For example, a script can discover an MCP search tool, fetch three documents concurrently, then emit just their titles and relevant excerpts. The tool description budgets inline declarations; deferred tools remain discoverable through searchTools, describeTool, describeNamespace, and ALL_TOOLS. This reduces repeated model round trips and oversized tool-result messages, while making the script responsible for choosing what evidence to retain. [P25]

The coding-agent extension is registered but inactive until selected; MCP configuration can activate it when required. Each nested tool call goes through ctx.executeTool, preserving argument validation and tool_call/tool_result hooks. Code mode is therefore another caller of the tool pipeline, not a permission bypass. Tools with output schemas return structured content to scripts. MCP error results can resolve as structured objects, so scripts must inspect isError; other failures reject. Successful script store writes become branch-local session entries. The worker has a 256 MiB VM memory limit; cancellation and optional timeouts stop the script, without undoing completed external effects. [P25]

MCP authentication and transport stay on the host side. Connections negotiate capabilities and discover tools; tool-list changes refresh registrations. Resource read/list operations opt into one retry after a transient HTTP failure. callTool does not opt into that retry, regardless of a tool's read-only annotations, because the operation may already have run. An expired-session response has a separate reconnect-and-retry path because the server did not execute the request. These transport decisions are distinct from approval hooks and server-side authorization. [P21, P27]

The newer durable harness changes the recovery problem

Saving a conversation is insufficient when a process crashes after a tool changes an external system but before its result is saved. Replaying that call could duplicate the change.

pi-durable records tool intent before execution. Recovery reruns a call only when both the stored and current tool replay policies say safe; otherwise it settles an interrupted result explaining that the operation may have partially run. Durable subagent examples use ownership plus stable request IDs to reconnect to an existing child/submission instead of creating duplicates. This is explicit recovery design, not a blanket exactly-once guarantee for arbitrary external effects. [P20, P23]

Its documented storage options include memory, JSONL, SQLite, and Cloudflare Durable Object storage. Chord supplies application/document state infrastructure, while pi-env supplies remote execution environments. These are valuable advanced reading paths, but a first harness can start with the smaller Agent loop. [P20]

Multiplayer means shared humans or clients, not subagents

The coding-agent RPC interface is a long-lived subprocess speaking JSONL over stdin/stdout. It correlates command responses and streams session events for custom clients, but the inspected SDK/RPC path does not establish a turnkey authenticated multi-user session service. Merely sharing a transcript file would not solve concurrent input ownership or conflicting filesystem edits. [P1, P26]

The adjacent libraries offer useful building blocks: Chord exposes objects across workers, browser tabs, or servers, with per-principal variants and observable snapshots/deltas. The experimental durable harness's ConversationViews publishes committed conversation state to multiple observers and exposes a replicated state view. Initial snapshots and observer attachment share the session's serialized read boundary, preventing a missed commit between them. These are replication primitives; applications must still define membership, input arbitration, and permission policy. They should be labeled library support for multiplayer, not advertised as a finished coding-agent multiplayer UI. [P26]

Tests, observation, and three design lessons

Core lifecycle events expose assistant deltas, tool progress, final results, duration, and turn boundaries. Session events add compaction, retry, queue, and settlement state; session statistics aggregate usage across stored history. The repository's faux provider makes ordering and failure cases testable without real model calls. For example, regression tests check that a later preflight abort prevents earlier prepared tools from starting, and that injected custom messages never split a tool-call/result pair. These tests were inspected, not executed for this research. [P12, P24]

1. Keep the loop small, but define the boundaries rigorously. Provider streaming, application context projection, tool execution, and presentation are independently replaceable. The cost is more contracts: when callbacks run, whether failures throw, and which event means truly idle all matter.
2. Concurrency needs transcript rules as well as locks. Completion-order progress gives responsive UI; source-order results preserve a predictable transcript. Same-file queues solve a narrower mutation problem. None of these mechanisms makes dependent tools safe to run concurrently by itself.
3. Choose the persistence guarantee deliberately. An append-only conversation tree supports inspection, compaction and branching. Durable task intent and replay policies address interrupted execution. Adopting the latter adds real complexity and is justified when work must survive process loss.

Evidence matrix and reading path

Begin with P1 → P3 → P4/P5 → P12 → P17. Then explore provider adaptation, extensions, and the separate durable runtime. These claim IDs can be reused in the cross-harness comparison.

Scope limits: the core loop, coding-session orchestration, session projection, compaction and principal coding tools received the deepest inspection. Provider-specific wire formats, the full durable scheduler, all extensions, and OS sandbox enforcement were not audited exhaustively. Extra-package descriptions combine manifest/documentation evidence with the explicitly linked implementation/example paths. No dependencies were installed, real providers called, tests run, or source files modified. Negative findings apply only to the inspected default paths; extensions or host applications can change the behavior.

Goose: a persisted, ordered operation pipeline

Goose is useful for learning how an agent harness grows beyond “call the model, execute tools, repeat.” Its central design is an ordered pipeline of operations over a saved conversation. Approval, tool execution, compaction, retry policy, and model inference each get their own operation. After an operation changes state, Goose saves its effects and starts scanning the pipeline again. This makes control flow explicit, while moving some complexity into message metadata and operation ordering. G1, G2

Snapshot: local goose checkout, clean tracked worktree, commit 34a222e3f2b95d8b4a43482b53c2cc957ecce26d; origin https://github.com/aaif-goose/goose.git. Inspected October 18, 2026. All source links below are pinned to that commit. This chapter analyzes implementation, not benchmark performance. No Goose sessions, builds, or tests were executed.

Start with four layers

Scroll sideways to explore the diagram →

Simplified Goose-managed execution map. Providers that own their own loop follow a different path.

1. Clients: the CLI and Electron desktop provide input, rendering, and permission responses. Desktop sends prompt/cancel/steer requests using Agent Client Protocol (ACP). CLI definitions expose acp over stdio and, with the acp-http feature, serve over HTTP/WebSocket. These are delivery surfaces, not separate model algorithms. G3, G4
2. Goose application: Agent assembles provider, session manager, extensions, hooks, inspection, and steering. Its reply method returns a stream of AgentEvents. G5
3. Reusable engine: goose-agent supplies StateMachine, Operation, and InferenceRunner. The Goose application selects and configures operations. An effect is a requested state change, such as appending a message or changing message visibility. G1, G6
4. Capabilities and storage: providers translate model requests; extension clients implement tools; SessionManager stores conversations and session metadata in SQLite. G7, G8, G9

Do not equate every provider with a remote language-model API. The interface also allows providers that own their conversation context. The Goose builder skips its own full compaction for those providers. ProviderDef::RUNS_OWN_TOOL_LOOP separately identifies integrations handed MCP servers to run their own tool loop. This chapter's detailed trace describes Goose-managed execution. G2, G7, G10

One user request, several model requests

A user turn here means the work initiated by a user message until the harness yields or stops. A model request is one inference within that work. Goose's max_turns mechanism counts contiguous assistant blocks since the kickoff message; it is not a universal measure of user turns, tokens, or elapsed time. Streaming chunks in one assistant block do not individually consume that count. G6, G11

reply_with_state_machine_inner acquires a per-session active-turn guard and saves the user message. stream_state_machine_session resolves provider/model/context limit, builds the machine, and forwards its events. Inside the reusable machine, the first operation that applies wins that iteration. G1, G5

persist incoming message
build the ordered pipeline
repeat:
    session = load persisted session and conversation
    for step in pipeline order:
        stop if cancelled
        if step is inference:
            prepare a session/extension snapshot
            gather every operation's tools and prompt contributions
        result = run step
        if result applies:
            ensure message IDs
            persist result.effects
            break the scan
    if nothing applied, or result yields/cancels: stop
finalize cancellation and repair unanswered tool requests if needed

The concrete order begins with slash-command dispatch, approval, skills, recipes, extension tool execution, remaining-tool handling, subagents, steering, and bang-shell handling. Then come context operations and completion controls, ending with inference. Every restart begins at the front; an operation must recognize from persisted state that its work is already done. Operation::set_message_meta records past actions for this purpose. This is reconstructible control flow, not a claim that external side effects execute exactly once after arbitrary crashes. G1, G2, G6

A read → edit → test → finish trace

This illustrative Unix trace assumes the developer extension and an ordinary model provider. It does not execute commands. At this revision, developer tools are flat names such as shell, edit, write, tree, and read_image; its instructions use shell commands to read text files. A fictional read_file call would misrepresent this particular tool surface. G12

1. User: “Fix the parser's empty-input case and verify it.” The message is saved; inference advertises the currently leased tool definitions.
2. Model: shell({command: "cat src/parser.ts"}). The assistant request gets an ID and an advertised-tool list. The next pipeline scan inspects permission, dispatches the tool, and persists its response under the same request ID. G5, G13
3. The model sees the file and requests edit with path and exact replacement text. The tool description requires an exact, unique match. If approval is required, Goose records an action-required message and marks execution unavailable pending a decision. The outer agent stream waits for confirmation answers before rerunning the machine. G12, G14
4. After editing, the next model request asks shell to run the appropriate focused test. A failed tool result is information returned to the model; it can choose another edit. The harness does not inherently know that this particular test proves the original request correct. G13
5. The model returns a plain final answer. If no recipe/goal/stop-hook operation asks for more work, inference no longer applies and the pipeline yields. An optional recipe success check can reject apparent completion and initiate another attempt. G6, G15

The trace uses separate model requests because the edit depends on reading, and testing depends on the edit. Emitting dependent calls together would undermine that ordering.

Tools: contracts, routing, concurrency

Tools use MCP's Tool representation, including JSON input schemas and annotations. The developer extension generates schemas from Rust parameter types and deserializes arguments into those types at execution. The inference adapter canonicalizes recoverable tool names, coerces arguments using the advertised schema, and records advertised tool names on assistant messages. Execution checks that model-originated requests were advertised. G12, G16, G13

An extension lease is the selected set of extension instances and capabilities for a session snapshot. Its catalog maps a public tool name to its extension and server-side name. A call resolves that entry, builds a context containing session, working directory, and cancellation, and calls the extension's MCP client. Platform extensions run in the agent process; configuration also supports bundled, stdio, and Streamable HTTP extensions. MCP standardizes the contract, not the trustworthiness of the implementation. G8, G17

Concurrency has two stages. ToolExecutionOperation::run prepares dispatches in a loop, then uses select_all to poll their result and notification streams together. Thus tool futures can progress concurrently; this is not a sequential “await each complete result” loop. ToolBatch::record adds responses as observed, deduplicating by request ID. There is no request-order sorting in this operation: result order can differ from request order. Notifications can reach the client before the batch finishes; the final response message is emitted and persisted after the combined stream finishes. Cancellation finalization can preserve results already recorded in the batch. G13

The engineering implication is specific: independent reads can overlap, but the model or caller must preserve dependencies between writes and subsequent tests. The generic execution operation does not infer a filesystem dependency graph. That is a bounded observation about this operation, not a statement about every extension's internal scheduling.

Approval is policy; isolation is execution environment

Permission inspection distinguishes Chat, Auto, Approve, and SmartApprove. In the permission inspector, Auto allows calls; Approve and SmartApprove consult explicit tool permissions. SmartApprove additionally uses read-only annotations and model-based classification for candidates. Other security inspectors can override the permission baseline. A read-only hint or model judgment is a policy input, not an operating-system guarantee. G18, G14

Approval requests and responses are saved conversation records. The wrapper can reconnect to pending confirmations or apply a saved answer, but the original extension lease still matters: when an executable request has no valid retained lease, tool execution returns an “approval expired” response asking for a new request. Therefore “the conversation is persistent” must not be shortened to “every pending authorized action can be replayed after restart.” G5, G13

For execution isolation, the inspected stdio connector supports explicitly running an extension using docker exec when supplied a container; without it, it starts the configured local process. The developer platform extension's ordinary shell builder launches a host shell with the selected working directory. Neither path shown establishes a universal filesystem/network sandbox for all Goose tools. Container configuration is a separate deployment choice, and the stdio container branch should not be advertised as enclosing all in-process platform tools or remote MCP servers. G19, G20

Context and persistence are different views of history

Goose records user visibility and agent visibility separately. The provider receives a projection of the saved conversation, not necessarily the entire transcript. Inference removes unanswered old tool requests from prior turns, repairs provider-facing conversation structure, and merges compatible messages. G21

Compaction can run before inference when usage exceeds the configured threshold, or reactively after a context-length error. The operation bounds repeated reactive attempts; providers owning context bypass Goose's full compaction. compact_messages summarizes the agent-visible history, makes original messages agent-invisible, adds an agent-only summary and continuation instructions, and preserves the current text request for automatic compaction. The user-visible history can therefore remain longer than the model context. Optional tool-pair summarization is separately configured and defaults off in the inspected configuration helper. G22, G23

SessionManager persists messages and session configuration in sessions.db, using SQLite WAL mode. Effects include append/replace, request-metadata patches, visibility changes, recipe/extension state, and usage. Compaction saving updates existing records' agent visibility and inserts new summary records in a transaction rather than blindly deleting concurrent messages. Session persistence supports reconstruction, while streaming output may be visible before its operation's effects have been stored. Approval requests deliberately publish after persistence so an immediate answer can find them. G9, G24, G33

Steering and multiple clients

Steering is guidance submitted while a run is active. Agent::steer adds a message to a session's in-memory queue. SteerOperation drains that queue after tool results or a completed assistant response, emits the messages, invokes prompt-submit hooks, and returns persistence effects. It does not splice instructions into an in-flight model request. Undrained steers are not yet durable transcript entries; the ACP wrapper discards pending steers when clearing the run. G34, G35

For multiple humans or clients, the ACP server shares an ActiveRunRegistry across connection-specific agents. It admits only one prompt run per session; a competing prompt gets an error directing the caller to steer. A second connection can steer the agent owning the active run, supplying expectedRunId to reject stale targeting. The factory contains a test specifically for this cross-connection routing. G35, G36

This establishes shared input arbitration, not a complete multiplayer editing experience. The inspected forward_agent_stream sends notifications through the prompting connection; loading a session replays stored history. These paths do not establish a generic live broadcast subscription, per-human authorship model, or conflict-free shared editor. Treat those as unverified product capabilities, separately from the demonstrated multi-client steering and the separate subagent mechanism. G37, G38

Code mode for MCP: a second tool-calling shape

Goose's optional Code Mode is compiled behind the code-mode feature and registered with default_enabled: false. It exposes list_functions, get_function_details, and execute_javascript. Discovery lists exact callable names and short descriptions; details return original input/output schemas. The model writes async function run() and invokes await tools.call(name, args), using Promise.all for independent calls. The callback catalog is taken from the dispatching extension lease; it excludes app-only tools, MCP resource tools, the code-mode extension itself, and Summon's delegate tool. G39, G40

The model-facing tool list is reduced, but first-class platform tools and MCP resource tools remain directly visible. Thus code mode is not “all tools replaced by one interpreter.” Its practical advantage is letting JavaScript combine/filter results before returning text to the model, reducing intermediate context traffic; this is a design rationale, not a measured speed or token claim. G41

The interpreter uses Boa with an idle module loader and injected tools/console, rather than a Node.js environment. It limits instructions, loop iterations, recursion, output, worker slots, and execution time. Its own documented interface has no ambient filesystem, shell, network, packages, or environment access; external effects go through callable tools. These interpreter restrictions do not confine the external tools themselves. G40, G42

There is an important policy boundary: the outer execute_javascript request follows normal tool approval/execution, while nested dispatch_tool invokes the lease directly. The inspected nested path does not re-enter ToolApprovalOperation or run_pre_tool_hooks for each callback. Therefore do not assume approving one script provides the same per-call review as separate model-issued tool requests; underlying tools may also enforce their own policy. Nested tool errors reject the JavaScript call and can be caught, and cancellation propagates to dispatch tasks. G13, G14, G40

Cancellation, errors, and three distinct retries

The machine races each step against a cancellation token. On cancellation, it asks operations to retain partial output and creates interrupted responses for unanswered tool calls. Inference retains accumulated chunks; tool execution retains completed batch results. The developer shell handles cancellation by killing and waiting for its child, but this does not establish rollback of file edits or cancellation guarantees for every remote tool. G1, G13, G20, G21

Keep these mechanisms separate:

• Transport retry: Goose's provider-stream adapter retries qualifying transient first-item failures before accepting stream output, with backoff and provider retry policy. This branch skips providers owning context. It is not a general replay of partially emitted output. G25
• Empty-response retry: InferenceRunner can retry an empty response up to three times, with special handling after successful tool responses. Exhaustion leaves a marker that completion operations can handle. G21
• Task retry: configured recipe checks can reject a completed answer, run an optional failure command, reset history to the kickoff, and try again within max_retries. Resetting conversation history does not undo files or external side effects. G15

Unrecovered errors eventually reach ExitOnErrorOperation, which yields. The operation pipeline therefore expresses both continuation policy and exit policy; neither should be inferred merely from whether the model emitted text. G26

Extensions, recipes, and subagents

Recipes package instructions/prompt, extensions, provider/model settings, parameters, optional output schema, sub-recipes, and retry configuration. They describe reusable task setup and completion contracts; they are not a second implementation of the base state machine. Skills and hook operations add prompt material or lifecycle behavior around the same pipeline. G2, G27

Foreground subagents have separate persisted sessions linked to the parent. Their saved recipe, provider/model, extension selection, and turn budget are used to prepare execution. The reusable SubagentOperation tracks multiple child update streams with SelectAll, persists delivered outcomes as agent-only parent messages, and records cancelled children so reconstruction does not relaunch them. A child can forward approval or elicitation requests to the parent. G28, G29

The inspected Goose runner refuses to discover new subagents when the parent is already a SubAgent, and validates that a child belongs to the requesting parent. This is a bounded nesting restriction in this runner. These children are foreground work managed by the parent turn; the source explicitly distinguishes this implementation from older background-delegation history. G30

What a builder should borrow

Separate a decision from its durable effect. A permission answer can arrive immediately. Persisting its request before emitting it prevents a race; recording operation notes lets reconstructed control flow recognize completed work. The cost is that metadata becomes part of the state-machine contract and needs migration/reconstruction tests. G24, G6

Separate transcript, model context, and live events. A learner might delete old messages to save tokens, inadvertently destroying the audit trail. Goose's visibility projection and summary records preserve both views. The cost is more invariants: hidden messages, request/result pairing, and compaction must all agree. G21, G23

Make capability ownership explicit. A provider may own context; an extension owns execution; the client owns interaction. This allows many integrations, but means “supports MCP,” “has approvals,” and “runs safely in a container” are three different claims. A new harness should expose those boundaries before adding more integration options. G7, G10, G17, G19

For verification, the repository includes a dummy-provider pipeline test that reconstructs after every applied step and checks a two-inference tool exchange. Observability includes streamed message/usage/MCP events and tracing spans for inference, tool execution, and agent invocation. These are source-level capabilities; their presence is not evidence that tests passed here or that a telemetry exporter is enabled in every installation. G31, G32

Evidence matrix and reading path

Read G1 → G6 → G2 first, then G5 for the outer wrapper. Follow G13/G14/G8 for execution, and G21/G23/G24 for model context and durable state. Each row is a bounded implementation claim.

Limitations: the detailed path is Goose-managed inference and extension execution at this commit. Provider-specific internal loops, every remote MCP server, every platform-specific process behavior, and full crash consistency were not exhaustively audited. Code-mode restrictions and multiplayer conclusions are bounded to the cited paths. No runtime benchmark or security certification is implied.

Sidekick: event-driven orchestration around a durable agent loop

Reviewed 2026-10-19 at commit 463d653124e1c2cda46a3f14f01f3d6f384326b1.

Source access: Sidekick is a private repository. This chapter publishes an explanatory review, not its implementation. The pinned GitHub citations require repository access. The findings describe the inspected checkout; they are not a claim about every deployed configuration.

Sidekick makes the execution boundary especially visible: a conversation turn can outlive the worker attempt that is currently processing it. Work is admitted into database records, dispatched to workflows, processed through explicit state transitions, and projected to connected clients. Its native worker still contains an ordinary iterative model/tool loop. The distinctive lesson is how that loop participates in a larger event-driven system, rather than the disappearance of iteration. Native loop Turn attempts

Scroll sideways to explore the diagram →

Two execution paths, one conversation product

Start with routing. A native Sidekick or model-API provider runs the cluster agent loop. A provider that wraps an external harness requires an Outpost, Sidekick's bridge to a separately hosted execution environment. Supplying an Outpost for a native provider can instead make local tools available while the model loop stays in the cluster. A missing required Outpost is an error, not permission to silently substitute the native loop. Execution routing

This matters when comparing capabilities. An external harness may supply its own permissions, compaction, MCP support, or session format. Hosting it behind Sidekick does not establish that the native Sidekick runtime implements the same mechanisms. Conversely, Sidekick's admission, shared conversation, and result presentation remain useful orchestration features even when another harness owns model execution.

The native path combines four cooperating mechanisms:

These are concrete layers in the inspected implementation, not four names for the same event bus. Reducer Effect executor Tool scheduling

A concrete turn trace

Imagine a user requests a search followed by a change. The trace below follows the native path; it is a teaching example, not a recorded production conversation.

1. Admit work. A pending turn records the request before dispatch. Admission policies run under a conversation lock so concurrent submitters do not independently decide the same conversation is idle. Policies vary for ordinary work, panel seats, and delegated children. Admission
2. Claim an attempt. The worker claims that turn, records the workflow identity and attempt number, and maintains a heartbeat. A fresh competing attempt normally refuses to claim it. A permitted retry or stale-worker takeover changes which attempt owns the turn. Attempt claiming
3. Prepare the next request. At an iteration boundary, the loop checks cancellation, places eligible queued steering messages, and reconstructs model context from conversation records. A steer is therefore consumed at a defined boundary, rather than spliced into an already executing model request. Iteration boundary
4. Process the model stream. Text deltas update streaming state and are broadcast. Completed assistant messages produce persistence and broadcast effects. When the assistant requests tools, the loop records tool intents alongside the assistant message and commits before dispatching those tools. Model response Intent boundary
5. Dispatch and collect tools. The executor submits tool workflows with correlation identifiers. Each tool's intent must be claimed before its effect. Results may arrive independently; the executor builds the returned batch in the model's original call order. The loop records those results, commits, and reconstructs context for the next model request. Tool scheduling Intent gate
6. Settle the turn. When a terminal answer is reached, the owning attempt writes the settlement in the final commit. Post-commit processing runs hooks and publishes a completion projection. The database outcome, not merely a last token or segment result, tells the product that the turn has finished. Final answer Settlement Post-commit finish

Tool workflows can overlap after submission; this is not a sequential interpreter that waits for every tool before submitting the next. It is also not an unlimited parallelism promise: workflow infrastructure, tool routing, and downstream services still constrain execution. Dependent tool calls must be separated or explicitly coordinated. Tool scheduling

What “event-driven” means here

The session reducer accepts backend messages, user commands, and system signals. It returns new session state plus declarative effects such as persist, broadcast, or forward to an Outpost. It performs no I/O itself. The effect executor applies those effects in order; actual Outpost sending belongs to the WebSocket handler. This separation makes message semantics inspectable without running the network or database. Reducer Effect executor

The native model loop dispatches its messages through the same session machinery. Tool work crosses workflow boundaries and wakes its caller through correlated results. Admission, cancellation, suspension, recovery, and settlement have explicit durable state transitions. These mechanisms justify describing Sidekick as event-driven orchestration. A streaming UI alone would not: emitting token events is observability, whereas admitting and claiming work changes who may execute it.

There are limits to the label. The model/tool continuation itself is a bounded loop inside a conversation workflow. The code does not make every model step a separately scheduled event, and the session reducer is not a complete event-sourced reconstruction of all runtime state. Durable turn rows and tool intent rows hold authority that cannot be inferred from the transient stream. Sidekick offers a useful architectural contrast within this comparison; this review does not establish that it is the only harness with event-driven execution. Native loop Session dispatch Turn lifecycle

For a cross-harness treatment of these choices, read Event-driven execution, including the tradeoffs and a small failure-injection exercise.

A turn is not an attempt

The turn states include pending, running, awaiting input, and settled. An attempt identifies one producer trying to advance the turn. This distinction answers a practical question: if a worker disappears, did the user's task fail, or did one process fail while work remains recoverable?

Claiming uses database locking and a conditional update against the observed state and current attempt. A current heartbeat normally prevents takeover. On the cluster path, a stale attempt can be replaced; a higher retry count for the same workflow run can also replace its earlier attempt. The latter rule distinguishes a workflow redispatch from an ordinary duplicate delivery. Ownership checks prevent a superseded attempt from recording a valid settlement for work it no longer owns. Attempt claiming Settlement

These checks are a form of fencing: a record identifies the producer currently authorized to change durable state. They do not physically stop an old process or undo a remote operation it already started. That is why tool intents and recovery policy are separate from turn ownership.

Outpost execution deliberately differs. Its dispatch attempt is not automatically taken over and re-sent, because the external harness might still be acting. A timeout or disconnected observer is not proof that the external action never began. Attempt claiming

A native request for user input can suspend the turn and end the worker run. The response resumes the same turn through a later attempt. Asking a question alongside ordinary tools is refused for that question, allowing the other tools to produce their results; otherwise suspension would strand sibling tool calls. User cancellation is also distinct from workflow interruption: the former settles a cancelled task, while an engine interruption can end only the attempt and leave recovery possible. Question suspension Cancellation distinction

Tool recovery records what is known

A durable conversation transcript does not by itself solve tool replay. Suppose a tool changes a remote system and the worker dies before writing its result into the conversation. Repeating the last model request may repeat that change. Sidekick records each intended tool call and tracks whether an executor claimed it and whether an outcome was stored. Intent boundary Intent gate

Recovery runs before the next model request and treats the recorded phase as evidence:

The implementation re-reads a row when a competing writer wins, so a result arriving during recovery can change the action from rerun to redelivery. Delegation adds a child-turn check and does not simply launch the child again. These are recovery decisions based on durable knowledge, not a guarantee that outside effects happen exactly once. “Replay-safe” also does not mean “authorized by the user”: retry policy and permission policy answer different questions. Recovery decisions Recovery policy check

The inspected integration tests make the uncertainty concrete. A simulated crash after tool effects but before result persistence causes a replay-safe search to run twice across attempts, while an unsafe operation runs once and receives an interrupted-result explanation. A different crash before the tool is claimed permits execution because the record proves it has not started. These tests are valuable specifications; this review inspected them without running Sidekick's services or test suite. Crash-window tests

Live streams and durable truth

The reducer persists completed assistant messages, tool results, and selected audit messages; text deltas and progress events are ephemeral. Dispatch replaces local sequence proposals with atomically allocated sequence numbers so concurrent writers can share a conversation. It also checks the persisted sequence floor when a cache counter is lost. Sequence ordering is therefore coordinated beyond one loop's memory. Reducer Sequence allocation

Persistence effects flush rows through the current database session. They do not commit the transaction. Ordinary dispatch can broadcast before its caller commits; explicit deferred-broadcast paths publish acknowledgements after commit. Consequently, seeing an event or sequence number does not prove that its transaction survived a crash. This is a boundary to preserve when building clients, replay tools, or monitoring. Effect executor Session dispatch

Turn completion has a stronger contract. Its event is a projection of the settled row, emitted after commit and reconstructible for clients from durable state. The code distinguishes a segment result from whole-turn completion and attaches parent identity so a delegated child's completion cannot masquerade as its parent's. Turn lifecycle

Durability still has operational limits. A sweeper can settle stale producers or expired input waits, and a special recovery path can redispatch a lost queued submission. Post-settlement hooks and projections remain post-commit work; the sweeper explicitly logs cases where these steps are lost rather than claiming a durable retry for all of them. A single authoritative settlement is stronger than a UI flag, but it is not a transaction spanning every notification and downstream system. Sweeper recovery and limits

Context, collaboration, and tool authority

Rebuilding context is not the same as summarizing it. The inspected native context factory registers optional removal of older, large tool outputs from the request projection. That is useful context management, but it does not establish automatic full-history summary checkpoints or overflow-compaction recovery. External harness behavior remains a separate question. Context pipeline

Human collaboration is concrete product behavior: conversation access distinguishes view, edit, and manage permissions, including direct and group shares. Sending requires edit access. Shared observation, durable input admission, and sequencing solve different parts of the collaboration problem; none should be treated as a substitute for the others. Conversation access Sending

Multi-agent work has two useful forms. Delegation admits an owned child turn and links it to the parent's tool intent before dispatch. Panels instead grant peer agents a floor; completion updates persisted grants, and the last completed or failed statement can make a moderator wake eligible. That latter dependency is a concrete example of events determining subsequent execution. Child admission Panel completion

Skills are attached, versioned instruction packages with a prompt catalog and on-demand instruction/reference tools. A stable skill-tool dispatcher checks the attached binding and declared tool, then uses the normal system dispatcher; it currently supports built-in system tools. Skill discovery does not itself grant broader tool authority. Skill resolution Skill invocation

The native tool path has deterministic capability checks for declared governed tools, with Mail and Files covered at this pin. That is not a general model-based Bash safety classifier. Likewise, the frontend's auto_approve setting automatically answers permission requests; its name does not establish a classifier that evaluates safety and user authorization. Capability coverage Automatic response

No native MCP client or MCP code-mode orchestration was established in this review. The inspected ACP adapter passes an empty MCP-server list, and native tool routing covers system, configured CLI, and stored Python tools. These mechanisms must not inherit capability claims from a wrapped external harness. Tool routing ACP session configuration

What to borrow when building a harness

Begin with the failure you need to survive. If one local process owns a short-lived task, a clear loop and append-only transcript may be sufficient. When multiple submitters, workers, observers, and external executors participate, Sidekick's separation becomes useful: define a durable unit of admitted work, identify the current attempt, and decide what may be retried from evidence rather than from silence.

The most reusable ideas are the split between turn and attempt, declarative session effects, tool-intent gates before execution, and separate treatment of live projections and committed records. Each has a cost: more states, more transactions, more identifiers to correlate, and tests for the gaps between them. Adopt these mechanisms where your failure model needs them, rather than treating distributed orchestration as a prerequisite for the first working agent.

Source reading path

Read routing first to choose the native or external path. Follow the native iteration boundary and model/tool handling, then read the reducer and dispatch to see what a message actually does. Move to claiming, settlement, and intent recovery for failures. Finish with crash-window tests and ownership tests, which expose the temporal contracts more clearly than a happy-path diagram alone.