# Build a harness by making its contracts explicit

The goal of this lab is a small runtime whose behavior you can explain. The pseudocode below is a proposed design, not an extracted implementation or production-ready library. The codebase chapters supply concrete implementations of the tradeoffs.

## Start with the smallest observable loop

```text
run(input, session, cancellation):
    record input
    repeat while run budget remains:
        stop if cancelled
        context = prepare_context(session)
        response = request_model(context, tool_schemas, cancellation)
        record completed response

        if response failed:
            return a classified failure

        if response contains tool calls:
            for call in response.tool_calls:
                resolve name and validate arguments
                decision = authorize(call, current_policy)
                result = execute_or_record_denial(call, decision, cancellation)
                record result with call.id

        input = take_steering_at_defined_boundary()
        if input exists:
            record input
            continue

        if response has tool calls:
            continue

        input = take_follow_up()
        if input exists:
            record input
            continue

        return completed

    return budget_exhausted
```

Use a fake model and fake tools first. A fixed sequence of model responses is enough to test the runtime. It isolates loop bugs from provider behavior and costs no inference tokens.

### Exercise 1: maintain call/result identity

Have the fake model request two tools, one with an unknown name. Record the successful result and the unknown-tool error against their original IDs. The next request should contain a valid representation of both outcomes.

**Completion criterion:** no orphan tool calls; a tool error reaches the model as a tool outcome; malformed requests cannot dispatch an unintended function.

**Further reading:** the tool validation and execution sections in [pi](../snapshots/pi/482c1381a00d/chapter.md), [Codex](../snapshots/codex/8c076942cb70/chapter.md), and [Goose](../snapshots/goose/34a222e3f2b9/chapter.md).

## Add policies before expanding the tool surface

Expose a read-only in-memory file tool first. Add file writes only after defining which paths are permitted. Add process execution after defining environment, working directory, output limits, timeout, and cancellation behavior.

### Exercise 2: deny a valid request

Send a structurally valid write request for a disallowed path. The tool executor must never run. Feed the denial back as a result the model can respond to.

**Completion criterion:** validation and authorization have separate tests; a denied action is observable; a tool's “read-only” label is not accepted as operating-system enforcement.

The necessary part is an actual boundary around effects. A classifier, remembered approvals, or a policy language is optional complexity until your application needs it.

## Define interruption before adding parallelism

Cancellation requests that active work stop and must reach cooperating tools. It does not undo completed effects. Steering supplies new instructions. They should not share an ambiguous “interrupt” operation.

### Exercise 3: steer during a slow tool

Run a tool that waits for a test-controlled signal. Queue “Use the other file instead.” Verify whether the current tool completes, whether other calls in its batch start, and when the next model request sees the new message.

**Completion criterion:** tests assert the chosen boundary; the UI describes that behavior accurately; an explicit cancel reaches the tool independently of steering.

### Exercise 4: reverse tool completion order

Let the model request reads A and B. Complete B first. Decide whether events and persisted results use completion order or request order, then assert both independently. Repeat with two writes to the same file.

**Completion criterion:** results retain IDs; transcript order is intentional; dependent mutations are serialized or rejected. A faster happy path is not enough to validate a scheduler.

## Make context preparation a visible operation

### Exercise 5: compact around a tool boundary

Create a transcript close to your context budget. Include a tool request, its result, and a later instruction correcting the task. Produce a summary and retain a recent tail.

**Completion criterion:** the next provider request is structurally valid; the correction survives; the original history remains inspectable according to your storage policy; failure to summarize has an explicit outcome. Test a summary that grows larger than expected.

Treat context estimates as estimates. Reserve space for the model's response and tool results. When the provider reports overflow, make recovery bounded; repeatedly summarizing an already tiny context can otherwise loop forever.

## Separate resume from safe replay

### Exercise 6: crash after an external effect

Use a fake counter service. Execute an increment, then simulate a crash before saving its result. Resume the session.

**Completion criterion:** the runtime either knows the operation is safe to retry, reconciles its external status, or reports an interrupted/uncertain outcome. It must not claim exactly-once behavior merely because the transcript is durable.

Record operation IDs, statuses, and replay policy before needing recovery. For an application with external side effects, this is more consequential than adding another model provider.

## Extend the harness without bypassing its rules

### Exercise 7: put the same tool behind MCP and code mode

Model Context Protocol (MCP) lets a server advertise tools and execute calls over a defined interface. Start with one fake server exposing a read and a write. Then expose a code execution tool whose program calls those tools.

**Completion criterion:** direct calls and code-generated calls pass through the same policy checks, tracing, cancellation, and result handling. A denied write stays denied when wrapped in a script. Tool discovery is scoped to the current configuration and access rights.

Test errors at three layers: the script fails, an MCP call fails, and the connection fails after the server may have acted. These are not interchangeable retry cases.

## Add another participant deliberately

### Exercise 8: two clients, one active run

Connect clients A and B. A starts a run; B submits input while A's tool is active. Have both try to approve the same pending action. Disconnect A, then reconnect with an event cursor.

**Completion criterion:** the system defines who can steer and approve, detects stale run IDs, avoids duplicate decisions, and reconciles history with live events. Presence indicators and shared cursors are optional UI; authority and consistency are runtime concerns.

### Exercise 9: a child agent with a bounded task

Delegate one independent read-only investigation. Give the child its own context and an explicit result contract. Cancel the parent while the child is active.

**Completion criterion:** you know whether cancellation propagates, whether the child can mutate shared files, how its tools are authorized, and how much of its output returns to the parent. More agents do not remove the need for ownership of shared state.

## Evaluate behavior, then task quality

Keep two test collections. Runtime tests check invariants using fake providers, clocks, tools, and storage failures. Task evaluations measure whether a real model solves representative problems under the chosen harness configuration.

For fair task comparisons, record model and version, prompts, tool schemas, permissions, initial files, context limits, retries, budgets, and result criteria. Repeat runs and report variability. This resource performs source analysis; it does not provide task-quality, cost, or speed benchmarks.

A useful first milestone is not “the agent solved one task.” It is “the runtime reliably represents completion, denial, failure, cancellation, and uncertain recovery, and I can inspect why it chose each state.”