fix(mcp): serve tools/call concurrently so a long shell cannot starve file ops #13

Merged

troed merged 4 commits from fix/concurrent-front-door into main

2026-11-08 14:59:41 +01:00

troed commented

2026-11-08 14:59:06 +01:00

Owner

Problem

Agents hang on umwelt file operations, most often in subagent sessions, and sit silently — a human cannot tell work from a wedge.

Measured on 2026-11-08 in a live session: a subagent's umwelt_shell ran a 45 s loop (START 1794143563, END 1794143608). A umwelt_read_file in the parent session returned the finished file, END line included; the next clock reading was 1794143611. The read waited the whole 45 s. Two calls batched in one message behaved the same way.

Root cause: opencode spawns one umwelt MCP server per opencode instance, shared by the main session and every subagent. Its front door (mcp_server.main) was a single-threaded for line in sys.stdin: loop: read one request, dispatch to completion, answer, then read the next. Nothing is written to the client while a call is dispatching, so a queued call is indistinguishable from a hang.

The file operations are not slow — they take no st.lock and are bounded at 60 s. They were only queued behind whatever ran first. A long shell is the usual blocker (600 s budget), which is why subagents, the sessions that run the long commands, make it common.

Fix

attach() serialised (ctx["attach_lock"]). Two concurrent cold calls previously left the loser of the race with None from the ATTACH_BACKOFF guard — a spurious "control plane unavailable" on a live plane. Remote transports are cached under per-(server, workspace) locks, so one host's cold start (which pays the whole relay launch window in _replace_if_stale) cannot hold every other host's calls.
tools/call on worker threads. initialize, ping, tools/list and notifications stay inline — they are instant and never touch the control plane, so handshake ordering is unchanged. Workers are daemon threads capped at MAX_INFLIGHT = 8 (the reader waits on a condition beyond that); every response frame is written under one lock, one JSON line at a time. Responses may leave out of order, which JSON-RPC ids make legal.
Bounded exit. In-flight workers are drained for JOIN_GRACE = 5 s once stdin closes; the owned control plane is terminated first, and relay transports are closed off-thread with TEARDOWN_GRACE = 1 s, so exit never waits behind a 600 s remote call.
Total frame writes. A closed fd raises OSError and a closed TextIOWrapper raises ValueError — not BrokenPipeError — and an unserialisable result must still answer its id. None of these can escape a worker as a thread traceback.

Tests

umwelt/tests/test_mcp.py, eight new tests, each watched failing first:

a slow shell does not block a file call (the reported bug)
every call is answered with its own id (no interleaved or lost frame)
initialize is answered ahead of a slow tool call
a worker that raises still answers its id
in-flight calls are bounded at MAX_INFLIGHT
a write failure (ValueError from a closed stdout) does not escape as a traceback
teardown is bounded when a transport close blocks
one host's cold start does not block another

Suite: python3 -m pytest -m "not integration and not root and not mitmproxy" -q → 1311 passed, 44 deselected (the 8 warnings are pre-existing: a fork-in-multithreaded-process deprecation from test_notes).

Not in this change

Deliberately unchanged, as scoped: no progress output for a long call, no front-door deadline (each method keeps its method_timeout), st.lock acquisition still unbounded (grant/revoke/egress/ingress/watchdog), and retry-once semantics untouched. A single relay channel still serialises calls to one remote host.

After merge

The running server is the uv-tool install, and opencode spawns it once per instance. This reaches a live project only after the package is rebuilt, reinstalled, and opencode is restarted.

Design and plan: docs/superpowers/specs/2026-11-08-umwelt-concurrent-front-door-design.md, docs/superpowers/plans/2026-11-08-umwelt-concurrent-front-door.md.

## Problem Agents hang on umwelt file operations, most often in subagent sessions, and sit silently — a human cannot tell work from a wedge. Measured on 2026-11-08 in a live session: a subagent's `umwelt_shell` ran a 45 s loop (START 1794143563, END 1794143608). A `umwelt_read_file` in the parent session returned the *finished* file, END line included; the next clock reading was 1794143611. The read waited the whole 45 s. Two calls batched in one message behaved the same way. Root cause: opencode spawns **one** umwelt MCP server per opencode *instance*, shared by the main session and every subagent. Its front door (`mcp_server.main`) was a single-threaded `for line in sys.stdin:` loop: read one request, dispatch to completion, answer, then read the next. Nothing is written to the client while a call is dispatching, so a queued call is indistinguishable from a hang. The file operations are not slow — they take no `st.lock` and are bounded at 60 s. They were only *queued* behind whatever ran first. A long shell is the usual blocker (600 s budget), which is why subagents, the sessions that run the long commands, make it common. ## Fix - **`attach()` serialised** (`ctx["attach_lock"]`). Two concurrent cold calls previously left the loser of the race with `None` from the `ATTACH_BACKOFF` guard — a spurious "control plane unavailable" on a live plane. Remote transports are cached under per-`(server, workspace)` locks, so one host's cold start (which pays the whole relay launch window in `_replace_if_stale`) cannot hold every other host's calls. - **`tools/call` on worker threads.** `initialize`, `ping`, `tools/list` and notifications stay inline — they are instant and never touch the control plane, so handshake ordering is unchanged. Workers are daemon threads capped at `MAX_INFLIGHT = 8` (the reader waits on a condition beyond that); every response frame is written under one lock, one JSON line at a time. Responses may leave out of order, which JSON-RPC ids make legal. - **Bounded exit.** In-flight workers are drained for `JOIN_GRACE = 5 s` once stdin closes; the owned control plane is terminated first, and relay transports are closed off-thread with `TEARDOWN_GRACE = 1 s`, so exit never waits behind a 600 s remote call. - **Total frame writes.** A closed fd raises `OSError` and a closed `TextIOWrapper` raises `ValueError` — not `BrokenPipeError` — and an unserialisable result must still answer its id. None of these can escape a worker as a thread traceback. ## Tests `umwelt/tests/test_mcp.py`, eight new tests, each watched failing first: - a slow shell does not block a file call (the reported bug) - every call is answered with its own id (no interleaved or lost frame) - `initialize` is answered ahead of a slow tool call - a worker that raises still answers its id - in-flight calls are bounded at `MAX_INFLIGHT` - a write failure (`ValueError` from a closed stdout) does not escape as a traceback - teardown is bounded when a transport close blocks - one host's cold start does not block another Suite: `python3 -m pytest -m "not integration and not root and not mitmproxy" -q` → **1311 passed**, 44 deselected (the 8 warnings are pre-existing: a fork-in-multithreaded-process deprecation from `test_notes`). ## Not in this change Deliberately unchanged, as scoped: no progress output for a long call, no front-door deadline (each method keeps its `method_timeout`), `st.lock` acquisition still unbounded (grant/revoke/egress/ingress/watchdog), and retry-once semantics untouched. A single relay channel still serialises calls to *one* remote host. ## After merge The running server is the uv-tool install, and opencode spawns it once per instance. This reaches a live project only after the package is rebuilt, reinstalled, and opencode is restarted. Design and plan: `docs/superpowers/specs/2026-11-08-umwelt-concurrent-front-door-design.md`, `docs/superpowers/plans/2026-11-08-umwelt-concurrent-front-door.md`.

troed added 4 commits

2026-11-08 14:59:06 +01:00