MCP session continuity — recurrence risk and residual work — implementation plan

Status: Planned · Author: Fable/Opus (TRIAGE partner netdocs gating) · Date: 2026-09-07, revised the same day after adversarial review · Repo: north-os (apps/mcp-server, deploy/preprod) · Related: docs/reviews/2026-09-07-partner-netdocs-gating-triage.html (#369), PR #370 (re-mint on any request), PR #371 (lawyer guidance), specs/2026-08-27-mcp-tool-list-refresh-design, PR #354 (static two-upstream router, merged, not cut over).

The question, and the honest answer. kwiss asked for a percentage that a chat never again loses its North tools. No such percentage is defensible from the evidence, and this page no longer offers one. A first draft of this plan did (95 % per deploy, 99 % per chat); adversarial review (omp GPT-6-Astra high) showed the denominator does not exist and the arithmetic ran the wrong way. What is defensible is a list of deterministic failure paths proven in the source, which of them the current wave closes, and a measurement programme that turns the estimate into a real number over time. Section 2 states what can and cannot be claimed. Section 6 records what the review changed.

1. Failure modes that make a chat lose North tools

M1 to M9 come from the triage. M10 to M21 were added by three rounds of adversarial review and by a deep SDK-level investigation (an omp GPT-6-Astra pane that built a disposable Postgres, Redis and mcp-server and drove the real client across two restarts). Each is proven in the source or on the wire, not hypothesised.

#	Mode	Cause, with evidence	Closed by	Status
M1	Restart, then the chat's first probe on its stale session is refused	Only `tools/call` and `tools/list` were re-mint eligible; hosted clients send `server/discover` first; 24 refused misses since 08-31 are directly followed by that probe on the same session; the client treats 404 as a dead connector. 21 boots since 08-26.	PR #370: any well-formed JSON-RPC request except `initialize` re-mints. Reproduced with the real SDK client across a local restart; six protocol tests, mutation-checked.	Closed for the first restart only. See M10.
M10	The second restart is a permanent 404	`REPLACE_SESSION_RECOVERY_SCRIPT` (`redis-limiter.ts`) does `DEL` on the stale record and `ZREM` on its index entry when it stores the replacement. The stale→replacement alias then exists only in `sessionRemints` in memory, and `closeSessions()` clears that map. So: initialize A, restart, re-mint A→B while the client keeps sending A, restart again, `loadInitialization(A)` is null, 404 forever. Healthy Redis, no downtime, no eviction needed. Verified in the branch source. Hosted clients do keep the original id: the 20 re-mint events are 4 distinct pairs, one of which served 16 further alias hits, and 20 successful settlements joined by `arrival_id` came from original-id arrivals.	Fixed in PR #370 at `06d73f9d`, after this review and the lane's own dual review found it independently. `replaceInitialization` and its Lua script are deleted from the branch; re-mint now refreshes the stale record and stores one for the replacement, so no record is destroyed. Verified in the branch source.	Reopened. The three-boot probe passes, but round-2 dual review found the fix incomplete and I verified it: `REMEMBER_SESSION_RECOVERY_SCRIPT` re-indexes only the new id and evicts the oldest same-principal record, so nothing retires the previous generation. N restarts leave N+1 indexed records for one chat, all counting against the per-principal cap of 8, and the refreshed original is never the eviction victim, so a recovering chat evicts the user's other chats' records, which then 404 permanently at their next restart. The probe could not see it: one chat, one principal. See M17.
M11	An alias hit refreshes neither durable retention nor the catalogue	On the alias path the code sets `lastSeenMs` in memory and dispatches. It never calls `refreshInitialization` (so the replacement's 7-day durable TTL runs from re-mint time even for a client active every day) and never calls `refreshDynamicTools` (so a `tools/list` arriving on the stale id returns the catalogue the replacement was born with). The known-session path does both.	Fixed in PR #370 at `06d73f9d`: the alias path no longer dispatches straight to the transport. The request is rebound to the replacement and served by the shared known-session path, which does the durable refresh of both ids, `refreshDynamicTools`, retention expiry retiring both records, and DELETE retirement of both. Verified in the branch source.	Closed pending tests. The lane reports both behaviours were only smoke-checked; a tests-only worker is pinning them now (a list on the stale id after a posture flip must show the new catalogue and lose it on the flip back). Untested is unproven.
M12	Live-session eviction destroys the recovery record	At the 9th concurrently retained session for one principal, the least-recently-seen transport is evicted; at the global cap another principal's transport can be evicted. There is no in-flight or stream-open exemption. Closing a session also forgets its recovery record. So raising only the recovery-record cap (draft R3) would not have helped.	Redesigned R3.	Open.
M13	Access-token expiry mid-chat	Authentication runs before session recovery, so an expired token is rejected regardless of session state. The SDK invokes its auth flow only when an auth provider exists and has a circuit breaker on an immediately repeated 401. Seven successful historical rotations say nothing about same-chat retry behaviour.	Nothing; separate from this lane.	Open, unmeasured. The draft's M8 conflated normal expiry with DCR churn.
M14	Effective-principal change after scope narrowing	Scopes are part of the principal key; a narrowed grant invalidates both the live session and its recovery record by design. Correct security behaviour; the chat still loses its tools.	Nothing. Must not be "fixed" by weakening the principal check.	Open by design; document it.
M15	Seven-day retention closes a dormant chat	A session idle past `SESSION_RETENTION_MS` is closed and its recovery forgotten; the next request is refused. Any promise of indefinite old-chat continuity is false.	Nothing; bounded service contract.	Open by design; state the bound.
M18	Hosted clients never adopt the replacement session id	Measured, not inferred: on 09-02 one stale id was re-minted 17 times to the same replacement, with 24 `tools/call` and 23 `server/discover` arrivals logged under the original id and zero under the replacement. The pinned SDK adopts the header (it assigns the session id before checking response success), so both local reproductions exercise a client population that does not exist in production, and the three-boot probe passes only because the harness forces the original id back. Every restart therefore mints a replacement nobody uses; the orphan decays only on its 7-day TTL or by evicting a sibling. 09-01 had 6 boots, so one conversation alive that day consumes 7 of a principal's 8 slots. It also means a probe 404 is not fatal: that chat ate 17 of them and kept working.	Resolved by changing the design rather than patching it. The lane is rebuilding the session under its original id: the replayed initialize passes the stale id to the transport's id generator, so the rebuilt session carries the id the client is already sending and neither client has to adopt anything. Everything built to work around non-adoption is deleted with it: no second durable record, no alias key space, no linking script, no orphan generations, no pair retirement. One record per chat for the life of the chat, and every later request takes the ordinary known-session path.	Design settled, in build. Expected to be a net deletion; if it is not, something was kept that should have gone.
M19	Live cap saturation is the steady state today	Redis holds 173 recovery records across 25 principals and 16 of the 25 sit at exactly the cap of 8, so the eviction branch fires on every new session for most users already, before any fix. Two further consequences: past about 32 principals the global cap of 256 starts evicting the globally oldest record across tenants, and the eviction script decodes every index member on every remember, which is 173 blocking round trips inside a single-threaded Redis script.	Same-id reconstruction removes the pressure at the root: re-mints stop creating records, so the eviction branch stops firing on them. The durable limit also gets its own constant at 16, leaving live transports at 8, since they are different resources.	Addressed by the M18 design.
M20	The one-hour token boundary crosses a chat far more often than a restart	Access tokens last one hour, so every conversation open longer than that crosses a boundary. Restarts run at 14 in 10.9 days, about 1.3 a day, so the token boundary is roughly 20 times more frequent. The journal holds 229 `401` responses on the MCP path, 198 of them from one hosted user agent, against 56 session misses of any kind. The access log records neither whether a bearer was presented nor which challenge was returned, so we cannot today say how many of those 198 are a benign handshake and how many are a chat losing its tools mid-conversation.	Nothing. The plan spent its budget on the 1.3-a-day path.	Open and mis-prioritised. Distinguishing the two challenges in the log is the cheapest next measurement in this whole plan.
M21	The fix breaks the metric the acceptance depends on	The tool-call arrival line carries the original id while the method line carries the replacement, and the re-mint event is now gated on creation, so that 09-02 chat's 17 events become 1. Every per-session-id quantity means something different either side of this deploy.	Acceptance must join on `arrival_id` only and must not compare per-session-id counts across the deploy.	Open.
M22	Same-id reconstruction makes a session-id collision possible for the first time	A session id was previously server-minted and unique per transport, so two transports could never share one. Rebuilding under the original id removes that guarantee: two concurrent eligible requests on the same dead id would each rebuild under it, and the second map write silently orphans the first live transport, which is then never closed and whose task store is never cleaned. The shared re-mint promise should coalesce them; that it still does, now that the replacement id equals the stale id, has to be proven rather than assumed.	Relayed to the lane as the one new failure class to pin: two concurrent requests on a dead id must yield exactly one transport, one map entry and one record, with both requests served, plus an assertion that a mismatched principal still cannot claim an id.	Open, in the same round.
M17	Recovery records accumulate per generation and evict other chats	Storing a replacement neither retires the generation it replaces nor exempts other chats from eviction. Three more defects share the root: a failed refresh of the original still stores the replacement and answers 200, so the chat dies silently at the next restart; a DELETE arriving on the adopted replacement forgets only that id, leaving the original able to re-mint a deleted session; and pair retirement is two calls that can leave one record behind. Verified in `redis-limiter.ts` on the branch.	Fix in flight in the session-recovery lane: one capped record per chat plus uncapped alias keys, linked by a single Lua script that asserts the record exists and the principal matches, so a lost record fails the request closed instead of silently ending the chat; retirement resolves through the alias and deletes record and index member atomically whichever id the client sent.	Open, blocker on PR #370.
M16	Recovery is POST-only: GET, DELETE and notification-only requests on a stale id still 404	The eligibility gate is `method === "POST"` plus a JSON-RPC request schema, so a standalone GET, a `DELETE` teardown and a bare `notifications/initialized` on a stale id are all refused, and the DELETE leaves the durable record to expire on TTL instead of retiring it. The deep investigation proved with the real SDK that a replacement id returned on a GET is not adopted by the client, so re-minting GET would not repair it either; only same-external-id reconstruction would. It also observed that the SDK's initialized-notification path is what opens the stream, so a client that restarts into a notification never opens one.	Nothing. Recorded as a known limit in PR #370.	Open, low priority: 3 GETs since instrumentation, all 401, so hosted clients open no authenticated stream.
M2	Request arrives during the restart's downtime window	The unit is stopped and started in place; Caddy proxies to a single upstream with `dial_timeout 2s` and no configured retry, so the client gets a 502. One measured shutdown-to-boot interval on 09-07 was 7.3 s; that is one sample, not a distribution.	Nothing in the wave.	Open. Overlaps M3; do not add them as independent risks.
M3	Restart with a tool call in flight	`shutdown()` calls `server.close()` then closes every session at once; the 180 s stop budget is not used to drain.	Nothing in the wave.	Open. R1a.
M4	Recovery records lost with Redis	Redis on the dev instance has `appendonly no` and RDB snapshots at 3600 s/1 key, 300 s/100, 60 s/10 000. A crash loses up to the last snapshot; a clean restart need not.	Nothing in the wave.	Open. R2. Cannot repair M10, M12 or M15, which delete records deliberately.
M5	Recovery record evicted by the lawyer's own newer sessions	Eight retained sessions per principal; each connector toggle mints one. Superseded in mechanism by M12: the live eviction is what destroys the record.	Redesigned R3.	Open.
M6	Connector toggled; the open chat does not adopt the tools	Each toggle opens a fresh session with us, answered in 20–60 ms. What the client does with a fresh list for an already-open chat is the client's business. Nine such sessions and zero tool-call arrivals in the incident window.	PR #371 (guidance).	Cannot be closed from the server. Mitigated and measured, not fixed.
M7	Catalogue changes while a chat is open	Nothing pushes a change into a live session. Zero authenticated GET streams were observed (3 GETs, all 401). That is not proof hosted clients cannot receive notifications: a notification carrying a `relatedRequestId` can ride the POST SSE response. The deep investigation added a decisive negative: closing and reopening the stream does not cause a re-list. Two same-id reconstruction probes reopened the stream and issued zero `tools/list` until a notification was actually sent; and a client re-lists on that notification only when constructed with `listChanged.tools`, not through a manually registered handler.	Partly M11's fix (pull parity on the alias path).	Open; R4 rewritten, no impossibility claim.
M8	DCR churn forcing re-consent	PR #284's subject. Split from M13.	Out of scope.	Unchanged.
M9	"Not in NetDocuments" declared from page one	Three searches returned full-looking pages, never paginated, no fetch; totals not persisted.	Absence-discipline lane and observability lane.	Not a continuity mode. Keep out of this gate.

2. What can and cannot be claimed

Not claimable: any per-chat or per-deploy recurrence percentage. There is no denominator: the journal records arrivals, not conversations, and cannot see a chat that dies silently or a 502 returned by the proxy. The first draft's numbers are withdrawn.

Claimable, and worth stating to kwiss plainly:

Before PR #370 a probe after a restart was usually refused, but not universally and not fatally. Across all 36 refused misses the next readable method is 25 server/discover, 8 tools/call and 3 initialize: the 8 tool calls were already eligible and were refused for want of a record, so #370 does not fix them. And one chat on 09-02 absorbed 17 refusals and kept working. The earlier "24 of 24, that is why the chat died" was a denominator error and is withdrawn.
PR #370's first commit recovered that probe once: re-minting deleted the stale record, so the second restart was a deterministic permanent 404. With 21 boots in twelve days that was not a corner case. Commit 06d73f9d closes it by keeping both records, and a three-boot probe on the branch now passes on the original id.
What remains before the claim can be made: the same-id reconstruction must land with M22 pinned (one capped record per chat, aliases uncapped), the tests pinning M10, M11 and M17 must bite under mutation, M12 is still open, and the hosted client must be seen recovering the same conversation across two restarts, which only a canary on kwiss's own account can show.
The honest form of "never again" is a deterministic two-restart gate in the test suite plus that canary, not a statistic.
Arithmetic the review supplied against the draft's own framing: four consecutive clean deploys give a one-sided 95 % lower bound of 47.3 % on the success rate; roughly 299 consecutive zero-failure deploys would be needed for that bound to clear 99 %. Counting deploys is not the way to earn this claim. Proving the state machine is.

3. Residual lanes, reordered

Order changed on review: define the outcome and the evidence first, fix the deterministic recovery defects second, evaluate the cheap deployment mitigation third, adopt blue/green only if measurement says the cheap one is insufficient.

Lane	What	Cap	Needs
R0 two-restart recovery gate (first)	A local, deterministic test: initialize A, read successfully, restart, send probe/list/read on A keeping A even after B is returned, restart again with Redis intact, repeat on A. Assert completed reads, not log lines. Now exercised by PR #370's three-boot probe; the standing test that pins it is the lane's open work.	≤120 lines of harness	No. Belongs to the session-recovery lane.
R3 eviction (M10, M11 and M17 land in PR #370)	M12 only, once M17's record model lands: make capacity retirement stop destroying recoverable identity, while explicit teardown and revocation still do.	≤200 lines	No
R8 separate the 401 challenges in the log (new, cheapest measurement in this plan)	The code already distinguishes a missing credential from an invalid token; the access log throws the distinction away. 198 of 229 MCP 401s come from one hosted agent and we cannot tell a handshake from a chat dying. Log the challenge class, then measure M20 before spending another lane on the 1.3-a-day restart path.	≤40 lines	No
R6 evidence design (promoted, before acceptance)	Status and reason on the refused line, boot and release correlation, arrival→settlement joins by `arrival_id` (settlements carry no session id), and a canary cohort. The MCP journal cannot see proxy 502s; that needs the Caddy access log. Fifteen minutes after boot is too early for a chat that returns hours later; the "listed then no call in ten minutes" proxy is weak (one observed session listed at 12:47 and called at 13:19).	≤150 lines	Alerting through Better Stack only
R7 same-chat canary (promoted)	kwiss's own account, one conversation kept open across two consecutive restarts and one connector toggle, per client application, driven by him. A re-mint log line is not sufficient: the conversation must receive and use the result.	Procedure page	kwiss drives the session
R1a drain	`shutdown()` stops admission, lets accepted work finish streaming within a deadline, keeps DB and Redis alive until it settles, then closes.	≤120 lines	No
R1b failed-dial retry, evaluated before blue/green	Bounded retries on failed connections only in the Caddy MCP block, sized against the measured restart interval and the client's request budget. Never blanket POST retries: a tool may have executed before its response was lost.	Config + test	kwiss's word (deploy machinery)
R1c blue/green for mcp-server, only if R1a+R1b measure insufficient	Two instances behind PR #354's static router with an `active.<colour>` marker. Note: port 5175 is worker-meetings; pick an unallocated port. Health must probe the MCP process and build, not Caddy's static `/health`.	≤400 lines	kwiss's word; #354's own cutover is a prerequisite and is not done
R2 Redis durability	Specify the recovery-point objective, fsync policy, safe enablement, validation and rollback before touching `appendonly`. Not a one-line change, and it repairs none of M10, M12, M15.	Design first	kwiss's word (ops on preprod)
R4 Phase B, restated (now evidence-backed)	Delete the spec's Phase B fallback. "Close the standalone stream so a compliant client reopens and re-lists" was proven false against the pinned SDK: the stream reopens and no `tools/list` follows until a notification is sent. Record that unassociated push needs a stream nobody opens, that a request-bound notification can ride the POST SSE response, and that re-listing depends on a client option we cannot set. Phase B, if ever built, is tracked invalidation delivery with a re-list acknowledgement, not stream closure.	Docs	No
R5 absence eval	Folded into the existing absence-discipline lane, which already owns an eval fixture. It must exercise the MCP prose actually changed, or it proves nothing.	—	No

4. Measurement

Deterministic, local, blocking: R0's two-restart gate. This is the one that earns the claim.
Per boot, from the journal: refused misses with their reason and status; re-mint events with stale→replacement pairs (count distinct pairs, not events); tool-call arrivals joined to settlements by arrival_id, since settlements carry no session id.
Per deploy, canary: the same conversation across two restarts and a toggle, confirmed in the client, not only in the log.
Not acceptance: counting clean deploys, or counting sessions that listed and did not call.

5. Out of scope

Session resumption with an event store; OAuth expiry, refresh availability and DCR churn (M8, M13, tracked separately); the NetDocuments capability lanes (browse and ids, filing back, watch), which have their own plan and spec; answer-quality work (M9).

6. What the review changed

First draft said	Reality
PR #370 makes a restart invisible to an open chat	At `9aa323fc` it made the first restart invisible; the second was a deterministic permanent 404 (M10). Fixed at `06d73f9d` by keeping both records, which round-2 dual review then showed was itself incomplete: generations accumulate and a recovering chat evicts other chats (M17). A durable alias keyed to one capped record per chat is now in flight. Two rounds of review, two incomplete fixes: this is why the acceptance gate is a test and a canary, not a probe run once.
My own report to kwiss that the blockers were closed, citing "two durable records per chat, the original is never the eviction candidate"	Wrong, and I repeated it from the lane without re-deriving the cap accounting. The original is protected precisely by making some other chat the victim. Corrected here and to him.
"The first probe after a restart was refused every time it was sent, 24 of 24. That is why the partner's chat died."	A denominator error: 25 of 36, with 8 already-eligible tool calls refused for want of a record. And a chat elsewhere survived 17 consecutive refusals. The causal story was overstated.
Bound the orphan records the fix creates	Better: stop creating them. Rebuilding under the original id removes replacements, aliases, orphans and the cap pressure together, and deletes the two rounds of machinery built to work around a client behaviour that was never measured until now.
Both local reproductions prove restart recovery	They prove it for the pinned SDK, which adopts the replacement session id. The hosted client does not adopt it, measured over 17 re-mints of one chat. The population under test was the wrong one (M18).
The plan's budget belongs on restart recovery	Restarts run about 1.3 a day; the one-hour token boundary crosses every conversation older than an hour, roughly 20 times more often, and we cannot currently tell a benign 401 handshake from a chat losing its tools (M20).
About 95 % of deploys pass unnoticed, 99 % per chat	No denominator exists; the numbers are withdrawn. Even the draft's own illustrative workload, worked through, gives 50–88 % overlap probability, not "a few percent".
Four clean deploys accept the fix	Four clean trials support a 47.3 % lower bound. Acceptance must be a state-machine gate plus a canary, not a deploy count.
A chat learns of a catalogue change at its next `tools/list`	False on the alias path, which skips the posture refresh entirely (M11).
Raise the recovery-record cap (draft R3)	Live-session eviction is what forgets the record (M12); raising only the durable cap changes nothing.
Hosted clients hold no GET stream, so Phase B is not applicable	Zero authenticated GETs were observed, but a request-bound notification can ride the POST SSE response. No impossibility claim. Separately, the spec's own Phase B fallback (close the stream to force a re-list) was proven not to work against the pinned SDK, so R4 now deletes it rather than restates it.
My addendum: "the re-mint path has never been observed carrying a chat forward"	Wrong. Four stale→replacement pairs, one serving 16 further alias hits, and 20 successful settlements from original-id arrivals. The triage's implied "every open chat lost its tools at every deploy" is wrong for the same reason.
Blue/green for mcp-server (draft R1)	Unproven as necessary. Try a real drain plus bounded failed-dial retry first; port 5175 in the draft is already worker-meetings.
Observability: put `total_reported` in `redactArgs`	Impossible: `redactArgs` sees inputs, not the provider's envelope. Relayed to that lane.