NetDocuments filing back + folder watch — implementation plan

Status: planned Author: Claude (Fable, then Opus after the Fable credit stop), for Christophe Date: 2026-09-07 (R2 the same day, after two adversarial plan reviews) Repo: north-os Related: 2026-09-07-netdocs-filing-and-watch-design (the spec; §8 is the gate, §9 the probe list), 2026-06-10-document-editing (patterns copied), 2026-08-27-netdocs-firmwide-connection (resolution, exclusivity, reconnect), PR #216 (packages/connector-netdocuments, extracted here)

Revision R2 — what changed after the second review

The REST client is no longer written from scratch: PR #216's verified closed package is extracted and gains one upload primitive, and its allow-list guard travels with it (ponytail: a second REST dialect would have been the largest avoidable diff in the plan). The poller no longer calls NetDocuments: it goes through a connectors-api watch-list proxy, so no lane puts a DMS bearer in a worker. Probes are two separate scripts run in sequence with one runbook owner. Lane caps are recomputed from enumerated files, including review-binding.ts, run.ts and the real matter path. Enrolment, connector generation, the claim lease and the two-key limiter reservation each get an owning lane. The watch log branch leaves phase 2 entirely. The retiring-assertion list is enumerated. C-core no longer amends the pin test (the existing pin already forbids what we wanted to forbid, and naming the literal would break it on itself). Every guard now names the mutation that would fail it.

0. Gate and order

Nothing starts before spec §8.1 and §8.2 are answered by HSE, phase 0 included (it uploads into the firm's DMS). Lanes inside a phase never share a file. One lane touches packages/db/src/schema per wave. Every lane: bun run lint, bun run typecheck, the suites of every package it touched, then dual adversarial review (fresh omp gpt-5.6-sol high + a fresh claude reviewer on Opus, no implementation context, a named mutation for every assertion it adds or touches, deletion audit), two rounds then BLOCKED. bun install after any rebase. Workers: omp --approval-mode yolo for TypeScript, schema, routes, loops and tests; claude --dangerously-skip-permissions --model opus panes for UI, copy and tool descriptions. The orchestrator writes no product code. The factory spawns lanes on kwiss's word.

Phase 0 — probes (omp, throwaway scripts, deleted after; run sequentially, one runbook owner)

P-write (≤ 150 lines). apps/connectors-api/scripts/netdocs-upload-probe.ts, never wired, run once by hand under a token minted for the partner's connection, into the scratch folder the partner designates. An authorized write into the firm's DMS: kwiss says go in the pane; the partner owns cleanup (North cannot delete). Exercises W-1, W-2, W-3, W-4, W-6, W-7, W-5b, and captures the sanitized DMS audit entry for the upload (actor, application, action) so §8.1 states facts. A 403 on the first upload means W-3 failed: a re-consent lane precedes phase 1 and this plan is revised.

P-read (≤ 150 lines). apps/connectors-api/scripts/netdocs-read-probe.ts, GET only: ancestry operations for W-8; a folder listed twice while a colleague adds a document, for P-2; a folder with a subfolder for P-5; one hour of the repository audit log for P-6. Runs after P-write; the same worker appends both sanitized sections to docs/connections/netdocuments-live-oauth-runbook.md.

Q — the data question (≤ 30 lines, read-only, no schema). A SELECT-only query against preprod through OWNER_DATABASE_URL counting HSE's netdocs_mcp rows by reach and status, so §8.3 asks HSE an exact question ("the partner alone" or "the partner plus N"). Reported in the lane pane, not committed.

Phase 1 — filing back

Lane S — the shared REST client (omp, ≤ 400 lines of new code incl. tests; the extraction diff is mechanical and counted separately)

Files: extract PR #216's packages/connector-netdocuments onto main (its client.ts, auth.ts, cabinets.ts, folders.ts, download.ts, error-mapping.ts, errors.ts, ids.ts, config.ts, profile.ts and tests; walk.ts and the ingestion pieces stay out), add upload.ts (the probed multipart primitive), identity.ts if the extraction does not already carry GET /v1/user/info, and allowlist.test.ts (the guard: identity, container listing, document read, upload — a fifth shape fails; no other package constructs the client with a bearer). Rate-limit integration stays where the client is constructed, not inside it.

Mutation for the guard: add a fifth call shape ⇒ test fails; move the client construction into another package ⇒ test fails.

Lane A1 — filing schema (omp, ≤ 450 lines hand-written incl. tests; generated migration and snapshot not counted; sole schema toucher)

Files: packages/db/src/schema/netdocs_filing.ts (user_netdocs_filing_target, netdocs_filing, user_netdocs_enrolment, exactly spec §3.1, §3.3, §4.4; organization RLS in the user_drafts_folder shape, owner predicates in queries, never a user-GUC policy), packages/db/src/schema/connectors.ts (add consent_generation), packages/db/src/schema/index.ts, one migration from bun db:generate, packages/db/scripts/provision-roles.ts (spec §6 grants verbatim: app_role SELECT-only on netdocs_filing; sync_role INSERT/UPDATE there, SELECT on targets, watches and enrolment; no DELETE anywhere), tests: owner predicate proven from both the app-role and the sync-role paths, invocation_key UNIQUE, grant assertions (an app_role INSERT into netdocs_filing throws).

Mutations: drop the owner predicate from a query ⇒ test fails; grant app_role INSERT ⇒ test fails.

Lane B2 — the write boundary (omp, ≤ 500 lines incl. tests)

Files: apps/connectors-api/src/lib/netdocs-filing.ts (new: target resolution with the owner predicate, live container re-resolution, byte decode with recomputed digest and length, the three-step claim/upload/settle protocol with the lease, the stale-pending sweep), apps/connectors-api/src/routes/netdocs-mcp.ts (POST …/filing-targets, DELETE …/filing-targets/:id, POST …/file, and the custody hooks that bump consent_generation, invalidate targets and pause watches inside the token-replacement and severance transactions), apps/connectors-api/src/env.ts, tests against a mocked host: success; same key same payload returns the stored row and uploads nothing; same key different payload ⇒ 409; two concurrent requests ⇒ one claim, one upload (real Postgres); digest or length mismatch ⇒ 409 before any mint; live container renamed or re-parented ⇒ refusal; abandoned claim swept to unknown; in-flight claim receipt; enrolment missing ⇒ refusal; non-owner ⇒ refusal.

Mutations: remove the recompute ⇒ the altered-bytes test fails; move the claim inside the upload transaction ⇒ the abandoned-claim test fails; return instead of refusing on a payload mismatch ⇒ that test fails.

Depends on A1 and S.

Lane C-core — the tools (omp, ≤ 500 lines incl. tests)

Files: packages/agent-runtime/src/tools/netdocs-file.ts (new; spec §3.2 steps 1–6 client-side: credential, enrolment, resolveSourceBytes with origin, binding construction and digest, the strict interrupt, the boundary call), packages/agent-runtime/src/tools/netdocs-filing-targets.ts (new read tool, product chat and shared def data), packages/agent-runtime/src/tools/netdocs.ts (both join netDocsTools), packages/agent-runtime/src/review-binding.ts + its index.ts export (new ServerReviewBinding variant), packages/chat-runtime/src/resume-guard.ts (hasStrictBindingShape case + the binding comparison), packages/analytics/src/index.ts (netdocs.file_settled), tests: packages/agent-runtime/src/tools/netdocs-capability.test.ts (both new tools named; absent without the feeder, from replay, automation and desktop), resume-guard tests (wrong toolCallId; rebuilt binding differing in each field; a different participant re-executing without their own credential), background-toolbelt enumeration.

Not touched: the pin test (the existing pin already fails any new reader of the pre-approval literal, and it deliberately assembles the token at runtime, so naming it would break the test on itself).

Mutations: make the interrupt kind non-strict ⇒ the fall-through test fails; skip the binding comparison on resume ⇒ the rebuilt-binding test fails.

Lane C-surface — MCP defs, family registry, prepared audit (omp, ≤ 400 lines incl. tests)

Files: apps/mcp-server/src/tools/netdocs-file.ts (confirmed tier; render required, prepare resolving the binding, issuing the deterministic invocation key and returning the closed audit payload), apps/mcp-server/src/tools/netdocs-filing-targets.ts, apps/mcp-server/src/chassis.ts (merge the prepared audit payload after preparation, with a content-field guard), apps/mcp-server/src/access.ts (netdocs_write: "owner-write"), apps/mcp-server/src/tools.ts (the union registry; the pilot gate applied in the write loop; syncNetDocs toggles the union), apps/mcp-server/test/netdocs-tools.test.ts (replace the "no north__netdocs_* in the write catalogue" assertion with the union invariants; add the retry-collision expectation). C-surface owns the description constants; D delivers copy as review input, not as a file C-surface waits on.

Mutations: drop the gate from the write loop ⇒ hiding test fails; drop the def from the union ⇒ fails; return content in the prepared audit payload ⇒ the guard fails.

Depends on C-core and B2 (the route contract is pinned by S and B2 before C-core builds against a typed interface; C-core's interface is the contract, and no HTTP stub exists in either lane).

Lane D — chat UI, settings, enrolment, copy (claude opus pane, ≤ 450 lines)

Files: packages/chat-ui/src/chat/confirm-tools.ts + netdocs-file-widget.tsx (card from the binding: live folder identity, recorded path with its date, filename, size, source title and origin), apps/web/app/(settings)/connections/ NetDocuments detail (filing destinations, watched folders list, the non-owner hint), the enrolment screen and its server action calling the enrolment row, the connect-page copy replacement, the server-instructions clause, the retired-by notes on federation §3.3, §7 headline, layers 2 and 3, and §9's exclusions. Semantic tokens only, shadcn primitives, DESIGN.md.

Phase 2 — folder watch

Lane A2 — watch schema (omp, ≤ 400 lines hand-written incl. tests, sole schema toucher)

packages/db/src/schema/matter_netdocs_watch.ts (matter_netdocs_watch, netdocs_watch_item, netdocs_watch_event, exactly spec §4.1), index.ts, one migration, provision-roles.ts (app-role DML under owner predicates; scheduler SELECT on scan columns, SELECT/UPDATE on delivery columns), tests: owner isolation on all three, the paused-reason and coverage CHECKs, and that scheduler_role cannot write snapshots.

Lane E0 — lifecycle backend and the watch proxy (omp, ≤ 450 lines incl. tests)

Files: apps/connectors-api/src/routes/netdocs-mcp.ts + lib/netdocs-watch.ts (POST …/watch/list: takes { organizationId, watchId, skipToken? }, re-derives the watch, revalidates connector, generation, reach, membership, account status and matter access, reserves in the two-key limiter script, performs the one allow-listed listing call, returns items and the next token; never accepts a user id), packages/connector-netdocs-mcp/src/redis-limiter.ts (the two-key reservation in one script, ≤ 60 lines), apps/web/app/(app)/matters/actions.ts and apps/web/app/(settings)/connections/actions.ts (register, pause, resume, delete, calling the custody hooks B2 owns rather than duplicating them), tests per transition and for a watch id that does not belong to the caller's org.

Lane E — the poller (omp, ≤ 450 lines incl. tests)

apps/worker-connectors/src/loops/netdocs-watch-poller.ts (spec §4.2: scan predicate including due lockout and reauth, revalidation, proxy calls, snapshot diff, coverage handling, three-partials pause, persisted backoff, per-watch lag metric, the 30-day sweep under full coverage only), apps/worker-connectors/src/index.ts and env.ts (connectors-api URL and internal token — deploy machinery, kwiss's word before merge), deploy/preprod unit env. Tests with fake timers and a mocked proxy: first listing silent; new/updated per item; ties; partial writes nothing; three partials pause; lockout backoff survives a restart and recovers; each pause reason; membership loss; sweep does not age a partial snapshot.

Listing only. The audit-log feed is lane L, gated on its own spec.

Lane F — the notifier (omp, ≤ 300 lines incl. tests)

apps/worker-notifications/src/loops/netdocs-watch-digest.ts (claim with SKIP LOCKED, re-check membership and banned state inside the claim, send through src/newsletter/mail.ts with an idempotency key from the claimed event ids, confirm, reclaim at 15 minutes, cap 5), apps/worker-notifications/src/newsletter/mail.ts (two templates: document batch, and watch-state notice naming the reason), apps/worker-notifications/src/index.ts. Tests: dev outbox; crash between send and confirm resends once under the same key; attempts cap; banned owner not emailed.

Lane G — matter page and connection page (claude opus pane, ≤ 350 lines)

apps/web/app/(app)/matters/ (owner-only watch section: add by URL/id with the §4.4 copy verbatim, pause/resume/delete, last 20 events with their stored links, paused states with reasons), the connections detail watched-folders list and the disconnect dialog line. Calls E0's actions only.

Phase 3 — readout (lane R, docs)

After a week on preprod: filings by outcome including unknown and swept claims; watch coverage, lag, pauses by reason, events and emails; limiter denials and lockouts; whether W-5b, W-6, P-2, P-4, P-5 held. Output a docs/reviews/ page and flip the spec status. Firm-shared watches, the log feed and subfolders are decided there.

Gates per lane

• bun run lint, bun run typecheck at the root.
• Suites: S @workspace/connector-netdocuments; A1/A2 @workspace/db; B2/E0 connectors-api; C-core @workspace/agent-runtime, @workspace/chat-runtime, @workspace/analytics; C-surface mcp-server; D @workspace/chat-ui, web; E worker-connectors, @workspace/connector-netdocs-mcp; F worker-notifications; G web.
• bun run eval:gate:cheap for C-surface and D.
• Guards that must exist and bite, each with the mutation named in the PR body: REST allow-list (S); owner predicate on both roles and the app-role ledger denial (A1); recomputed digest, claim outside the upload transaction, payload-mismatch refusal, stale-claim sweep, live-container refusal (B2); strict binding and rebuilt-binding comparison, capability hiding of both new tools, background-toolbelt exclusion (C-core); write-loop gating, union membership, prepared-audit content guard, MCP retry collision (C-surface); watch-proxy refusal of a foreign watch and of a user id (E0); owner isolation on the three watch tables and scheduler write denial (A2); partial listing writes nothing, sweep skips partial coverage, lockout recovery (E); no double email across a crash, banned owner skipped (F).
• Live behaviour no mock can establish (ancestry, cursor, recursion, log fields, rate, provider idempotency) is settled by the phase-0 probes and re-measured in the readout; no acceptance criterion claims what only a live probe could prove.

Deletions and retired assertions to audit in review

• apps/mcp-server/test/netdocs-tools.test.ts asserts today that no north__netdocs_* def appears in the write catalogue: that assertion is deleted by C-surface and replaced by the union invariants.
• Federation spec §7 retires in three places: the headline "demonstrable inability to write", layer 2 (the sole-REST-call claim, which was never a test) and layer 3 (readOnlyHint: true and read-catalogue-only registration). §3.3's "technically unable to write" sentence and §9's two exclusions (any write scope; background use of a lawyer's token) also retire. D writes all of them as retired-by notes.
• The connect-page sentence changes (D).
• packages/db/src/schema/connectors.ts gains a column (A1) — reviewers check no other connector semantics move.
• Everything extracted in lane S is a move, not a rewrite: the reviewer diffs the extracted files against the #216 branch and flags any behavioural edit that is not the upload primitive.
• Any other deletion in a lane's diff is a finding.

Shortcuts the lanes must name, not absorb

Watch and filing are tested against mocks; the live facts come from the probes and the readout. The recorded container path is a dated snapshot until W-8 lands; the card says so. unknown filings need a human to look in the folder; there is no reconciliation read in v1 (W-7). The two-key limiter reservation is new code in a package other lanes read; E0 owns it and states the blast radius in its PR.

Status tooling

bun docs/superpowers/status.ts set 2026-09-07-netdocs-filing-and-watch in-progress in lane S's first commit; implemented only after phase 3's readout.