Cheaper classification — design

Use GLM for thin mail classification and Gemini Flash Lite for document classification. Accept occasional semantic misses, not weaker privacy or exclusion controls. Implemented in f09db4d1; unmerged, undeployed, PR CI pending.

Implementation plan · Author: @write, Astra high.

Decision and evidence

Retained evidence: /tmp/cheaper-classification-eval/alternate-continuation-final-report.md, doc-eval-final-report.json, and doc-eval-metadata.json in that directory. These are synthetic evaluations, not production parity or invoices.

• GLM mail: 31/36 scored; actions 16/16; private 4/4; one waiting request unscored. Matched Sonnet: 32/36. Approximately USD181 versus USD3000 mail spend is a projection, not measured deployment savings; missing charges remain unknown.
• Fresh documents, twelve cases twice: Gemini 23/24 joint-exact, Kimi 21/24, GLM 20/24. Gemini returned cost USD0.0030722 versus Kimi USD0.01574874: 80.5% cheaper measured versus Kimi, not the incumbent. Applying current Haiku batch tariffs to Gemini’s same tokens estimates 82.5% savings; incumbent usage was not measured.
• The old report’s perfect 24/24 gate is superseded by kwiss’s acceptance of occasional misses. Gemini’s round-two spreadsheet, matter → needs_review, is explicitly accepted; keep its original label and report the mismatch. Unknown broader-scope or private exposure is not approved.

Lane-specific client-data approval

This is the written lane addendum to the OpenRouter §4.5 client-data standard: kwiss’s 2026-09-10 approval covers only these classifier/model/host combinations. Record this sign-off beside each local route; changing a model, host, or data surface requires fresh written approval. No global registry entry or chat eligibility is implied.

Both use synchronous https://openrouter.ai/api/v1/chat/completions, temperature 0, response_format: {type: "json_object"}, usage inclusion, and shared OPENROUTER_PROVIDER_POLICY: collection deny, ZDR true, fallbacks false, plus only the exact host. No prompt logging, tools, plugins, alternate provider, or batch API: its 30-day retention is unacceptable for these new routes. Existing other-stage transports remain unchanged.

Metadata captured 2026-09-09T21:33:56.490Z matches both model ID and tag in the endpoint and ZDR lists: Wafer | z-ai/glm-5.3-flash-20260826 / wafer, and Google | google/gemini-2.5-flash-lite / google-vertex/eu. Orchestrator rechecked direct public /api/v1/endpoints/zdr on 2026-09-10: both exact model_id/tag pairs status 0; prices below confirmed. Reader proxy omitted Gemini; direct API resolved it. Absence blocks, never widens routing. Model origin offers no security guarantee. ZDR is neither proof of EU residency nor an independent audit; returned “Google” does not attest region.

Small cutover

Mail: replace the Anthropic SDK request inside classifyMailThin with the existing direct-fetch pattern from knowledge’s callOpenRouter, locally, not a shared abstraction. Keep the prompt, 8000-character body cap, MailClassificationSchema, and complete needs_action/confidence-zero/firm_shareable-false fallback. Migrate ThinClassifierCtx, its public barrel surface, processor, both mail evaluation callers, and affected tests. Keep Anthropic for unrelated mail stages.

Knowledge: use callLlmForStage → modelFor → callOpenRouter; change CLASSIFY alone. Pin its default/configuration and settings locally, rejecting unapproved classify overrides. Never apply Gemini reasoning-off globally. Preserve prompts, exact scope values, validation, filenameHeuristic, and downstream policy handling; no spreadsheet special case.

Keep mail’s legacy sonnet stage strings, deterministic hash inputs/IDs and DB enum deliberately stable: they identify persistence stages, not the actual provider. Record requested/returned model, provider, response ID and usage through operational logs/accounting; no schema migration.

Failure and cost contract

These classifiers make one HTTP attempt, with a 120-second deadline covering headers and body consumption; knowledge’s new deadline applies only to CLASSIFY. Recognize HTTP200 embedded errors, 429, 5xx and body timeout. Mail transport failures propagate to the existing source-email job retry: five attempts, exponential 5000ms base (packages/source-events/src/bull.ts). No nested retry. Knowledge currently catches call failures in classifyDocument; preserve its visible heuristic fallback, not a fictitious queue retry. Missing key or unapproved route must instead fail startup and direct-entry preflight before that catch can silently classify a corpus.

Malformed, missing or truncated completion content receives conservative classifier fallback. Provider/model identity mismatch fails closed before accepting output: expect returned z-ai/glm-5.3-flash / Wafer for mail, as evidenced by retained scored responses in doc-eval-results.json, and google/gemini-2.5-flash-lite / Google for documents. The dated GLM endpoint metadata ID above is not the completion model field. Never compare display names to request endpoint slugs wafer/google-vertex/eu. Log safe error codes/status/stage only, never email, filename, prompt, model output or upstream error text; remove those existing leak paths.

Use returned finite usage.cost when present; otherwise label token-priced estimates explicitly. Gemini’s captured USD/MTok input/output is 0.10/0.40; GLM’s is 0.10/0.35. No batch discount on sync calls; reasoning tokens are already output. Extend existing stage snapshots with cost provenance and unknown-call counts; unknown charges cannot become zero or a complete total. Preserve known subtotals and mark incomplete totals, without a billing framework.

Bounds and rollback

No schema/migrations, reclassification, shadow runs, cascade, generic provider framework, model-catalog addition, billing framework, UI, retrieval, facts/clauses/rules/relations changes, prompt tuning or golden relabeling. Roll back via the previous deployment and its configuration; do not retain a dead mail adapter or compatibility shim.

Implementation evidence — 2026-09-10

Retained /tmp/cheaper-classification-eval/implementation-report.json and implementation-results.json: approved paid --run invoked once through real classifiers, 49 scored, no retry. Mail chips 33/37; actions 16/16; all field/private checks passed. Documents: type, exact scope and exclusion each 12/12; zero fallbacks across all 49. This separate smoke does not erase the historical two-round Gemini 23/24 exception above.

All 49 returned actual costs: mail USD0.0084061 + documents USD0.0015591 = USD0.0099652. Aggregate reservations USD1.456302270/10 include retained unknown earlier calls; reservations are not money spent. Actual/estimate/unknown accounting remains distinct. GLM/wafer mail and Gemini/google-vertex/eu document CLASSIFY changed; other stages, prompts, goldens and DB IDs remain unchanged. Local gates and review evidence follow in the implementation plan.