Skip to content

fix(stateless): contain watch loss and preserve Desktop chats - #849

Merged
alfredolopez80 merged 114 commits into
devfrom
fix/stateless-incident-containment
Sep 30, 2026
Merged

alfredolopez80 merged 114 commits into
devfrom
fix/stateless-incident-containment

Conversation

@alfredolopez80

@alfredolopez80 alfredolopez80 commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

PR A contains the immediate #396/#826 incident containment on top of current dev.

  • HCC treats unsynchronized CommunicationChannel authority as an authority loss and holds the Host instead of acting on partial data. The durable lifecycle state (active, suspended or draining) is preserved, not rewritten to active; the effective view is flagged suspensionBlocked, so no new suspend or drain is issued while the channel inventory is unknown. The existing stateless Deployment template, mounts, PVC, and SQLite path are kept while reconciliation recovers; HCC does not replace them with a fallback layout. The cache-derived effective view (effectiveLifecycleFromCache) still reports state: 'active' for a held Host that has no persisted state yet. Confirmed channel, Desktop, and placement incompatibilities continue to follow the existing rejection policy.
  • A stateless Host that is rejected (active CommunicationChannel, spec.desktop, unsatisfiable SharedFileSystem co-location) or that has stateless disabled is forced to durable state=active with one replica, including when it was suspended or draining, so a rejected Host never sits suspended without an operator-visible condition. This is documented in the assessLifecycle contract.
  • HCC wake/provisioning preserves the trusted zero-replica stateless template, and the lifecycle tests cover two authoritative recovery cycles without template churn. The selective watch-loss fixture leaves unrelated Kubernetes operations available so the test exercises the actual authority-loss path.
  • Desktop keeps selection, transcript, and composer state across ambiguous 404, 503, timeout, offline, restart, and cross-Host journeys. A deletion confirmed by the server removes the chat locally and records a local deletion marker. Only a 403 that rpc-proxy marks as host_access_revoked clears cached remote data. Any other 403 is treated as uncertain: persisted local data is kept, but the Host is hidden and deselected until it is verified again. Uncertain Hosts are hidden rather than shown because, until the mint 403 carries a revocation code (host access: return a machine-readable revocation code on the RPC token mint 403 #941), a real revocation reaches Desktop as uncertain whenever no cached RPC token exists for that Host; showing it would expose cached transcripts of a revoked Host in the most common case. Revocation is confirmed only while a cached token younger than 5 minutes exists for that Host (option B); option A is tracked in host access: return a machine-readable revocation code on the RPC token mint 403 #941. Late notifications and task results cannot resurrect a deleted chat or a chat of a revoked Host in the same team scope.
  • rpc-proxy prewarms and holds stateless Host requests without treating “wake accepted” as “runtime Ready”. Finite upstream deadlines, retry producers, and wake admission are covered by deterministic tests. Its own authorization denials keep status 403 and the existing error text and add a machine-readable code: host_access_revoked for team_membership_missing and directory_grant_missing, and host_access_denied for every other own denial (host_disabled, host_missing, missing claim, subject_mismatch, 404 without a direct-run binding, a userId or hostRef mismatch in the lookup response, empty URL, unknown reason). Passthrough 403 bodies from mcp-host never get a code.
  • mcp-host lazy tool registration reduces cold-start work while retaining full tool behavior after startup. The workflow-trigger intent recovery accepts a direct request for a recipe name (run daily-report). Direct recipe triggers accept a trailing ., !, ?, ,, ; or : on the recipe name and the filler words now, again, please, immediately; any other tail is not a trigger. It stays linear-time on adversarial input (a 200k-character test asserts it finishes under 1000 ms) and still rejects shell-like text and tokens over 128 characters.
  • Control UI can create stateless Hosts explicitly. This change is deliberately part of this PR and not a separate one: the hold/wake, rpc-proxy and Desktop changes above only take effect on a stateless Host, so the merged code cannot be exercised on the clerum-dev cluster until an operator can create one from the UI. Shipping the UI change here lets the team test the whole path on dev right after merge, without opening a PR whose only purpose is to expose the creation flow. The change is limited to the explicit creation option and its payload/copy; it does not change how existing Hosts are created or reconciled.
  • The T1 harness reports failed Vitest assertions and reporter counters instead of only a stable failure code. A suite that fails before any test runs (for example a beforeAll or import error) has no assertion results, so its file-level message is now printed after the FAILED FILE: line. The reporter redaction keeps the JSON valid when a DSN appears inside a JSON string (it no longer consumes the escape that closes the string) and redacts the branch database password verbatim, JSON-escaped once and twice in both ensure_ascii spellings, and percent-encoded by Python and JavaScript. The macOS stress suite bounds its HTTP agent below the kernel accept queue, and the 150k-deep fixture is verified by an iterative depth measurement instead of a V8 build-specific serializer stack limit.

PR B remains responsible for the durable DB migration from #825 and the progressive catalog from #827. Local Desktop deletion markers are not server-side cross-device tombstones.

Exact-head verification

  • Head certified below: 051cefaa84a18d9dbb6e9573a229c39fb761ea28, the current PR head (review-fix round 3 below). Every result in this section was produced on that exact head.
  • Base: origin/dev aaa85813009d0748735e2edfb907865c269969d1, which includes feat(files): add FileReference v1, inline file attachment ingress and structured GFS references #822 (file reference ingress); it is an ancestor of the head (checked by the launcher before the run and again after it).
  • GitHub checks at 051cefaa8: 42 of 42 check runs completed with conclusion success (verified with gh api on the commit). The previous certified head 65bc16c26 also had 42 of 42 and a MINIKUBE_T2_PASS.
  • Merge state is MERGEABLE and BLOCKED only by the required review.

Canonical local Minikube T0/T1/T2 certification on the branch-owned profile/context, run with make minikube-t2 (each figure read from the run log and the harness evidence.json):

MINIKUBE_T2_PASS
T0=PASS
T1=PASS
T2=PASS
NP08_HCC_AUTHORIZATION=PASS
PortForwards=PASS
Health=NOT_RUN
Playwright=NOT_RUN
T2_EXIT=0
  • T0 (1101 s): ShellCheck, affected package suites/build/typechecks, harness contracts, image provenance/lease safety and public-boundary checks. The worktree stayed clean after the T0 builds. Suite counts: control-api 6,611 passed (501 files, 361 skipped tests in 63 skipped files); control-ui 2,358 (214 files); Desktop 3,102 (288 files); host-context-controller 2,603 (115 files); mcp-host 5,284 plus one known todo (422 files); rpc-proxy 789 (48 files); external-rest-api 164 (30 files).
  • Reconcile and pre-gate-sync: the profile had been stopped and was started with the branch-profile helper; the planner selected full-reconcile (200 s), then pre-gate-sync (724 s) ran the package suites and renewed the port-forward hold before the final preflight.
  • T1 (139 s): Real PostgreSQL executed 373 of 373 tests across 65 of 65 expected files with one worker, no pending tests and no silent skips; control-api isolated suites 63/63 files and 364 tests.
  • T2 and NP08: exact marker, image, PostgreSQL, namespace, Service and deployment readiness passed (10 s), and the deployed same-Context, cross-Context, caller-token and credential-disclosure authorization journey passed (4 s).
  • Health and Playwright are separate evidence lanes and were not supplied to this run, so they are NOT_RUN; they are not part of this verdict.
  • The certification evidence now closes the preflight phase (preflight PASS after the planner), per 086648635.
  • This is the first run on 051cefaa8. Earlier attempts on the previous head 65bc16c26 failed for reasons outside the product code and are not counted: a random-state false positive in the unrelated control-api/test/services.grokSubscriptionOAuth.test.ts leak assertion (its assertNoLeak regex is case-insensitive and can match the random base64url state; tracked in test(control-api): grokSubscriptionOAuth assertNoLeak fails at random on the OAuth state #930), a dirty tree caused by next build rewriting profile-ui/next-env.d.ts (fixed in this PR), and stale node_modules in workflow-recipes on the local worktree (reinstalled with npm ci).

Review-fix round (head 65bc16c26, after a696391a6)

Thirteen commits since a696391a6 address the round-1/round-2 review findings and the findings found while verifying them, plus two merges of origin/dev (713391c36, then aaa858130 with #822). Each fix ships a test that fails when the fix is reverted (mutation or revert proof), and each negative assertion has a liveness witness.

  • HCC: held-runtime OAuth-server scope taken from a live McpServer read (see Known deferred boundaries), PVC/Service errors thrown before the hold branch, readiness polled after a held wake, verified applied runtime required before waking during inventory loss, bootstrap bound to the Deployment UID and runtime-token revision. A stale McpServer cache now fails closed instead of being trusted, and a running suspended runtime is held rather than acted on (6df1be439).
  • rpc-proxy and control-api: control-api sends x-host-access-denial-reason (body unchanged), rpc-proxy maps it to code; wake hold default 48000 ms in both; artifact download deadline; retry only for /messages; 1 s admission floor. rpc-proxy strips a reserved code from a passthrough mcp-host 403 (ce53e25ec). rpc-proxy never re-sends a mutating POST after the response headers arrive: a body failure after the headers returns 502 without a second request. A connection error before the headers is treated as a stopped Host, and approve, deny, set-model and cancel are sent once more after the wake. The first mutating attempt has no client-side timeout by design; it is aborted only when the calling client disconnects. Deploy control-api before rpc-proxy.
  • Desktop main: 403 classification from code only; tombstone sidecar; cleanup retry by environment and user; home team pinned during a team hop.
  • Desktop renderer: catalog merge and cursor recovery, host authority hold, single me.teamId source, toast dedupe, and the chat store is rebound after a failed team restore (13f5aa0f0).
  • mcp-host: direct recipe triggers accept a trailing ., !, ?, ,, ; or : on the recipe name and the filler words now, again, please, immediately; any other tail is not a trigger.
  • T1 harness: file-level failure messages, escaped DSN redaction, more password spellings redacted while the reporter JSON stays valid (f3d709789), reporter contract test registered.
  • Merge of aaa858130 (nine conflicts): package versions kept at this branch's higher values (hcc 0.1.368, mcp-host 0.1.473, rpc-proxy 0.1.102, with releaseManifest.ts); the stress test takes dev's maxSockets: 100; useAgentChatController builds the feat(files): add FileReference v1, inline file attachment ingress and structured GFS references #822 file references before the send clears the composer and keeps this branch's chat-store rebind and deletion fences; the sendAgentMessage.fileReferences accepted-ack witness reads upsertMessages, which is how this branch persists the user turn on an async ack.
  • profile-ui: the tracked next-env.d.ts is now the production variant (./.next/types/...), which is what next build writes, so the T0 build no longer dirties the tree and the exact-head preflight no longer stops with DEVELOPMENT_SCOPE_REQUIRED (cc111db4d); this follows control-ui, which already tracked that variant.

tsc --noEmit and the package builds are part of the T0 run above; the Desktop UI project still reports only the three pre-existing errors in ui/src/lib/imageHeaderDimensions.ts.

Review-fix round 3 (head 051cefaa8, after 65bc16c26)

Thirty commits address the round-3 review (comment 5896495722) and the findings of two follow-up audits of every review comment on this PR. Each fix is test-first: the test was red before the fix, and a vacuity mutation turns it red again.

  • HCC: oauth:user-token is deferred instead of downgraded when the McpServer read is unobservable (2327789ed); a fresh bootstrap is consumed on a GFS-only refresh when the Deployment runs its revision (7eed4e73c); a Ready runtime stays deployed when the held GFS renewal fails (98685c9d9), and that failure is reported as telemetry, not as a failed administrative intent (6ec74d72d); the draining revert requires a stateless fresh read (e073d00b4); the runtime credential fixture is derived from a real mint pass (aafb50229).
  • rpc-proxy: idempotent GETs retry until the wake-hold deadline (a9a11cd0e); the deadline is pinned before host resolution (51ea9a154); mutating upstream calls are aborted when the client disconnects (ab9240721), including a client that left during host resolution (2279431a5) and the session rename (70b7ef4ec); task cancel refuses an empty caller subject (f0dab25a6); subject_mismatch is a denial, not a revocation (cd8e11598); cross-service guard for the host-access denial contract (ba864331d).
  • Desktop main: the tombstone sidecar is written before the chat index (0432fdbcd, tests in df3286049); pending chat cleanups are swept per agent on read and once per bind (48e3b7fbe); over-long session cursors are dropped and 408 is no longer terminal (a8e36982c).
  • Desktop renderer: the session catalog cache is invalidated when a Host is held (30ecd0dfe), and a late rejection of an invalidated request keeps the newer entry (b29077dae); IPC-shaped 403 fixtures with same-scope controls (777278c0f); the unreachable rename alternative is removed from the revoked matcher (62f6968f0); the chat fixture delegates to the real ChatStore (f48785b21); the catalog merge composes over disjoint pages (051cefaa8).
  • mcp-host: trailing comma, semicolon and colon on direct recipe triggers (07c16b0fb); an invalid sessions cursor is rejected (7e56e5fc7).
  • Scripts and CI: non-ASCII passwords are redacted in both JSON escape spellings (15dd1ba6c); the preflight phase is closed in certification evidence (086648635); the autoresearch gate self-test runs in CI (c6690eac2).

Review items resolved without a code change:

  • R3-L5 refuted: docx/pdf are binary formats and skip redaction; text redaction is bounded at 50 MB and measured at 1.2 s for 100 MB.
  • R3-L4: the rename PATCH keeps the 8 s first-attempt timeout because it is idempotent and Desktop retries it; since 70b7ef4ec a client that leaves also releases the upstream call.
  • R3-L10: 409 stays terminal for load-more because no producer on the sessions path emits it.
  • R1-L7: fixtures now import the HOST_ACCESS_* constants; remaining literals in untouched tests are accepted.
  • R2-L5: versions bumped in 65bc16c26 and by every commit here.
  • Catalog merge composability (round-3 note): the property holds for disjoint pages, which is what cursor pagination produces. With overlapping pages a later report updates the title while updatedAt stays from the first report; this is intended and documented in the test.

Follow-ups

Size

Diff against origin/dev: 202 files, +21345/−1866 (desktop-app 84, rpc-proxy 36, host-context-controller 22, scripts 20, mcp-host 15, control-api 8, control-ui 6). The change spans every hop of one incident path (Host hold in HCC, wake-and-hold in rpc-proxy, revocation handling in Desktop) and the certification harness that proves it; splitting by service would ship a hold without its client behavior or a client behavior without its server. The autoresearch gate scripts were merged from dev and are not part of the change.

Security review and CodeQL

  • Codex Security and multi-agent review cycles covered HCC authority loss, proxy wake/hold behavior, Desktop chat/revocation/deletion state, harness ownership, and the final delta. Confirmed material findings were fixed; intentional non-blocking/deferred items are tracked separately rather than described as fixed here.
  • CodeQL alerts 1558, 1559 and 1599 to 1602 are fixed at this head (confirmed by review). The later 1605 useless-conditional finding is reported closed by the author. The intent-recovery change in this round keeps the \S{1,128} bound, uses non-overlapping character classes, has no nested quantifiers, and checks the tail with string operations instead of a second regular expression.

Known deferred boundaries

  • While CommunicationChannel authority is unavailable, HCC intentionally preserves the last trusted template/scope identity so it does not recreate the Host from partial authority. An administrative scope/mount/template revocation that cannot be observed authoritatively during that window is therefore a containment tradeoff, not silently claimed as applied; authoritative observation resumes after cache recovery. During CommunicationChannel cache loss the OAuth-server scope is taken from a live McpServer read; when that read is unavailable the retained false is used, and a retained true is neither re-granted nor revoked until the read succeeds or the credential reaches its renewal window, at which point HCC mints fail-closed without the scope. hasChannelIngress during the hold still comes from the retained annotation, and GFS-only renewal does not consult OAuth; both are the deferred boundary tracked in hcc: re-observe channel ingress and refresh the wake template after a CommunicationChannel cache outage #942. A failed GFS renewal during the hold keeps the runtime deployed:true and reports the failure in the status message.
  • During CommunicationChannel cache loss, a Host with no retained scope annotation (every Host right after this deploys, and every new Host) reports ready:false with 'Waiting for authoritative scope observation', even when its pods are Running; the first synced mint writes the annotation and the status clears. This is the fail-closed choice pinned by the B2 integration test.
  • The short team-hop UI notification race remains a follow-up (desktop-app: short team-hop notification race in the renderer chat path #945). Durable main-process suppression still prevents the known unauthorized write/reveal path, so it does not gate this PR.
  • Generic auth denial remains deliberately interaction-blocking for this PR. Treating “uncertain” differently from “confirmed denial” beyond the 403 code contract above is a separate product-contract change.
  • Local Desktop deletion markers remain local until stateless(storage): preserve one canonical conversation store across lifecycle transitions #825/PR B provides durable server-side deletion state. They are intentionally never pruned: a deleted chat is never reopened, and a cap or expiry would let a late write resurrect it (about 209 bytes per entry). The delete confirmation requires the same environment, user and team. The pending-cleanup retry identifies a deletion by environment and user only, so a cleanup left pending in one team completes after the same user switches team; a different user or environment never retries it. The team read for the chat-list cache scope and for the fence comes from a single me.teamId source in useAppController, because Remove Team Token PR1: User session, live authorization, and aggregate all-team access #334→Remove Team Token PR2: Runtime actions use validated access paths #594→Remove Team Token PR2.1: Host-route admission security prerequisite #797→Remove Team Token PR2.2: Host-ref authority and validation #795 keep SessionMe.teamId while deprecating the team-bound session server-side; the known merge conflicts with that stack are the package manifests and lockfiles, external-rest-api/src/releaseManifest.ts, rpc-proxy/src/routes/rpc.ts and rpc-proxy/src/services/wakeAndHold.ts (plus its test). A corrupt index.json is quarantined, and the tombstones are recovered from the .tombstones.json sidecar, which is read only when the index is missing, malformed or quarantined. saveIndex writes the .tombstones.json sidecar before the index whenever the tombstone or pending-cleanup lists change, each file with its own atomic rename, and a failure in either write rejects the operation before the index is committed. No sidecar is written for an agent that never had a tombstone. A crash between the two writes leaves the sidecar a superset of the index, which a valid index ignores and the next save realigns. Every chat:getIndex retries the pending cleanups of that agent only; a full sweep runs once per authority bind. An unreadable sidecar (invalid JSON or wrong shape) logs one warning and is treated as empty; a filesystem error such as EACCES or EIO is rethrown. If index.json and the sidecar are both unreadable, no other copy exists and a chat deleted locally can be hydrated again from the server, which is not deleted server-side. Quarantining the sidecar and surfacing the loss in the renderer were evaluated and deliberately not adopted here; durable server-side deletion in stateless(storage): preserve one canonical conversation store across lifecycle transitions #825/PR B is the fix for that case. This PR does not claim that local deletion is permanent.
  • Deploy order: control-api before rpc-proxy. control-api now sends the x-host-access-denial-reason header on its 403 (the 21-byte {"error":"Forbidden"} body is unchanged) and rpc-proxy maps it to code; rpc-proxy alone would report host_access_denied for every own denial.
  • The wake hold default is 48000 ms in both rpc-proxy and the control-api rate-limit derivation (4 calls per instance, 8 per fleet, 10.0/min against the default of 30/min).
  • The T2 ownership scan is unchanged by this PR: scripts/minikube/t2-common.sh has no net diff against dev, and the earlier planner-only PID auto-clean was removed. A dead, malformed or duplicate registered port-forward record still fails the final exact-head preflight with PORT_FORWARD_CONFLICT and is never deleted; test-minikube-t2-process-owner.sh covers the dead-PID and malformed-PID cases.

Gate boundaries

Refs #824, #396, #826.

🤖 Generated with Claude Code

@alfredolopez80
alfredolopez80 marked this pull request as ready for review September 24, 2026 23:30
…, and preserve template

- HCC: keep applied template during unknown channel inventory, block suspension
  without changing deployment, and add wake capability to the containment gate
- Desktop: fence chat deletion to the issuing binding generation and authority
  scope; distinguish confirmed host revocation from uncertain 401/403; clear
  drafts and tabs on scope changes; reverify access on fresh authorization
- IPC: add scoped chat:delete contract with binding-generation fence
- Gate: extend selective hold budget and wire wake capability
- Tests: cover cold-start template preservation, cross-principal delete, and
  host authority classification
HCC now keeps the applied Deployment bound to the current Host UID, reports missing applies truthfully, and derives held readiness from the full PVC, Service, and NetworkPolicy boundary. Desktop captures an environment/user/team deletion fence, persists scoped tombstones with retryable cleanup, separates unconfirmed authorization failures from confirmed revocation, fences authority/navigation epochs, and isolates no-chat drafts by agent. The stateless cache-containment gate gains wake capability, renewable two-cycle holds, CI contract coverage, and an explicit owned-profile T2 entry point.
* origin/dev: (23 commits)
  fix(evendrive): epoch-guard breadcrumb reconstruction against stale retries
  fix(evendrive): rebuild move-recovery ancestry from live crumb state
  fix(evendrive): label the stale trail when post-move path reconstruction fails
  fix(evendrive): rebuild breadcrumb trail after a crumb move; defer link-opened mutations
  fix(gfs): align inherited access and breadcrumb versions
  revert: remove backend changes from GFS UI PR
  fix: rate limit GFS routes before auth
  fix: clear CI and CodeQL findings
  fix: resolve GFS access review regressions
  style(evendrive): use the 2xs font token for the sticky EvenDrive list header (N3)
  fix(evendrive): skip empty file-alignment grant after a successful parent update (N2)
  fix(evendrive): treat re-selecting the displayed merged role as a no-op (N1)
  docs(evendrive): unify inherited-walk semantics at both walk sites (R1-L3)
  refactor(evendrive): extract pure inherited-access merge core with property tests (R1-M2)
  fix(evendrive): keep every contributing ancestor and edit them honestly (R1-H1)
  fix(evendrive): describe and restore focus in the parent-role dialog (R1-L2)
  feat(evendrive): use hard-drive icon for EvenDrive nav and header
  fix(evendrive): surface total inherited-derivation failures in share dialogs (R1-M3)
  fix(evendrive): judge parent-role bits by the folder's affordances (R1-M1)
  fix(evendrive): revoke file's direct shares on parent-role confirm (R1-H2)
  ...
@alfredolopez80

Copy link
Copy Markdown
Contributor Author

@claude please perform a full re-review of PR #849 at head d99a0b3dfe5dbd5e877da5465979e8d559ce65fb.

CI/CodeQL is fully green (41/41 checks). Please review the complete PR and the corrections made in response to the earlier internal review, with special attention to:

  1. HCC cache-loss authority: active stateless Host hold, suspension blocking, exact template/mount/SQLite-path preservation, verified Deployment identity, and the two-cycle recovery behavior.
  2. Desktop transient-failure containment: conversation/selection/draft preservation on ambiguous 404, 503, and timeout; confirmed deletion; and immediate protection after Host access revocation.
  3. Scoped deletion and privacy: principal/user/team/environment fences, durable local cleanup, stale event suppression, tab/draft quarantine, and IPC/renderer trust boundaries.
  4. Test and CI correctness, especially whether any assertion only mirrors implementation without proving the incident outcome.
  5. Security regressions or unsafe assumptions introduced by coordination contracts or the merge with current dev.

Please report each finding with file/line references and severity. Distinguish blocking defects from non-blocking observations and coverage gaps.

@claude

claude Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Claude finished @alfredolopez80's task in 2m 27s —— View job


Full re-review of PR #849 @ d99a0b3

  • HCC cache-loss authority
  • Desktop transient-failure containment
  • Scoped deletion & privacy (IPC/renderer trust boundaries)
  • Test/CI correctness
  • Security regressions / merge-with-dev risks

Verdict: no blocking defects. The incident containment is implemented correctly and fail-closed across HCC and Desktop. The design proves real outcomes rather than mirroring the implementation. Five non-blocking items and a few coverage gaps below, ranked by materiality.


1. HCC cache-loss authority — ✅ correct, fail-closed

  • Unsynced CC cache → {stateless:true, state:'active', suspensionBlocked:true} with a StatelessEnableRejected=False / CommunicationChannelCacheUnsynced condition (statelessLifecycleExecutor.ts:227-228, :360-374). The hold path returns {...existing, spec:{...existing.spec, replicas:1}}, so the live template, mounts, env, and SQLite dir (CLERUM_SESSION_DB_DIR=/var/lib/clerum/state) are preserved byte-for-byte; only replicas is forced to 1 (hostReconciler.ts:3630-3660).
  • Suspension blocking is watertight. The two suspend/drain preconditions were converted from .stateless to canSuspendFromCache (:807, :1073), which returns false whenever suspensionBlocked. The tracker heartbeat (statelessLifecycleTracker.ts:438-442) returns drain:false and cancels drain; idle D8 evaluation is skipped (:798-801); the wake fast-path returns before the unfenced scale patch when unsynced (statelessLifecycleExecutor.ts:1410). I independently verified the two guards still using plain .stateless (:982 publishSuspendBlockedReason, :1166 cancel-drain) are move-to-active/annotate-only paths that can never scale down — leaving them is correct and precise.
  • Deployment identity verification is strong. Before preserve/scale, HCC requires ownership, name/namespace match, non-empty uid + resourceVersion, no deletionTimestamp, a template spec, and clerum.io/host-uid == host.uid; any mismatch throws unverified Deployment (no replace/patch), and the 409 retry re-reads and re-fences (hostReconciler.ts:3636-3706).
  • Two-cycle recovery is correct: a missing Deployment during unsynced cache defers creation (buildDesiredDeployment → null → applied=false) and creates it on the first post-sync cycle; the deployment boundary re-checks suspensionBlocked after the async OAuth lookup so authority lost mid-reconcile converts to hold.

Coverage gap (minor): the tracker's suspensionBlocked → cancelDrainOnEvidence wiring (statelessLifecycleTracker.ts:440) is only exercised for an idle host; no test asserts an actively-draining host is reverted to active when authority is lost mid-drain. Rollout note (by design): a legacy Deployment lacking the clerum.io/host-uid annotation encountered while the cache is unsynced fails that host's reconcile with unverified Deployment until the cache syncs — fail-closed (no state destroyed, pod keeps running, annotation re-stamped post-sync), but worth being aware of during the HCC upgrade window.

2. Desktop transient-failure containment — ✅ correct

  • Ambiguous 404 / 503 / timeout now route to WENT_OFFLINE (outcome offline) — no eviction, no FSM RESET, no draft clear (reconcileChat.ts:239-244). isNetworkError now covers 502/503/504 + timeout; httpErrorStatus/isHttp404 parse the first status token so a 5xx body that merely mentions "404"/"403" cannot be misclassified (format.ts:159-215). Composer draft is keyed by (chatId, agentRef) and survives an offline reconcile.
  • Auth errors are checked before the network/404 branch in both the retry loop and outer catch, and revocation requires generation === startGeneration, so a stale request from an old principal/team scope cannot revoke a newer one (reconcileChat.ts:169-172, :230). Confirmed send/rename revocation (403 Forbidden: host_access_revoked) → hideAgent fully purges latestChatSessions, active view, cache, composer, and deselects (useAgentChatController.ts:1886-1930). The old destructive delete-on-404 hazard is removed.

F1 — read-path revocation is downgraded to "uncertain," leaving chat titles visible in the sidebar. Severity: non-blocking (privacy coverage-gap). listSessions/loadSessionMessages throw generic ... failed (403): ... and do not project the host_access_revoked marker (rpcProxyClient.ts:1008-1012, :1050-1054), so isConfirmedHostAccessRevoked is false for every read-path denial. A catalog-refresh 403 — the primary way a revoked-but-idle host is discovered — therefore routes to hideHostAccess('uncertain'), which deliberately does not call hideAgent (useAgentChatController.ts:1909). SidebarNav renders latestChatSessions unfiltered (components/SidebarNav/index.tsx:212) and the latest-sessions effect deps exclude the block state, so the revoked agent's chat titles persist in the recent-chats preview until an unrelated reload changes the scope/agents key. Interaction is blocked and no transcript content leaks, but title metadata for an explicitly-denied host stays visible unless revocation also drops the agent from the catalog — a partial miss against "explicit Host access denial hides protected data." Fix this →

3. Scoped deletion & privacy / IPC trust boundary — ✅ sound, no escalation

  • ChatAuthorityScope = {environmentKey, userId, teamId}; tombstones key on the full identity and isDeletedInScope matches chatId + full scope, so team A's deletion cannot block team B creating the same chatId. Legacy deletedChatIds are honored only when teamId === null (chatStore.ts:355-370).
  • Tombstone + pendingChatCleanup persist inside serializeIndex before artifact removal and are retried on any chat:getIndex (chatStore.ts:1636-1660, ipc.ts:1677) — durable across restart.
  • No resurrection path. createChat throws for a tombstoned chat; saveMessages*/appendMessages/replaceMessages early-return on isDeletedInScope; renameChat/upsertServerSessions/index updates only mutate chats already present. The renderer merges server sessions against scoped tombstones end-to-end.
  • IPC boundary re-verifies everything. New channels call assertTrustedSender; sanitizeChatAuthorityScope/sanitizeChatDeleteFence validate shape and bounds; the renderer-supplied fence is rechecked against the main-derived authority (appService.ts:1017-1031) and store binding generation, then deleteChat re-asserts the store's own scope. A renderer can only delete within its current authenticated scope — no team/user/env escalation. preload.ts adds only three narrow invoke wrappers.

Non-blocking notes: (a) deletedChatTombstones is never evicted/compacted — index.json grows unbounded for heavy deleters; (b) the chat:delete IPC handler awaits but does not return the ChatDeleteResult typed in renderer.d.ts:702-706, so the renderer always sees undefined (harmless — durable retry covers pending cleanup — but the typed contract is unmet and untested); (c) write-path suppression uses the current-binding scope by default, so a late write for a team-A chat while bound to team B wouldn't be suppressed by the team-A tombstone (not exploitable given unique server-session IDs, but a residue gap for cross-team chatId reuse).

4. Test / CI correctness — ✅ proves outcomes, not implementation

  • The selective-watch proxy cuts only CommunicationChannel LIST/WATCH: the 503 gate matches GET + collection pathname (query stripped, so it catches both LIST and ?watch=true), everything else falls through upstream (hcc-watch-api-proxy.mjs:123-128, :274-281). The unit test independently proves channel LIST→503 while .../hosts and .../pods→200 and that blocked retries never reached upstream — real behavior, not a mock echo. A wrong CHANNEL_PATH would fail loudly (count>=1 required), not false-green.
  • The two-cycle assertion is corroborated by independent observables: .spec.template byte-equal to the pre-fault baseline, Ready pod UID unchanged, SQLite path env stable, and transcript access proven end-to-end by grepping for the actual marker content through rpc-proxy while the fault is verified still active. The one HCC-set status string is backed by these independent checks, so it isn't a false-green.
  • Desktop suites discriminate cases a naive echo could not (generic 403 → hold vs verified-host-403 → revoke; a 503 body mentioning "403" must not block; epoch-advance discards a stale pre-verification 403; late sends/task-results after deletion produce zero appends/notifications).
  • CI wiring is honest: the shell contract gate runs in ci-public.yml:531; the live cluster journey is reachable only via the opt-in make minikube-t2-hcc-stateless-cache-containment (Makefile:786-796), matching the PR's declared "not run in this PR" limits. Both new shell scripts use set -euo pipefail, trap cleanup EXIT, and fail-loud (no false-green path observed).

5. Security regressions / merge-with-dev — none found

  • The change is fail-closed in every direction: unverified Deployment → throw, missing → defer, unsynced → hold active replicas:1. Status writes never diverge from scale (replicas always derive from lifecycle.effective, which is active under suspensionBlocked).
  • No new renderer-trusted identity/scope; all destructive IPC re-derives authority in the main process. console.* → structured hccLogger conversions in statelessLifecycleExecutor.ts align with the repo logging standard. No secrets/tokens logged.

F2 — CodeQL unused bindingGeneration is a dropped host-binding fence. Severity: non-blocking (dead code / incomplete guard). useChatListController.ts:918 assigns const bindingGeneration = await chatStore.getBindingGeneration() but never reads it; the following stillOwned() guard compares only the renderer scope generation/string. getBindingGeneration is a real IPC returning a fence clearly meant for a before/after comparison, so this is an incomplete guard, not a stray local. Real impact is negligible (applyLocalTitleOnly does only a local title-cache write and is already scope-guarded), and it won't break the build (noUnusedLocals not set, no eslint config), but it should be either used in stillOwned() or removed to clear the CodeQL alert. Fix this →

Two further minor observations: F3 — the approval-decision notification path (useAppController.ts:774-795) calls decideApproval without the isHostAccessBlocked/isChatDeleted guard the conversation-open path has (not a data reveal — the RPC 403s — but an unnecessary action against a blocked host). F4 — hideHostAccessRef.current!(…) non-null assertions (useAgentChatController.ts:282,629,1205) would throw rather than degrade if an auth error were ever processed before the post-mount effect commits; safe in practice.


Bottom line

Merge-wise this PR does what it claims and does it safely: watch loss no longer destroys HCC state or the Desktop transcript/draft, confirmed deletion is durable and resurrection-proof, and the IPC boundary prevents cross-scope escalation. No blocking defects. The most valuable follow-up is F1 (read-path revocation → visible sidebar titles); the rest (unbounded tombstones, dropped ChatDeleteResult return, bindingGeneration, the two coverage gaps) are non-blocking cleanups suitable for PR B or a follow-up. Consistent with the PR's own scope note, this contains #396/#826 but does not fully close them.

@alfredolopez80

Copy link
Copy Markdown
Contributor Author

@claude please re-review the corrections at PR #849 head b70a66e5e096483aaf18566379d444b03f54a937.

CI/CodeQL is fully green again (41/41 checks). Your findings went through independent adversarial review. The surviving corrections are:

  1. Exact Host-wide 403 projection now covers listSessions, loadSessionMessages, and getTaskResult; generic 403s remain uncertain.
  2. Already-delivered approval notifications are filtered when their Host is blocked/revoked, including the unread count.
  3. chat:delete now returns the real ChatDeleteResult; the IPC contract test asserts cleanupPending.
  4. Local rename captures/rechecks live user identity and passes the binding generation through IPC to the main-process store fence.
  5. HCC cache-loss cancellation now bypasses only the stale cached-state precheck and invokes the fresh-read drain revert writer; a focused test covers cached active versus durable draining.

F4 was intentionally not changed: the adversarial reviews found no reachable pre-mount call and replacing the non-null callback with a no-op could lose a revocation. Unbounded tombstones remain intentionally deferred to durable server-side deletion state in PR B.

Please verify these corrections against your original findings and report any remaining blocking defects with file/line evidence and severity.

@claude

claude Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Claude finished @alfredolopez80's task in 2m 55s —— View job


Re-review of corrections @ b70a66e

  • Correction 1 — Host-wide 403 projection (listSessions, loadSessionMessages, getTaskResult)
  • Correction 2 — delivered approval notifications filtered when Host blocked/revoked (+ unread count)
  • Correction 3 — chat:delete returns real ChatDeleteResult; IPC test asserts cleanupPending
  • Correction 4 — local rename rechecks live identity + passes binding generation through IPC fence
  • Correction 5 — HCC cache-loss cancellation bypasses stale precheck, fresh-read drain revert
  • Cross-check F1/F2/F3/F4 and regressions

Verdict: all five corrections land correctly; my prior F1 and F2 are now resolved. No blocking defects. Details with file/line evidence below.


1 — Read-path 403 projection ✅ (resolves my prior F1)

exactHostAccessRevokedError gates strictly on status === 403 && errorCodeFromBody(body) === 'Forbidden: user cannot access this host' (rpcProxyClient.ts:459-463). It is now applied on the read paths: listSessions (:1026-1027), loadSessionMessages (:1070-1071), and getTaskResult/sendMessage via requestJsonProjected (:558, :543). Generic 403s fall through to the descriptive ... failed (403) error, so isConfirmedHostAccessRevoked stays false for them.

This closes the sidebar-title leak I flagged: a catalog-refresh 403 with the exact code now throws host_access_revoked, so useChatListController.ts:403 takes isConfirmedHostAccessRevoked(error) → onHostAccessRevoked(agentRef) (the hideAgent purge) rather than the uncertain hold. The scope fences (authorityScopeGeneration, hostAuthorityEpoch, currentAuthorityScope) still guard the branch, so a stale cross-scope 403 can't revoke a newer scope. Tests assert both the exact-projection and the generic-stays-uncertain cases (rpcProxyClient.test.ts:47-76). Correct and precise.

2 — Notification filtering ✅

notifValue now filters both the list and the unread count by !vm.isHostAccessBlocked(notification.agentName) (App.tsx:2216-2221), and vm.isHostAccessBlocked replaced vm.unreadNotificationCount in the memo deps (:2249) so the count recomputes when block state changes. agentName is the same agent-ref token routed to handleSelectChatAgent elsewhere in the file (App.tsx:586, :796), so the key matches isHostAccessBlocked(agentRef). Blocked/revoked hosts no longer surface delivered notifications or inflate the badge.

3 — chat:delete returns ChatDeleteResult ✅

Handler now return await store.deleteChat(...) (ipc.ts:1541), and the contract test asserts result equals { cleanupPending: false } (ipc.test.ts:204-207). The typed renderer contract is now met and covered.

4 — Rename identity/binding fence ✅ (resolves my prior F2 / CodeQL alert)

bindingGeneration from getBindingGeneration() is now consumed: passed to chatStore.renameChat(agentRef, chatId, newTitle, bindingGeneration) (useChatListController.ts:926), threaded through preload/renderer.d.ts/useChatStore, and enforced in the main process via requireChatStoreForBindingGeneration(bindingGeneration) with a Number.isSafeInteger shape check (ipc.ts:1476-1489). This clears the CodeQL "unused bindingGeneration" alert — the fence is now live, not dead code. Separately, handleRenameChatForAgent captures authUserKeyAtRequest and rechecks authUserKeyRef.current after each await (useChatListController.ts:1126, :1138, :1141), so a user/identity change mid-rename aborts before the local title write. Sound.

5 — HCC cache-loss drain cancellation ✅ (also closes my prior mid-drain coverage gap)

cancelDrainOnEvidence(hostRef, host, bypassCachedState=true) is invoked from the suspensionBlocked heartbeat branch (statelessLifecycleTracker.ts:443). bypassCachedState skips only the stale cached-state precheck (:629); the actual revert still runs through markHostActiveFromHeartbeatCore, which does a fresh GET and reverts only when the server-side lifecycle is draining, no-op'ing for active/suspended under a 409-safe precondition patch (statelessLifecycleExecutor.ts:1158-1190). So a cached active/durable draining host is correctly reverted, and the fresh read prevents resurrecting an actually-suspended CR. The new focused test exercises exactly the cached-active-vs-durable-draining case (statelessLifecycleTracker.test.ts:193-210), which also fills the "actively-draining host reverted mid-drain" gap I noted earlier.


Remaining non-blocking items (unchanged status — appropriate for PR B / follow-up)

  • F3 (useAppController.ts:774-795): decideApprovalNotificationTarget still calls decideApproval without an isHostAccessBlocked guard. Impact is reduced now that blocked-host notifications are filtered from the UI (Correction 2), but an already-delivered OS notification could still funnel a decision to a blocked host. No data reveal (RPC 403s server-side); non-blocking.
  • F4 (useAgentChatController.ts non-null hideHostAccessRef.current!): intentionally unchanged; the adversarial finding of no reachable pre-mount path, and the risk that a no-op fallback could drop a revocation, is a reasonable call.
  • Unbounded deletedChatTombstones: still deferred to durable server-side deletion state in PR B, consistent with the PR's scope note.

Regression / security check

No new regressions in the correction diff. All new IPC surface (bindingGeneration on rename) re-derives authority in the main process and rejects malformed/unsafe generations; no renderer-trusted identity was introduced. HCC change remains fail-closed (fresh-read gate cannot resurrect a suspended CR). Notification/read-path projections are strict-match, so they can't over-revoke on unrelated 403s.

Bottom line: the surviving corrections do what they claim and are backed by tests that discriminate the real outcome (exact vs generic 403, durable-draining vs cached-active). My two most material prior findings (F1 read-path title leak, F2 dropped binding fence / CodeQL) are resolved. Consistent with the PR's own scope, this contains #396/#826 without fully closing them.
· branch fix/stateless-incident-containment

…-containment

* origin/dev:
  fix(web-search): exclude fallback-only containers from prose
  fix(web-search): recover malformed comments and inert closes
  fix(web-search): preserve XHTML self-closing content
  fix(web-search): bound malformed HTML scanning
  fix(web-search): respect HTML context and bound CI image build
  fix(web-search): address title and argument review findings
  fix(web-search): address internal review regressions and bounded tooling
  test(web-search): allow runner group to read ephemeral TLS fixtures
  fix(web-search): cancel upstream work on HTTP disconnect
  fix(web-search): release fetch slots on HTTP protocol switches
  fix(web-search): validate fetch destinations and bound response resources
@alfredolopez80

alfredolopez80 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor Author

Local validation completed on profile clerum-fix-stateless-incident-containment-21489a9a at head 1e15b4c:

  • T0: PASS
  • T1: PASS — 263/263 tests
  • T2: PASS
  • NP-08 HCC authorization: PASS
  • CI at this head: 42/42 checks green, including CodeQL
  • Port-forwards / Health / Playwright: NOT_RUN in this local T2 execution

Evidence: .local-notes/infra/runs/20260925T155633Z-1e15b4cae555-cec1a6ead8d84c5ebb7286e855360256/evidence.json

alfredolopez80 and others added 12 commits September 30, 2026 11:57
…ecipe triggers

Closes R3-L11 (PR #849 review round 3). stripTrailingSentencePunctuation
stripped only `.!?`, while DIRECT_TRIGGER_HEAD captures the recipe token
up to the next whitespace, so `run daily-report, please` captured
`daily-report,` and was not recognised as a direct trigger. The strip set
is now `.!?,;:`; a comma followed by a non-filler tail is still rejected.

The adversarial linear-time bound is raised from 100 ms to 1000 ms: it
guards against super-linear backtracking, not latency, and shared CI
runners made the tighter bound flaky.

Fail-first: `recovers a direct named recipe request with a natural tail:
run daily-report, please` (and `run daily-report,`, `please run
daily-report, now`, `run daily-report;`, `run daily-report: now`) failed
at 65bc16c (5 failed | 36 passed). Paired control `run daily-report,
and delete it` stays false; disabling the tail check turns it red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes R3-L17 (PR #849 review round 3). The T1 reporter sanitizer built
the JSON-escaped spellings of the password with json.dumps and its
default ensure_ascii=True only, which writes non-ASCII characters as
\uXXXX. The Vitest JSON reporter writes with JSON.stringify, which keeps
them literal, so a password such as `Qzä"ss` survived as `Qzä\"ss` in
the report. The sanitizer now adds the once and
twice escaped spellings for ensure_ascii in (True, False).

Fail-first: new case 2a in scripts/tests/test-minikube-t1-reporter-report.sh
writes the fixture with Node's JSON.stringify (message and a quoted inner
JSON document) for two passwords; at 65bc16c it failed with
`FAIL: non-ASCII password Qzä"ss was not redacted in both spellings`.
Disabling password redaction entirely turns the suite red as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes R3-L18 (PR #849 review round 3). scripts/tests/test-autoresearch-gates.sh
exists and is executable but no workflow or Makefile target ran it, so
the autoresearch gate wrappers had no CI coverage. The shell-syntax job,
which already runs every sibling scripts/tests/test-*.sh contract, now
runs it too. The script needs only git and mktemp, sets its own fixture
git identity, and the job already checks out with fetch-depth: 0.

Evidence: before this change `grep -n test-autoresearch-gates
.github/workflows/*.yml` matched nothing (exit 1); the script passes
locally with 10 PASS lines, 0 FAIL, EXIT=0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes F5 (PR #849 review). t2_evidence_init opens every evidence file
with a `preflight RUNNING` phase. The standalone planner closes it, but
neither certification writer did: `make minikube-t2` (scripts/minikube/
t2.sh) and the T1 standalone lane (scripts/e2e/minikube-real-postgres.sh)
left `preflight RUNNING` as the phase's latest status, so a run with
attestationStatus PASS still reported an open preflight.

t2.sh now writes `preflight PASS` at the end of run_preflight_plan, after
local-preflight and the planner have passed. The T1 lane writes it once
the marker, process and control-postgres readiness checks pass, before
the suites start. For certification evidence a `preflight PASS` event
changes neither attestationStatus nor laneAttestationStatus, so the T2
contract and the attestation schema are unchanged.

Fail-first: two static assertions in scripts/tests/
test-minikube-t2-contract.sh failed at 65bc16c with `FAIL: T2
certification evidence never closes the preflight phase opened by
t2_evidence_init`. Removing either new write turns its assertion red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…handler

handleSessionsList decoded query.cursor with decodeSessionsCursor, which
returns null for a malformed cursor or one minted for another user/agent
scope, and then treated null exactly like "no cursor": the store was
queried from the start and page 1 came back as if it were the requested
page. The handler now throws `Invalid sessions cursor` before reading the
store when a cursor was supplied but does not decode for the scope.

The HTTP route (handleSessionsListRoute) already answers 400 `Invalid
sessions cursor` through parseSessionsCursorParam before calling the
handler, so its contract is unchanged; the guard covers every other
caller of the SessionsListHandler contract.

Fail-first: in sessionRouteHandlers.test.ts a malformed cursor and a
cursor from another agent scope resolved with page 1 at 65bc16c
(`promise resolved ... instead of rejecting`). They now reject and the
store spy records 0 reads; a valid-cursor control returns the next page
with exactly 1 store read. Disabling the guard turns both cases red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The renewal mock added for the fresh-bootstrap case built its token key
from a string concatenation, so its type widened to an index signature
and `tsc --noEmit` on the test project failed to assign it to
GfsHostToken. A literal key type keeps the same runtime value and makes
the mock assignable again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… intent

When the GFS renewal of a Ready held runtime failed, the catch reported a
controller error through enqueueControllerError, which also enqueues an
administrative `failed` outcome. The pass then ends with the runtime
deployed and Ready and reports `succeeded`, so one administrative intent
received both outcomes for the same generation and control-api stored
both. The catch now emits only the controller_error telemetry; the
pass's single outcome is the Ready runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
forwardCancelToHost replaced a missing userId with 'desktop-app', so
mcp-host would run its cancel ownership check against a stand-in
identity. The route always passes the verified token subject, which is
typed as a string; the forwarder now requires it and throws before any
upstream request when it is empty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t resolution

mutatingCallSignal only listened for the response 'close' event, but the
signal is built after the host resolution and wake-hold work. A client
that disconnected during that window had already fired 'close', so the
listener never ran and the approve, deny, set-model or cancel POST was
still sent upstream with nobody waiting for it. The signal now starts
aborted when the response is already closed without having finished.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne rejects

A session catalog request issued before invalidateSessionCatalog() that
rejects afterwards used to delete its cache key unconditionally, evicting
the newer request cached under the same key and forcing a redundant
upstream call. The rejection handler now evicts only its own entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent leaves

The session rename PATCH used a plain upstream timeout, so a client that
disconnected (before or after host resolution) left the upstream call
running until the timeout. It now uses mutatingCallSignal with that same
timeout. The R3-L6 disconnect table covers the route, and the test token
carries host:session:write so the request reaches the upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the composition property requested in review: merging cursor page A
and then page B equals one merge of both pages when they report disjoint
chats. A witness checks both pages reached the result, so a merge that
ignores its sessions fails. Overlapping pages do not compose by design (a
later report updates the title, updatedAt stays from the first report);
fast-check finds that counterexample when page B is allowed to overlap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alfredolopez80

Copy link
Copy Markdown
Contributor Author

@claude please run a full re-audit and a closing audit of this PR at head 051cefaa84a18d9dbb6e9573a229c39fb761ea28 (30 commits on top of 65bc16c26), and state whether it can be merged.

Scope, in this order:

  1. Every finding raised in any comment or review on this PR, not only the latest round. That includes:

    Give each finding exactly one verdict with file:line or commit evidence at this head:

    • FIXED: counts only if the code path is correct and a test fails when the fix is reverted.
    • ROUTED: name the issue.
    • REFUTED: give the evidence.
    • STILL-OPEN or REGRESSED.

    Any finding without a verdict is itself an open item.

  2. The follow-up issues opened for the deferred items. Check that each one exists, describes the residue accurately, and that nothing deferred in the PR body lacks an issue:

  3. Regressions and new bugs introduced by git log 65bc16c26..051cefaa8. Look for:

    • races, fail-open paths and swallowed errors;
    • contract drift between control-api, rpc-proxy, HCC and Desktop;
    • negative assertions without a liveness witness.

    Pay particular attention to the client-disconnect abort in mutatingCallSignal (a client that left during host resolution, the session rename), the tombstone sidecar write order, and the catalog request invalidation.

  4. The PR description against the code. Every sentence that describes behavior must match the code at this head. Report any claim that the code does not implement.

  5. Decisions already taken; please do not re-litigate them. They are documented in the description:

Evidence at this head:

  • GitHub checks: 42 of 42 success.

  • Canonical make minikube-t2 on the branch-owned profile: MINIKUBE_T2_PASS, T0=PASS T1=PASS T2=PASS NP08_HCC_AUTHORIZATION=PASS PortForwards=PASS, T2_EXIT=0. Health and Playwright are NOT_RUN (separate lanes).

  • T0 suite counts:

    Package Tests passed
    rpc-proxy 789
    Desktop 3,102
    host-context-controller 2,603
    mcp-host 5,284 (+1 todo)
    control-api 6,611
    control-ui 2,358
    external-rest-api 164
  • T1 Real PostgreSQL: 373/373 tests in 65/65 files.

  • The Desktop UI project still reports only the three pre-existing tsc errors in ui/src/lib/imageHeaderDimensions.ts.

Please report a table (source comment, id, verdict, evidence), then a list of anything still open with severity (Blocker / High / Medium / Low), then a final line: MERGE-READY or NOT MERGE-READY with the blocking items.

@claude

claude Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Claude finished @alfredolopez80's task in 8m 13s —— View job


Closing re-audit @ 051cefaa (30 commits on 65bc16c26)

I audited the full 65bc16c26..051cefaa8 delta by area (HCC, rpc-proxy, Desktop, mcp-host + scripts), reading source and the paired tests, and checking that each fix is revert-sensitive (a test fails when the fix is reverted). CI and CodeQL are both success at this head (verified via the CI API).

Headline: every finding raised across all rounds is FIXED (revert-sensitive), REFUTED, or a documented DEFERRED/DECISION routed to a follow-up. No new code regression in the delta. The only defects are two documentation inaccuracies in the PR body — the code is correct and fail-safe in both cases — so they don't block merge.


1. Verdict table

Source Id Verdict Evidence @ 051cefaa
R3 R3-H1 McpServer relist → token downgrade+rollout FIXED frontsOAuthServer collapse removed; normal path DEFERS (hostReconciler.ts:2169, status :5272); test lifecycle.test.ts:1931 asserts issue not called + zero deployment writes; 2327789e
R3 R3-H2 refreshGfsOnly rolls w/o mint FIXED consumes bootstrap when Ready+same revision (hostReconciler.ts:1934-1940); test asserts bootstrap-state='consumed', rollout-required≠'true'; 7eed4e73
R3 R3-M1 idempotent GETs lost post-wake retry FIXED 7 GETs+/messages pass retryUntilDeadline:true, mutating routes maxAttempts=1 (wakeAndHold.ts:709); real-http test 3 reads vs 2; a9a11cd0
R3 R3-M2 revocation arrives as "uncertain" DEFERRED-OK option B (confirmed only while a cached token <5 min exists) documented in body; option A → #941
R3 R3-M3 hand-built 403 fixtures (no witness) FIXED IPC-shaped ipcHostAccessRevoked/ipcGenericForbidden + same-scope controls that fire the classifier; 777278c0
R3 R3-L1 sidecar failure rejects committed index FIXED sidecar written before index, each atomic; test witnesses sidecar write + unchanged catalog; 0432fdbc
R3 R3-L2 crash window drops tombstone FIXED sidecar is a superset a valid index ignores, realigned next save; 0432fdbc
R3 R3-L3 sidecar claims untested FIXED df328604 covers EACCES/EIO rethrow, wrong-shape→empty+warn, precedence
R3 R3-L4 cancel/rename 8s first attempt cancel FIXED / rename DEFERRED-OK cancel first attempt no client timeout (rpc.ts:1402-1407); rename keeps 8s by design (idempotent, Desktop retries)
R3 R3-L5 download header wait ⊇ redaction REFUTED headers use upstreamTimeoutMs, body re-armed with artifactDownloadTimeoutMs (rpc.ts:1560-1585); docx/pdf skip redaction, text bounded 50 MB
R3 R3-L6 mutating: no deadline/abort on disconnect FIXED mutatingCallSignal aborts on res.closed+once('close') incl. during host resolution & rename; ab924072/2279431a/70b7ef4e
R3 R3-L7 only /messages pins deadline pre-resolve FIXED wakeDeadlineMs computed before resolveHostConnectionForUser in all 12 held routes; describe.each test; 51ea9a15
R3 R3-L8 subject_mismatch not a revocation FIXED (code) ⚠️ body drift hostAccessDenial.ts:25-28 → host_access_denied; test host-access-denial-code.route.test.ts:226; cd8e1159. PR body summary still lists it under host_access_revoked
R3 R3-L9 denial reasons hand-copied FIXED cross-service guard test reads all 3 sources & asserts agreement; ba864331
R3 R3-L10 invalid cursor w/o status never dropped FIXED over-long cursor dropped, 408 non-terminal, 409 terminal; a8e36982 + mcp-host 7e56e5fc
R3 R3-L11 run daily-report, please no trigger FIXED strips .!?,;:, filler now/again/please/immediately; revert-sensitive test; 07c16b0f
R3 R3-L12 cache-loss ready:false for new hosts DEFERRED-OK fail-closed choice documented in Known deferred boundaries; → #942
R3 R3-L13 failed held GFS mint → deployed:false FIXED keeps deployed:true, telemetry not failed-intent; test asserts single succeeded+<1000ms; 98685c9d+6ec74d72
R3 R3-L14 hand-written credential fixture FIXED mintedRuntimeCredentialRecord from a real mint pass + assertProducerRuntimeAnnotations; aafb5022/9f4d347c
R3 R3-L15 ipcHostWaking accepts any channel FIXED 777278c0
R3 R3-L16 dead rename revoked-matcher alt FIXED 62f6968f
R3 R3-L17 T1 reporter leaks non-ASCII password FIXED both ensure_ascii spellings redacted (minikube-real-postgres.sh:284-293); Node-built fixtures assert; 15dd1ba6
R3 R3-L18 autoresearch gates not in CI FIXED ci-public.yml:538 runs the self-test; c6690eac
Claude r1 F1 read-path revocation title leak FIXED (prior) resolved in earlier rounds
Claude r1 F2 dropped bindingGeneration fence FIXED (prior) consumed in rename fence
Claude r1 F3 approval-notif lacks block guard DEFERRED-OK mitigated by UI notif filter + server 403
Claude r1 F4 non-null hideHostAccessRef.current! DEFERRED-OK intentional; no reachable pre-mount path
Trigger F5 not located no finding labelled F5 appears in the PR thread text available to me — please point me at it if it exists (flagging per "no verdict = open")
Trigger F6 T0 attestation lost when T1 fails ROUTED → #940 listed in Follow-ups
R1/R2 R2-B1, R2-H1, R1-H1..H5 FIXED (prior + hardened here) jozer approved at 65bc16c26; HCC/rpc hardening this round
R1 R1-M1 / O1 hold fail-open on admin revocation DEFERRED-OK OAuth re-grant now fail-closed; channel-ingress residue → #942
R1 R1-M4 tombstone scoped to session team FIXED deletion identity = env+user (chatStore.ts:108-110), survives team hop
R1 R1-M6 exact 403 body not unique FIXED header contract x-host-access-denial-reason, body pinned {"error":"Forbidden"}
R1 R1-M7 getIndex O(N²) cleanup FIXED per-agent on read + full sweep once per bind; 48e3b7fb
R1 R1-M11 fake upsertMessages diverges FIXED delegates to a real ChatStore; f48785b2
R1 R1-M8 / D3 uncertain acts like revoked DECISION documented (hidden+deselected until re-verified); not re-litigated
R1 R1-M14 HostWizard stateless creation DECISION kept in PR by design; documented
jozer #3 team-hop UI race DEFERRED-OK → #945 durable main-process suppression holds
GHAS 1552/1556/1558/1559/1599/1600/1601/1602/1605 FIXED CodeQL run success at this head; intent-recovery keeps \S{1,128}, disjoint classes, no nested quantifiers

2. Follow-up issues

The PR body routes each deferred item to a specific issue: #940 (attestation/F6), #941 (mint-403 revocation code / R3-M2 option A), #942 (channel-ingress scope + wake template / R1-M1 residue + R3-L12), #943 (turn duplicated on reconcile), #944 (verifyRpcToken empty sub), #945 (team-hop race / jozer #3), #825 (durable deletion), #930 (flaky grok leak assertion). Every deferred boundary in the body has a matching number, and no deferred item is left un-routed.

⚠️ Verification gap: gh is not permitted in this sandbox (requires approval on a private repo), so I could not independently confirm that #940–#945/#825/#930 actually exist and that each issue body describes the residue accurately. The routing is present and internally consistent in the PR description; please confirm the issues themselves out-of-band.

3. Regression sweep (65bc16c26..051cefaa8)

No new code regression found. The delta is hardening; every negative assertion I checked carries a liveness witness. Specifically on the areas you flagged:

  • mutatingCallSignal (rpc.ts:200-213): aborts on the already-fired close during host resolution (res.closed && !res.writableFinished) and on later disconnect (res.once('close')), guarded by writableFinished so a normal completion never aborts — correct.
  • Tombstone sidecar order (0432fdbc): sidecar-before-index, atomic, a sidecar failure rejects before any catalog change — correct.
  • Catalog request invalidation (b29077da): the late rejection evicts only its own entry via identity check (get(key) === entry), so an invalidated request can't clobber the newer one — correct, revert-sensitive test with a witness.
  • Task cancel (f0dab25a): now rejects an empty caller subject before any request (was a 'desktop-app' stand-in identity) — a genuine hardening.

4. PR-description accuracy — 2 stale statements (code correct, doc wrong)

  1. subject_mismatch mapping (R3-L8). The summary's rpc-proxy bullet lists host_access_revoked for "team_membership_missing, directory_grant_missing and subject_mismatch". The shipped code (hostAccessDenial.ts:25-28, commit cd8e1159) maps only the first two to revoked; subject_mismatch → host_access_denied. The code is the safer choice (does not hide an agent the user still owns). Fix the body sentence to drop subject_mismatch from the revoked list. Fix this →
  2. Intent-recovery timing claim. The body says the 200k-char test "asserts it finishes under 100 ms"; the assertion in toolUseLoopIntentRecovery.test.ts was relaxed to <1000 ms (commit 07c16b0f) to avoid CI flake. The super-linear-backtracking guard still holds; only the stated bound is stale.

5. Decisions — not re-litigated

Control-UI stateless creation in-PR, uncertain-hosts-hidden (option B), never-pruned tombstones (until #825), first-mutating-attempt-no-timeout / rename keeps 8 s, and the T2 attestation change deferred to #940 — all confirmed present and consistent in the code and body; left as your decisions.


Still open (by severity)

MERGE-READY

Every blocking finding from every round is fixed with a revert-sensitive test, the delta introduces no regression, CI/CodeQL are green, and the residuals are documented-and-routed decisions. The only cleanups are the two non-blocking PR-body wording corrections in §4; I'd fix those before merge but they don't gate it. Please independently confirm the follow-up issues exist (I couldn't from here). Consistent with the PR's own scope, this contains #396/#826 without fully closing them.

This is my read from reviewing the diff, not a formal GitHub approval.

@alfredolopez80
alfredolopez80 merged commit f6cb749 into dev Sep 30, 2026
42 checks passed
@alfredolopez80
alfredolopez80 deleted the fix/stateless-incident-containment branch September 30, 2026 12:25
@alfredolopez80

Copy link
Copy Markdown
Contributor Author

Thanks. Follow-up on the closing audit at 051cefaa8:

§4 PR description, both Low items fixed in the body:

  • subject_mismatch now appears under host_access_denied, matching REVOKED_CONTROL_API_REASONS in rpc-proxy/src/services/hostAccessDenial.ts (only team_membership_missing and directory_grant_missing map to host_access_revoked). The bare "mismatch" is now spelled out as the userId / hostRef mismatch in the lookup response (controlApiRestService.ts).
  • The intent-recovery bound now reads "under 1000 ms", matching expect(elapsedMs).toBeLessThan(1000) in mcp-host/src/core/orchestration/toolUseLoopIntentRecovery.test.ts.

Verification gaps you reported:

No code changed; the head stays at 051cefaa8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants