Skip to content

refactor(transport): share outgoing transfer phase ownership - #4551

Merged
thymikee merged 14 commits into
t3/audit4441/remote-budgetsfrom
t3/audit4441/phase-owner
Nov 15, 2026
Merged

thymikee merged 14 commits into
t3/audit4441/remote-budgetsfrom
t3/audit4441/phase-owner

Conversation

@thymikee

@thymikee thymikee commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Summary

Transfer phase timing has one owner in host-kit. Remote uploads compose its connection/finish/response observer with #4552's completed-write observer. Completed hops retire both observers before any next hop.

This changes 8 files against #4552. The attempt now has one abort-reason mapping and no duplicate success disposal. Its sole response-path join is inlined; failure paths retain the outer final join. Existing watchdog tests move with the owner. The unused suspend API remains deferred. Refs #4441. Proxy, daemon and download deadlines remain separate.

Validation

Tested 024cb04a4d38de644148233ae79108191a28507d, parent 13a4c1d9, merge-base d78be6c8.

Pinned pnpm 11.17.0 check:affected --run --head <sha> --json passed once: exactCommit=true, checkoutChanged=false, failOpen=false. Nine checks passed, including wire compatibility, 376 related tests and 19,847 unit tests (10 skipped). Seven CI-owned/manual checks were skipped; #4050 full-unit fallback remains. Focused baseline passed 127 tests.

Cold review found no findings. Removing the central abort-reason preference fails three typed-reason assertions; restored controls pass. The phase-observer disposal call site retains source/listener-unit evidence rather than a runtime-discrimination claim.

Gross sizes: lower 2404, this layer 699, combined 2497. Current-head Coverage, Integration, Apple and other required checks are green. Sentinel approves with no open threads; two below-minor items are hidden. Linux Coverage passed 20,196 tests (70 skipped), plus 11 fuzz tests. Earlier failed/cancelled runs stay recorded. Node-floor, Bun and remaining #4441 runtime obligations are unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.28 MB 6.28 MB +168 B
Package (unpacked) 6.28 MB 6.28 MB +168 B
Package (download) 1.90 MB 1.91 MB +59 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.2 ms 24.4 ms -2.8 ms
CLI --help 48.0 ms 48.0 ms -0.1 ms

@thymikee
thymikee added this pull request to stack #4553 November 14, 2026 15:27

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files

You’re at about 91% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@thymikee thymikee changed the title t3/audit4441/phase owner refactor(transport): share outgoing transfer phase ownership Nov 14, 2026
@apex-sentinel-staging

apex-sentinel-staging Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Apex Sentinel · ✅ Approve

Since the last review: 0 new · 0 still open · 0 no longer raised

No open findings.

2 findings below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

💬 Talking to Sentinel

In a finding's thread, reply with:

Command What it does
@apex-sentinel-staging note <context> Adds context; Sentinel re-checks the cited code
@apex-sentinel-staging pivot <why> Says the finding is wrong; Sentinel re-checks the cited code
@apex-sentinel-staging like Marks the finding as useful
@apex-sentinel-staging dislike <why> Marks it as not useful, so it is not raised again

On this conversation, comment @apex-sentinel-staging review to run a full review once again.

Earlier reviews

Review of af5c6344 · 14.11.2026 20:49 UTC

Apex Sentinel · ✅ Approve

Since the last review: 0 new · 0 still open · 1 no longer raised (1 marked fixed)

No open findings.

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

Review of 10ea602e · 14.11.2026 18:53 UTC

Apex Sentinel · ✅ Approve

Since the last review: 1 new · 0 still open · 1 no longer raised (1 marked fixed)

1 finding(s): 1 minor.

🟡 Minor (1)

Where Finding
packages/host-kit/src/internal/transport.ts:257 🆕 Phase sequencing moved out of the digested wire declaration, silently narrowing the gate

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

Review of 217528ae · 14.11.2026 15:34 UTC

Apex Sentinel · ✅ Approve

1 finding(s): 1 minor.

🟡 Minor (1)

Where Finding
packages/host-kit/src/internal/transport.ts:175 New watchdog.suspend() has no production caller

1 finding below minor hidden (user_settings.min_severity).

Reviewed with this repository's own Sentinel configuration (d78be6c88715).

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 new finding on 217528a. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

I found no code defects at 217528a that stop this PR. It is not ready to merge yet, because the Coverage check failed and its log is cut off before any failure line. The changed host-kit and remote upload tests run in that job, so I cannot say whether this diff caused the failure. Please rerun the job or share the full log, and confirm the failure is not in the moved transport tests.

Not blocking, and you can take or leave these: suspend() and its suspended state at https://github.com/callstack/agent-device/blob/217528a/packages/host-kit/src/internal/transport.ts#L175 have no production caller, only the tests in transport-phases.test.ts. Each member of the shared watchdog should have a production caller in the PR that adds it. The doc comment on TRANSFER_CONNECT_TIMEOUT_MS at transport.ts#L91 says the same budget covers a standalone artifact download, but remote/artifact-download.ts uses its own whole-request timeout. Please remove that clause until the download adopts the watchdog. The remote fixture at https://github.com/callstack/agent-device/blob/217528a/packages/remote/src/__tests__/support/upload-transfer-fixtures.ts#L24 spells the header and body budgets as literals but imports the connect budget from host-kit. Please pick one convention, and keep the literal pin in transport-phases.test.ts.

Could the PR move only what has a production consumer, which is the watchdog without suspend() plus observeClientRequestPhases? That drops about 15 production lines and 5 tests, and suspend() can land with the proxy caller that defines when nothing is owed. Beyond that, I found no smaller shape. The observer already replaces about 45 inline lines in upload-stream.

Of the open inline threads, #4551 (comment) (guard test passes anyway) and #4551 (comment) (comment says minutes) still apply, as does #4551 (comment) (no suspend() caller). Two do not apply and can be resolved. The connect budget cannot drift, because transport-phases.test.ts:28 pins it as a literal: #4551 (comment). The drain rule is unchanged from the base upload-stream, and the shipped upload route already hits backpressure with 64 KiB chunks: #4551 (comment).

I did not re-run your mutation controls or the closure, import-budget and layering gates, and I did not reproduce the Coverage run locally. The lazy-loading claim rests on reading the type-only imports.

@thymikee

Copy link
Copy Markdown
Member Author

I accepted the smaller surface: suspend() and its five exclusive tests are removed locally together; the original watchdog and observer coverage remains. The stale download wording, independent fixture pins, already-aborted zero-timer assertion and timing comment are also corrected. Fresh exact receipts are still owed, so nothing is pushed yet.

The drain thread has new runtime evidence beyond the default CLI path: a real HTTP loopback at 217528 completed four 1 KiB writes and received 4096 peer bytes with zero drains, then incorrectly expired at request-body/120000. The 64 KiB file path stayed alive. I am correcting the shared write-completion boundary while preserving backpressure, callback semantics and phase guards; this does not claim the current default file path reproduced it.

I reran the failed Coverage job once per your request. The original log contains no failure summary and remains failed with cause unproven; I will capture the rerun outcome and moved-test collection, then post final heads and validation after the review-round push. No merge.

@thymikee

Copy link
Copy Markdown
Member Author

The single requested Coverage rerun also failed at the unchanged published head. Its text log is still truncated, but the check-run annotations identify 6 failures in this PR's remote loopback tests: progress/stall phase cases and resume, redirect, connected-stall and completion lifecycle cases. The stalled-body case hits the existing 5000 ms test limit; the other failures are assertions. These are captured candidate-test failures, not a passing rerun or an infrastructure diagnosis.

The existing source owner now has the exact annotations and is reconciling real I/O with the controlled clock while making the already-assigned write-progress and fixture corrections. No additional worker, timeout increase, test deletion or CI retry is being started. No moved host-kit test is named in these annotations, but that alone does not establish its successful execution; final-head Coverage remains required. Original and rerun failures retain their heads and logs.

@thymikee
thymikee force-pushed the t3/audit4441/phase-owner branch from 217528a to 10ea602 Compare November 14, 2026 18:45
@thymikee

Copy link
Copy Markdown
Member Author

Addressed the review at 10ea602e701059acfb88140468e95b1cefd5d3c7, stacked on lower 22cf1d49 in #4552.

Removed suspend() and its exclusively obsolete tests until a proxy consumer defines the pause contract. The eight original watchdog tests and six phase-observer tests remain. Request-write progress now has one tested owner in the first layer; the upper observer handles connection, flush and response boundaries, composed with that owner. Both are disposed at the completed-hop boundary. The previously generic drain-only defect is covered by a real 1 KiB/no-drain loopback regression; the shipped 64 KiB path did not reproduce it on the measured runtime. The remote fixture pins all three phase budgets as literals, and the already-aborted test now asserts no armed timer before time advances. The comment states 30–120 s. All five existing inline threads have replies and dispositions.

One pinned pnpm 11.17.0 check:affected --run --head 10ea602e701059acfb88140468e95b1cefd5d3c7 --json passed: exactCommit=true, checkoutChanged=false, failOpen=false. Nine checks passed, including 376 related tests, 19,838 unit tests (10 skipped), and wire compatibility. Seven GitHub/manual lanes were skipped; the full-unit fallback remains. The moved transport tests passed in local focused validation and the full local unit gate. Fresh independent review found no actionable findings. Removing only the phase-observer dispose call still survives the runtime mutant; its call site is source-reviewed and listener removal is unit-tested, rather than claimed runtime-detected.

The old Coverage failure and its authorized rerun remain failed. The captured rerun annotations identify six remote loopback failures, with none naming host-kit; that absence does not prove the moved tests passed in the failed job. Revised fixtures wait for actual I/O and use structurally unfinished sources. Corrected-head Linux CI must establish the moved-test and Coverage result; macOS passes and delayed-I/O modeling do not establish the old Linux root cause. No additional blind retry was made. The unused standalone-download clause remains an optional comment cleanup; no download adoption is claimed. Node-floor, Bun, proxy and daemon work remain outside this layer.

@thymikee

Copy link
Copy Markdown
Member Author

I found no code problems at 10ea602, but the PR is not ready to merge yet because the CI run that proves the fix has not finished. The earlier loopback-test failures are addressed. The six loopback fixes live in #4552, not in this PR. The old drain listener and suspend() are gone. The fixture timeout, the timer-count assertion and the test header are also fixed. Before merge, I need a green Linux Coverage run (and Integration Tests) on 10ea602 that shows host-kit transport-phases.test.ts and the remote upload-stream-phases and upload-stream-lifecycle loopback suites passing, in the same lane where those six tests failed before. Those jobs were still running or cancelled by a newer run when I checked. One Smoke Tests job failed. It exercises device CLI routes that do not reach the upload or transfer-phase code, so I see no overlap, but I did not capture its log. I did not review the 733-line fixture rewrite in #4552, prove the Linux root cause, or re-run the local gates or your mutation controls. No device run is needed, because the change does not touch dispatch, capture, interaction, app lifecycle or native execution. There are no conflicts.

Not blocking, and fine to take or leave: upload-stream.ts:159 composes two host-kit observers that are always installed together, and three exits repeat the same three dispose calls. If observeClientRequestPhases installed the write observer itself and stopped it in ClientRequestPhases.dispose, each exit would need one dispose.

The five review threads, #4551 (comment), #4551 (comment), #4551 (comment), #4551 (comment) and #4551 (comment), are fixed at this head and can be resolved.

@apex-sentinel-staging apex-sentinel-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex Sentinel: 1 new finding on 10ea602. The verdict and the full list are in the summary comment.

@thymikee

Copy link
Copy Markdown
Member Author

The CI run I was waiting for has finished at 10ea602: Coverage and Integration Tests pass, so the host-kit transfer-phase and remote upload-stream suites now run green in the Linux lane, and the smoke jobs pass too. With no code problems found in the earlier review, this is ready for human review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 14, 2026
@thymikee

Copy link
Copy Markdown
Member Author

The completed CI run for the reviewed head 10ea602e701059acfb88140468e95b1cefd5d3c7 is now green: Coverage and Integration Tests, in successful run 43628395451. No rerun was requested by this sweep.

I captured the full Coverage log. Its instrumented command was vitest run --coverage --allowOnly=false --project=!fuzz-worker; it reports 2,194 files passed / 4 skipped and 20,187 tests passed / 70 skipped. The dot reporter has no individual-file pass lines. At this exact head, the unit-core include pattern covers transport-phases.test.ts, upload-stream-phases.test.ts and upload-stream-lifecycle.test.ts; none has a skip/conditional marker, and the exclusions are the separate fuzz-worker and apple-runner projects. Thus the successful all-files instrumented run includes the three requested suites; this conclusion uses the recorded command, configuration and source, rather than absence of failure annotations.

The newer Apple run also passed. The failed aggregator in the rollup belongs to the superseded cancelled run 43628394595; its log says RESULT: cancelled. The five old threads are already resolved.

I am leaving the optional observer composition unchanged: the accepted ownership split keeps the first-layer write observer consumed independently and the upper phase observer separate. The current caller installs one of each and retires both. This avoids a new composition contract in this correction round.

This evidence belongs to published 10ea602e, not the later local review corrections. The lower PR still has its own failed Coverage/iOS jobs and a cause-unproven diagnosis. New local heads require their own checks and review before the next push; nothing is merged.

@thymikee
thymikee force-pushed the t3/audit4441/phase-owner branch from 10ea602 to af5c634 Compare November 14, 2026 20:33
@thymikee

Copy link
Copy Markdown
Member Author

Restacked and pushed af5c634 over #4552 at 77a5e16. The phase owner still composes with the lower completed-write owner; no unused suspend API is retained. The lower layer now treats typed control-request expiry as terminal and preserves caller cancellation, with corresponding wire-manifest coverage.

Independent source review and the final test-only recheck found no findings. Focused tests passed 127/127; the exact affected gate passed once (19,847 unit tests, 10 skipped; 376 related tests; nine checks passed, seven CI-owned/manual checks skipped; full-unit fallback retained). Counterfactuals and historical failures keep their exact-head attribution.

Coverage, Integration and smokes passed at the predecessor 10ea602, as previously captured. New-head Linux CI and reviews are still required; no old result is transferred to af5c634 and no failed job was blindly retried. The optional observer-composition/producer-loop reshaping remains deferred; phase-observer disposal has source/listener-unit evidence rather than a runtime-detected call-site claim. No merge.

@thymikee

Copy link
Copy Markdown
Member Author

The PR is ready at af5c634. The earlier review of 10ea602 (#4551 (comment)) found no problems, and the changes since then are comment edits, a type-union reformat and pinned test literals, with no runtime change.

Not blocking, take or leave: the ordering comment "Move the phase before the request's response callback runs." now sits on the prependListener('response', onResponse) in observeCompletedRequestWrites (https://github.com/callstack/agent-device/blob/af5c634/packages/host-kit/src/internal/transport.ts#L129), which only stops counting writes. It belongs on the same call in observeClientRequestPhases (around line 265), or can be reworded here to "Stop counting writes before the request's response callback runs."

The wire-surface thread on observeClientRequestPhases (#4551 (comment)) does not apply. That function only sets local deadlines and sends no bytes on the wire, and the ledger rationale for streamFileToHttpRequestAttempt records the delegation. Please resolve it.

Some checks did not pass on this head. The Smoke Tests run has one failure plus cancelled sibling jobs, and the lint, coverage and integration run was cancelled. The smoke jobs drive local device flows and do not reach remote upload transfer, so these look like cancellation fallout rather than this change, but I have no failure log to confirm that. I did not rerun the focused tests, so the author-reported results are unverified here. Before merge, please rerun the cancelled and failed checks on af5c634 until they are green.

@thymikee

thymikee commented Nov 14, 2026 •

Copy link
Copy Markdown
Member Author

The completed replacement runs at af5c634 are green:

The older aggregate132579429731 actually logs RESULT: cancelled; its parent43638146720 is cancelled. Those superseded entries remain visible in the rollup, but the replacement runs above already supply current-head passes. No rerun was needed or requested.

The wire-surface thread4575183705 is already replied to and resolved with the local-policy versus peer-payload rationale; Sentinel now reports no open findings. I accept the optional wording correction: the completed-write observer stops counting writes rather than moving a phase. It is recorded for the existing correction round after the frozen source review finishes; no source was changed by this backstop.

These CI results belong to published af5c634 only. The pending #4552 request-finish fix requires restacking and fresh exact-head validation before publication, followed by CI on the pushed heads; current passes do not transfer to those local pins. No merge.

thymikee and others added 7 commits November 14, 2026 22:25
…going-request phase observer

Move the transfer phase watchdog into host-kit/transport, add suspend() for
demand-aware idle, and extract the node:http outgoing-phase observation that
remote upload adopts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ort internals

internal/transfer-phase.ts was a new module that every client, daemon, proxy
and remote entry evaluated eagerly through the transport barrel. The phase
watchdog, its typed expiry classifier and the outgoing-request phase observer
now live in internal/transport.ts, which those closures already evaluate.

The barrel exports only what production consumers import:
createTransferPhaseWatchdog, isTransferPhaseTimeoutError,
observeClientRequestPhases and TRANSFER_CONNECT_TIMEOUT_MS. The remote
loopback tests pin the response-header and body-inactivity policy values as
literals in their shared fixtures; the host-kit tests import the owner module
directly and are renamed for it (transport-phases.test.ts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ttempt wire digest

Moving the outgoing-request phase observation into host-kit/transport and
retiring it per attempt changed streamFileToHttpRequestAttempt's digest
without a ledger update. The digest is the one wire-compat.test.ts prints for
the current source; requests on the wire are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he agent-docs budget

docs/agents was 40081 bytes against the 40000-byte guidance budget. The one
module that owns transfer phase budgets and its host-kit/transport export
already state that ownership.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The outgoing-request phase observer stops treating drain as body
progress and owns only the connection, flush and response phases. The
upload attempt composes it with host-kit's observeCompletedRequestWrites
once per request and disposes both at every settlement. Generic write
cases live with the write owner; the phase suite keeps the composition
cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…upload attempt digest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee
thymikee force-pushed the t3/audit4441/phase-owner branch from af5c634 to 024cb04 Compare November 14, 2026 21:53
@thymikee

thymikee commented Nov 14, 2026 •

Copy link
Copy Markdown
Member Author

Pushed and restacked 024cb04a4d38de644148233ae79108191a28507d over #4552 at 13a4c1d9576777989ab3849cda9fb78661a33988. The optional comment now correctly says the write observer stops counting writes before the response callback. The phase observer remains the sole phase owner.

The smaller attempt removes duplicate success disposal and maps an aborted signal's reason once in fail. Its single response-path pipeline join is inlined with the same outcome handling; every failure still joins in the outer finally. Cold independent review found no findings. Removing that abort-reason preference fails three typed-phase assertions; restored controls pass 5/5.

The focused suite passes 127 tests. The exact affected gate passed once with exactCommit=true, checkoutChanged=false and failOpen=false: nine checks passed, seven CI-owned/manual checks skipped, 376 related tests, 19,847 unit tests passed and 10 skipped; full-unit fallback retained. Actual gross sizes are lower 2404, this layer 699 and combined 2497.

Current-head 024cb04a CI is green: Linux Coverage passed 20,196 tests (70 skipped), plus 11 fuzz tests. Integration and the replacement Apple smoke run passed. Sentinel approves with zero open findings or review threads; two below-minor items are hidden, with no actionable details supplied. The old cancelled jobs and failed aggregate remain historical. The dedicated watch is removed for human review; #4552 is also green and handed back. Predecessor af5c evidence retains its SHA attribution. The wire thread remains resolved; no new manifest waiver, protocol bump, timeout widening or CI retry. No merge.

@thymikee

Copy link
Copy Markdown
Member Author

I reviewed the changes since af5c634 at 024cb04 and found no problems. The new source-close assertions look like valid regressions, as long as the spy's 20s release timer runs on the installed transfer clock under fake timers. I did not run the tests locally, and I did not verify the claim that dropping the abort-reason preference fails three assertions. CI is green at this head: an earlier run was superseded and cancelled, and a rerun passes every required job. No further changes are needed from this review.

@thymikee
thymikee merged commit d64ac72 into main Nov 15, 2026
26 of 37 checks passed
@thymikee
thymikee deleted the t3/audit4441/phase-owner branch November 15, 2026 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant