Skip to content

Prepare stream buffers before capture - #291

Merged
nclack merged 2 commits into
mainfrom
prepare-stream-startup
Sep 23, 2026
Merged

nclack merged 2 commits into
mainfrom
prepare-stream-startup

Conversation

@nclack

@nclack nclack commented Sep 19, 2026

Copy link
Copy Markdown
Member

Starting capture immediately after stream creation can put initial page faults and worker startup into the first append. Prepare buffered input, CPU payload/output buffers, and shard tail/footer buffers during creation, and wait for worker startup before returning. This pays the memory and initialization cost before capture; the README explains explicit camera buffering, bounded drains, and sizing for a pause allowance.

Validation: all 50 non-S3 CPU checks passed. The memory-residency regression fails when the input-page preparation is removed and passes after restoration.

@codecov

codecov Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.28571% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/platform/platform_memory.c 75.00% 1 Missing and 1 partial ⚠️
Files with missing lines Coverage Δ
src/cpu/stream.c 81.85% <100.00%> (+0.23%) ⬆️
src/stream/host_output_pool.c 75.18% <100.00%> (+0.37%) ⬆️
src/threadpool/threadpool.c 79.56% <100.00%> (+1.26%) ⬆️
src/writer.buffered.c 90.28% <100.00%> (+0.40%) ⬆️
src/zarr/shard_delivery.c 74.13% <100.00%> (+0.22%) ⬆️
src/platform/platform_memory.c 75.00% <75.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Retain startup page preparation alongside GPU host placement and shard preparation. Resolve the README additions and platform source list together.
@nclack
nclack merged commit 93a29ab into main Sep 23, 2026
9 checks passed
@nclack
nclack deleted the prepare-stream-startup branch September 23, 2026 22:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant