Repository navigation
Release v2.2.5 (Data Layer: mesh TLS rotation, IDR-127 config, IDR-129 Query fix) - #49
Merged
Merged
Conversation
A docs-only journal commit cost a full 30 min CI run (run 39201550345: the single unittest discover step took 1583 s). Pull requests into dev now run only the suites their changed paths need; everything else stays full. - scripts/ci_change_scope.py: classify BASE...HEAD into full/python/rust/ contracts (full for non-PR events, PRs into main, label ci:full, unknown base, CI-defining files); --verify-needs backs the single ci-result status. - scripts/ci_test_shard.py: deterministic module shards of discover -s tests; import failures stay failing, shard 1 verifies the partition. - ci.yml: changes job, job-level gates, unit-tests as 4 shards, release-gates (former unit-tests steps unchanged), rust-workspace with exact-key cargo cache, always-on repository-scan and consolidated preflight, ci-result, workflow_dispatch, per-PR cancel-in-progress. - p184_local_ci: matrix expansion, receipt scope full/affected, --changed-since; p184_consolidated_build refuses non-full receipts.
…green full ancestor (CI1-CI6)
…uite (#34) ci: scope jobs by changed paths and shard unit tests
…cord the remaining drift Read-only docker inspect of every DL role against origin/main b5ab3c2 and the v2.2.4 certificate: all image source revisions are on main and all 18 certificate rows equal the live containers. Records the open drift (pre-KN stable compose, IDR-127 manifests/catalog r10/r11 vs r11/r12, stale create-compose caps, missing V1 tag) without changing runtime.
docs: reconcile the running v2.2.4 runtime with main (and journal the v2.2.4 publication)
Boot order with per-step checks, the docker-start-only rule, the stage B cache rebuild mechanism, rollback/fallback and a proposed observe-only KN boot unit packet (not installed). Journals the host-side retirement of qdl-v2-stable-boot-recovery.service. Rebuild measurement pending behind the host measurement lock.
…#37) docs(runbook): KN host reboot recovery; retire the legacy boot unit
…), versioned KN compose, mesh TLS rotation tooling - consumers/stable: alpha-okx r15 and trading-system r11 = deployed realm bytes; catalog r12 = deployed bytes (712 -> 716: exactly the four OKX BTC-USD-SWAP inverse bindings); acquisition r20 port-less with the inverse L2 materialization the core already runs; routing r29, demand r8, promotion scope r11; tests to deployed truth. - deploy/compose/qdl-v2-stable-kn.yml + v2.2.4 env rendered from live inspect (parity PASS 847 checks); liveness healthchecks for stream/projector/core, relaxed cadence for Kafka/Query/stable_redis. - scripts/rotate_mesh_tls.py (export/generate/verify), phase80 generator issues the KN principals, scripts/kn_mesh_rotation_rehearsal.py (isolated window rehearsal), python release image tool -> v2.2.5. - Plan journal entry; window packet and certificate scope drafts.
# Conflicts: # DATA_LAYER_UNIFIED_IMPLEMENTATION_PLAN.md
… rehearsal records offsets, cursors, per-route recovery, BOOTSTRAP->STRICT writer recovery and a re-roll phase
…91 s), TLS/parity/build/semantic receipts, packet + per-gate certificate scope, Codex dispositions; rehearsal readiness needs a fresh heartbeat
…consumer uid, public 0444, dirs 0750) with harden/perms modes; packet perms check, runbook link, start order
… rotation tooling + rehearsal (#38) DL v2.2.5: IDR-127 config (catalog r12, manifests r11/r15), versioned KN compose + healthchecks, mesh TLS rotation tooling
… (fixed Query before the writer recreate, RSS hold rule, delta proof for G5/G6)
DL v2.2.5 drafts: IDR-129 slot in window packet and certificate scope
…he hardened generation (one run, not two)
DL v2.2.5 drafts: final pre-window rehearsal item
…R-129 Query KN RSS retention
docs(plan): journal the 2026-10-19 host cleanup and open IDR-129
…is not supported)
docs(plan): IDR-128 portal cpu.weight A/B result
docs(v2.2.5): W12 decision record - TS writer proof package (verdict W12-A)
…(494 s) Isolated F3 drill 2026-10-19: production state topics copied read-only (assign mode, no group, no commit) into a drill broker; one projector c7ce7d3 with stage A and cleaner off rebuilt an empty cache in 494 s (bars 389 s, latest 494 s), 706 products published, 47,728 keys / 1.16 GiB. Copy impact on production brokers, resources, cleanup counts and what the number excludes are in section 5; plan journal entry added.
docs(runbook): KN market_cache rebuild measured on an isolated stack (494 s)
…nal rehearsal burst check - packet: W12a BOOTSTRAP -> W12b STRICT with rendered composes, monitor W14 on TS v2.0.1 or deferred, old-identity holders and cleanup, K6 exact pre/post compare, outage and one total deadline (F3 494 s contingency), R1-R4 exact commands, go/no-go checklist - certificate scope: IDR-129 delta proof items for G5/G6, W12-A decided - scripts/kn_window_writer.py: render / ready / soak (read-only) + unit tests - rehearsal tool: probes as root for the hardened generation, Query 2 GiB cap, --burst 64-warmup RSS check, --expect-query-image - plan journal entry incl. Codex final-phase review dispositions
Root cause (isolated 64-warmup bursts on the production image, RSS 472 -> 683 MiB over 12 bursts): a warmup body went out as one transport.write, so each TLS connection's ssl.MemoryBIO kept the capacity of its largest body; closed connections and refused-item tracebacks are reference cycles that the Query GC thresholds practically never collect; glibc kept the freed arena memory. - SlicedBodyResponse: warmup and warmup:batch bodies in 256 KiB more_body slices (same bytes, Content-Length, status). - assemble_batch releases each refused item's traceback (IDR-114 residual). - memory_maintenance: gc.collect + malloc_trim when RSS >= soft limit (QDL_QUERY_MEMORY_SOFT_LIMIT_MIB, default 448) and the large-warmup lane is idle, at most every 120 s, one qdl.stall line. - scripts/idr129_query_rss_isolated.py: the isolated reproduction harness. - tests/test_query_idr129_memory.py, plan journal entry.
v2.2.5 window packet: ready-for-go items, W12-A writer tool, final rehearsal burst check
# Conflicts: # DATA_LAYER_UNIFIED_IMPLEMENTATION_PLAN.md
fix(query): IDR-129 Query RSS kept after warmup bursts
…l PASS, DR snapshot step, IDR-128 limitation - python image built from dev 9b6747f (753604f8): receipt PASS, 20 files = git diff, pip equal - deploy/compose/qdl-v2-stable-kn.v2.2.5.env QDL_IMAGE_QUERY -> 753604f8 - final isolated rehearsal receipt (A->B->C->D + 64-warmup burst gate PASS, one open 401 observation) - packet: W0.5/row 17 TS DR equivalence before-snapshot, W14 on TS v2.0.1, outage 5-6 min (up to ~9) - certificate scope 3b + release notes draft: IDR-128 supported-load limitation (owner) - whole-token auth/protocol matching in the rehearsal tool, soak tool and R1 grep
v2.2.5: Query image 2.2.5-9b6747f re-pin, final rehearsal receipt, DR snapshot step, IDR-128 limitation
…S under P1, release notes - scripts/kn_versioned_compose.py parity: compare cpus within 1e-6 vCPU (compose passes cpus through float32: 2.25 -> NanoCpus 2249999872); tests in test_kn_stable_compose. R2 re-run read-only against live 2026-10-20T03:00:55Z: PASS, 863 checks, 0 problems. - Window 2026-10-20 receipts (non-secret byte copies + SHA256SUMS) under upgrade/evidence/releases/v2.2.5/window-20261020T0128Z. - Drafts moved to final files; packet section 10 records the window result. - CERTIFICATE.md / certificate.json / endpoint-report.json: status PASS under P1 (POST_HOC_PACKAGING_DECISION, owner confirmation required): the predeclared G5 item "concurrent hot-read latency" was not measured; gates_without_p1 keeps G5 NOT_MEASURED. Other gates inherited from v2.2.4 with delta proofs; SDK 2.4.1 wheel unchanged. - CERTIFICATION_LEDGER.md and the plan journal updated.
…armup bursts + warmup byte compare) scripts/v225_query_hot_read_ab.py: old Query 3e9e4d3a vs new 753604f8 on one private read-only copy of the market cache; open-loop TS EXECUTION snapshot reads at 10/s during 12 x 64-warmup bursts (incl. memory-maintenance runs), and an A/B/A' warmup byte comparison of every TS r11 requirement. Pass rules HR1-HR5 and BC are fixed in PREREG before the run.
…a foreign correctness lock, interleaved A1 B1 A2 B2, paired-delta rules, foreign activity sampling Changed before any run; v1 (7d28d9c) was never executed.
…pec (new-generation client CA bundle) Attempt 1 (2026-10-20 04:50Z) dropped /consumer-realms/query-client-ca-bundle.crt, so the isolated replicas refused the new TS client certificate; nothing was measured. Rules unchanged.
…ery spec (/session-liveness) Attempt 2 (05:02Z) bound the stable_state volume root, so every latest read was DATA_STALE (SESSION_STATE) on both images; stopped after 3 requirements, not scored. Rules unchanged.
…are-only rerun mode Attempt 3's BC driver could not write its raw record (/out not writable for the capped root driver); the per-requirement statuses survive in cmp/driver.out, the bodies by sha. The BC phase is re-run in the same session dir after the arms. The analysis now masks the raw token at each volatile path exactly, as the pre-registered BC rule states (the driver skipped small integers such as per-request ages). Rules unchanged.
…compare; certificate PASS without P1 - Session query-ab-20261020c (2026-10-20 05:04-05:50Z, owner "Đo luôn rồi tag", run beside other agents' jobs by owner instruction, arms interleaved A1 B1 A2 B2): verdict PASS. Hot reads (TS EXECUTION QUOTE/TRADE snapshots, 10/s) during 64-warmup bursts: paired mean d_p95 -41.9 ms, d_p99 -19.5 ms; maintenance runs 131/147 ms covered; non-200 equal. - Warmup byte compare old 3e9e4d3a vs new 753604f8, 65 TS r11 requirements x 2 passes: byte-identical outside per-request paths (byte-compare.json 41ebcc3c). - Certificate, endpoint report, release notes, scope header, ledger and plan updated; P1 superseded (never used). Harness analysis compacted; unit tests for masking/percentiles.
…d shapes, alpha-binance r16 + alpha-okx r15 BAR 1m) Same BC rule; --alpha re-runs only the compare phase in the session dir; analyze-alpha writes byte-compare-alpha.json.
…dence updated 24 alpha read shapes (alpha-binance r16 + alpha-okx r15 BAR 1m) x 2 passes on old 3e9e4d3a vs new 753604f8: statuses equal (44 x 200, 4 x 400), bodies byte-identical outside per-request paths (byte-compare-alpha.json c884e495). Certificate, endpoint report, ledger and plan.
…A/B + byte compare) (#48) v2.2.5: window result, parity cpus tolerance, Query A/B (G5 D3b + DR byte compare), certificate PASS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release PR dev -> main for Data Layer v2.2.5. Head: dev
098c94b4(merge of #48). Tree1e621911. Main is at v2.2.4b5ab3c2.Contents
deploy/compose/qdl-v2-stable-kn.yml, plus per-release env files that pin images by digest.cpuswithin a 1e-6 tolerance.qdl-v2-python:2.2.5-9b6747f, digest753604f8. Rust digests are unchanged. The TS writer2deb90b0is unchanged (W12-A).Certificate: PASS
upgrade/evidence/releases/v2.2.5/CERTIFICATE.mdandcertificate.json. No post-hoc decision is used; P1 was superseded by measurement.gates_without_a5).Evidence
window-20261020T0128Z/,SHA256SUMS):query-ab-20261020/):receipt.jsonverdict PASS.byte-compare.json41ebcc3c...: TS r11, 65 requirements x 2 passes. Bodies are byte-identical outside per-request fields.byte-compare-alpha.jsonc884e495...: alpha DR02/DR03 read shapes, 24 x 2. Bodies are byte-identical outside per-request fields.9f79e032is unchanged; its rebuild is byte-identical.release.ymlis unchanged. Its checks were simulated locally and pass.Expected CI
The head tree is the dev push tree, so CI1/CI9 inheritance applies:
ci-resultshould record the inherited run. Branch-dependent checks still run.Limits
Do not merge without the coordinator. The coordinator creates the v2.2.5 tag after the merge.