feat(storage): add CHIRP reference implementation

[ty-everett]

Program and scope

• Tracker or issue: BRC-167 proposal
• Program gate(s) advanced: CHIRP v1 reference package/CLI, portable conformance vectors, and CHIRP capability in both governed UHRP storage-server implementations.
• Why this change is needed: large UHRP payloads currently require whole-object transfer before the caller can establish integrity. CHIRP adds canonical chunking, merklization, progressive verification, resilient multi-host reads, ranges, resumable publication, and partial-host closure semantics while retaining BRC-26/UHRP identifiers and APIs.
• Explicitly out of scope: npm or container publication, deployment, changing legacy UHRP uploader/downloader behavior, defining additional chunking profiles, or treating draft BRC-167 as final.
• Exact head SHA reviewed: 947fadf34b39bebf98584e3ee465abfbf48302b9

The wire format follows the exact draft serialization in BRC-167. Until that proposal is merged, this implementation and release must track any normative BRC change.

Impact

• Public package source or manifest changed; affected packages are listed below
• Infrastructure source, dependency, image, or deployment configuration changed
• Public API, exports, types, runtime targets, or browser/mobile behavior changed
• Security-sensitive boundary changed
• Documentation or examples changed

Affected packages/services and intended patch versions (publication occurs only through the release workflow after approval):

• New @bsv/chirp@0.1.0 ESM/browser/Node package and chirp CLI.
• @bsv/uhrp-lite@0.1.33 with filesystem-backed CHIRP object/session/lease/GC support and Docker volume wiring.
• @bsv/uhrp-storage-server@0.2.35 with GCS-backed CHIRP object/session/lease/GC support.
• Existing StorageUploader, StorageDownloader, StorageUtils, BRC-26 identifiers, /upload, /find, /list, /renew, and legacy object serving remain unchanged and covered by regression tests.

Verification

• Local commands and results: pnpm build, pnpm typecheck, pnpm test, pnpm lint, pnpm format:check, pnpm health:check, pnpm check-versions, pnpm conformance, pnpm docs:examples, pnpm codegen:check, pnpm audit:security, pnpm license:pack-check, and the exact Node 24.18 npm release-artifact stage/verify/dry-run all pass. Both standalone storage services pass build/lint/test, and docker compose config --quiet passes with validation placeholders.
• Hosted CI evidence: exact-head CI run 32913187589, container runtime run 32913187665, CodeQL run 32913187688, and conformance run 32913187609 are terminal and successful. Scope-based skips were validated by the merge gate.
• Conformance evidence: 76 files / 6,690 vectors parse cleanly; the TypeScript dispatcher executes 6,479 and records 211 governed skips. All nine CHIRP portable cases pass, including the chunk-boundary and 256-, 257-, and 65,537-leaf tree-shape vectors.
• Coverage delta: CHIRP Jest coverage is 94.47% statements, 92.82% branches, 88.35% functions, and 95.31% lines. Exact-head Codecov patch coverage is 91.97% with the repository-required 90% gate passing.
• Lint/typecheck delta: zero findings across the workspace and both standalone services.
• Browser/mobile/packed-consumer evidence: clean ESM package, CLI-bin, OpenAPI subpath, and browser consumers pass. Browser artifacts are within policy: Vite 482,343 raw / 124,219 gzip / 104,577 Brotli; esbuild 373,491 raw / 114,505 gzip / 98,075 Brotli.
• Performance or bundle-size delta: governed browser budgets pass. The codec mutation target scores 94.16%; profile-1 property tests use at least 300 runs with replay support.
• I self-reviewed the complete diff for correctness, security, compatibility, public API, artifacts, dependencies, docs, and operations
• All applicable checks are terminal and successful on the exact head; any scope-based skip is expected and validated by the merge gate

Security and dependencies

• Changelog, runtime relevance, peer compatibility, transitive graph, and audit results were reviewed
• CodeQL/negative tests cover any changed trust boundary
• The exact-head CodeQL analysis has no new alert
• The exact-head repository quality gate reports zero new Sonar findings (including accepted or false-positive issue states) and zero unreviewed hotspots; Sonar's aggregate Quality Gate passed verdict alone is not merge evidence
• No new override, advisory dismissal, quality suppression, or skipped test
• Any temporary exception is registered with owner, evidence, review date, and removal condition
• Workflow permissions and lifecycle-script behavior remain least privilege

Negative coverage includes canonical CompactSize rejection, critical extensions, malformed/oversized/cyclic closures, hash and length mismatches, abort/timeout behavior, redirect and private-network protection, URL policy enforcement, checkpoint poisoning, incomplete/mismatched commits, authorization, expiry, rollback, lease renewal, and live-set garbage collection.

The existing governed uhrp-standalone-services duplication boundary now has precise CPD exclusions for byte-synchronized CHIRP copies. Those files remain in Sonar source analysis; only intentional copy/paste detection is excluded, and byte equality remains enforced by repository health.

Dependency evidence

• Release notes and necessity: @bsv/chirp is a new release unit documented in package/release governance. The conformance runner links it as a workspace-only development dependency to execute required vectors.
• Runtime, build, and peer compatibility: @bsv/sdk ^2.4.1 is the only peer; Node 22/24 and browser targets are governed. Existing UHRP services retain their current runtime model.
• Deduplicated lockfile: pnpm install --offline succeeds with the committed workspace lock; both standalone npm lockfiles install and test successfully.
• Audit and CodeQL: root audit passes with two existing governed high advisories ignored; exact-head JavaScript/TypeScript and Actions CodeQL analyses pass with no new alert.
• Package and consumer tests: 109 CHIRP unit/edge/CLI/property tests, clean packed consumers, legacy SDK storage regression tests (58), basic server tests (32), and cloud server tests (25) pass.
• Bundle and performance impact: CHIRP browser outputs remain under all raw/gzip/Brotli budgets; no legacy SDK bundle is enlarged because CHIRP is a separate package.
• Affected public package versions: @bsv/chirp@0.1.0, @bsv/uhrp-lite@0.1.33, @bsv/uhrp-storage-server@0.2.35.

Release and operations

• No npm publication was performed from a workstation or from this PR
• Required npm patch bumps are included or intentionally deferred by the controlling program
• Image/SBOM/provenance/deployment/rollback impact is documented
• Documentation, changelog, migration, and operational guidance are current

CHIRP routes are additive and authenticated. Hosts advertise a root only after closure validation and successful commit. Partial hosting remains representable at the object layer but incomplete closures are never advertised as complete roots. Rollback is removal/disablement of the additive CHIRP routes and data namespaces; legacy UHRP data and endpoints are unaffected.

Completion evidence

• The linked tracker is updated only for work fully proved by merged code, passing checks, resolved alerts, measurements, or an approved exception
• Review conversations are resolved
• Documentation, changelog, migration notes, release notes, and operator guidance are current or concretely not applicable
• No pending, failed, stale, cancelled, or unexpectedly skipped check is being handed to another contributor as “complete”
• One qualified maintainer approval is sufficient; no last-pusher restriction is assumed

[codecov]

Codecov Report

❌ Patch coverage is 91.06464% with 94 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
packages/network/chirp/src/uploader.ts	83.62%	19 Missing and 9 partials ⚠️
packages/network/chirp/src/cli.ts	82.35%	13 Missing and 8 partials ⚠️
packages/network/chirp/src/resolver.ts	93.01%	8 Missing and 11 partials ⚠️
packages/network/chirp/src/validation.ts	87.25%	6 Missing and 7 partials ⚠️
packages/network/chirp/src/codec.ts	93.24%	5 Missing and 5 partials ⚠️
packages/network/chirp/src/cache.ts	92.59%	0 Missing and 2 partials ⚠️
packages/network/chirp/src/hash.ts	96.15%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[BraydenLangley]

Before merge, I think four CHIRP issues need to be addressed:

1. The resolver trusts a profile-1 leaf's declared logicalLength as its response bound. A tiny logical range can therefore make it fetch and buffer a multi-gigabyte blob, bypassing maxDownloadBytes. Profile 1 should enforce the 4 MiB leaf bound during traversal, with a safe absolute per-object ceiling for other profiles.
2. Closure validation retains every unique blob in blobCache, so committing a conforming large object can retain approximately the entire logical content in process memory despite the 11 GB default limit. Validation caches need aggregate bounds or streaming/re-read behavior.
3. Each public object GET reloads, parses, and linearly scans the complete commit record. The 100,000-object and ordinary 11 GB profile-1 cases should not be conflated, but the cloud implementation still creates a material unauthenticated bucket-read/CPU amplification path and should use bounded cached/indexed membership with renewal invalidation.
4. Renewal broadcasts the new advertisement before extending the closure lease. If lease extension then fails, the advertised availability exceeds local retention, contrary to BRC-167. The safe ordering is to durably extend the closure before broadcast and reconcile ambiguous broadcast failures; blindly rolling the lease back would also be unsafe.

Relatedly, profileCanonical should only be true after profile-1 construction has actually been validated, not merely because the root declares profile 1. These look like merge blockers rather than theoretical concerns, even though the current CI head is green.

[ty-everett]

Addressed on exact head 947fadf34b39bebf98584e3ee465abfbf48302b9. Thanks for the concrete review.

1. Profile 1 now rejects leaves above 4 MiB during traversal. Unknown future profiles use a configurable absolute per-object ceiling (64 MiB by default), and every referenced length is enforced while streaming even when Content-Length is absent.
2. Closure validation no longer retains blob/node bodies. It re-reads one bounded object at a time and retains only bounded metadata, unique-object membership, and a separately bounded total-reference count.
3. Both storage servers now use bounded TTL commit-membership indexes with aggregate root/object limits, request coalescing, negative caching, O(1) membership, and generation-aware invalidation so stale in-flight reads cannot overwrite renewals or commits.
4. Renewal now durably extends the CHIRP closure (or legacy file record) before SHIP broadcast. Ambiguous broadcast failures retain the durable extension and are never blindly rolled back.
5. profileCanonical is now true only after a complete profile-1 traversal validates exact chunk boundaries and canonical tree shape; ranges and unvalidated/future profiles report false.

The companion BRC-167 update also fixes the v1 child-count ceiling at 256 independently of profiles, relaxes Content-Length to SHOULD while retaining strict stream-length verification, makes stagingExpiresAt a canonical decimal uint64 string, and adds deterministic chunkSize−1/exact/+1 plus 256-, 257-, and 65,537-leaf portable vectors. The incompatible/dead staging configuration path was removed.

Exact-head hosted evidence is green: CI, container runtime contracts, CodeQL, and conformance. The Sonar PR API reports zero open/confirmed findings and the exact-analysis zero-findings gate passed. CHIRP has 109 passing tests; conformance executes 6,479 of 6,690 vectors with 211 governed skips. Ready for re-review.

[BraydenLangley]

Approved! Just CI issues need resolved on main.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
2.0% Duplication on New Code

See analysis details on SonarQube Cloud