Skip to content

Promote latest main to testnet-canary - #1638

Merged
branarakic merged 12 commits into
testnet-canaryfrom
main
Jul 13, 2026
Merged

Promote latest main to testnet-canary#1638
branarakic merged 12 commits into
testnet-canaryfrom
main

Conversation

@branarakic

Copy link
Copy Markdown
Contributor

Summary

Promote the current main head (22a2225289464e06a20fac274053a3b0a75eda09) into testnet-canary using the normal merge-based canary flow.

This preserves the canary branch history while bringing in all changes currently on main. PR #1539 was reverted by #1637 before this promotion, so oxigraph-worker compatibility remains present in the resulting tree.

Branch state

  • source: main at 22a2225289464e06a20fac274053a3b0a75eda09
  • target: testnet-canary at 378114f8726ef3fb9b0e2df0bb51f0f615d89676
  • no existing open main to testnet-canary promotion PR

Verification

Full promotion CI is expected to run on this PR before merge.

branarakic and others added 12 commits July 13, 2026 08:57
… (#1617)

* fix(core,mcp): escape all C0 control chars + DEL in RDF literals (#416)

escapeDkgRdfLiteral (core) and its byte-identical MCP copy escapeRdfLiteralBody
only escaped the 7 ECHARs (\\ " \n \r \t \f \b), so NUL (0x00), VT (0x0B),
DEL (0x7F) and the rest of 0x01-0x1F passed through raw — producing an invalid
N-Triples/N-Quads STRING_LITERAL_QUOTED that the store rejected or stored
corrupted.

Both escapers now map every remaining U+0000-U+001F and U+007F to a \\uXXXX
UCHAR (uppercase, 4-digit); the five with ECHAR short forms keep them so the
byte output for already-covered chars is unchanged (stable merkle leaves).

- packages/core/src/publisher-extension.ts — escapeDkgRdfLiteral
- packages/mcp-dkg/src/tools/assertions.ts — escapeRdfLiteralBody (kept byte-identical)
- packages/mcp-dkg/test/rdf-object-normalization-conformance.test.ts — golden fixture
  cases for NUL/VT/US/DEL + escaper assertion (per the drift-guard back-pointer)
- packages/core/test/publisher-extension.test.ts — control-char coverage + ECHAR-stable assertion

core suite 1186 passed; mcp conformance 30 passed.

Fixes #416

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(rdf): share literal escaping policy

* fix: centralize RDF object normalization

* build: follow workspace dependencies for runtime builds

* test: cover Hermes RDF control escapes

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
… publish (#1575) (#1618)

* fix(publisher): stop the doomed post-confirm chain.verify on V10 sync publish (#1575)

Every confirmed V10 publish with a target context graph ran a legacy V9-era
explicit chain.verify() fallback in the publish tail. On V10 the KC is ALREADY
registered to the context graph inside publishDirect via a Hub-authorized
internal call to ContextGraphs.registerKnowledgeAsset (EOAs can't call it), so
the explicit verify always reverted with "Only Contracts in Hub" / CALL_EXCEPTION
— a guaranteed-to-fail on-chain call plus an estimateGas round-trip and
per-wallet serializer occupancy on EVERY publish.

Skip it: registration is already done. `registered` stays true so the GH#842
last-writer-wins per-cgId data promotion still runs (removing that would strand
published KCs from RS random sampling). Only the verify attempt is removed.

- packages/publisher/src/dkg-publisher.ts — replace the verify/sign/estimateGas
  block with an unconditional registered=true + a debug log.
- packages/publisher/test/dkg-publisher.test.ts — assert chain.verify is NOT
  called on a confirmed V10 publish (real EVM adapter, hardhat).

publisher suite 1482 passed; the new assertion is green against the real chain.

Fixes #1575

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(publisher): cover SWM verify removal

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…triples (#1161, #1270) (#1619)

* fix(query): DISTINCT on read-both graph unions — collapse dual-homed triples (#1270)

A scoped query (no explicit view) reads a UNION across the canonical root graph
and the per-cgId / per-KA …/_verifiable_memory partitions (wrapWithGraphUnion).
Finalization intentionally DUAL-HOMES a confirmed triple — PR #1098 made the
per-KA VM partitions queryable for pre-subscribed peer recovery, and promote
mirrors the canonical quads into the root graph — so the SAME triple matches in
two union branches and returns duplicate solution rows.

This surfaced as the flaky e2e-finalization failure "A enshrines; B promotes":
after B promotes, SELECT ?name WHERE { <E> schema:name ?name } returned 2
bindings instead of 1. (Reproduced at the query layer: ROOT+VM → 2 bindings.)

Fix: emit SELECT DISTINCT for the multi-graph union branch in wrapWithGraphUnion,
collapsing mirror duplicates. No-op for a single graph (a graph is a set), so only
the union path needs it; CONSTRUCT/ASK/DESCRIBE are untouched. The data is
correctly dual-homed — this is a read-path dedup, not a change to the writes.

Hardened the existing dedup probe into a strict regression test (each dual-home
layout must yield exactly ONE binding). Full query suite green (285).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 2f0195c)

* fix(query): dedupe mirrors without changing bag semantics

* test(agent): expect deduplicated VM mirrors

* fix: constrain VM mirror dedup rewrites

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(agent): keep outbox retries on schedule

* test(agent): pin scheduled-only reconnect behavior

* test(devnet): cover scheduled-only outbox retries

* refactor(outbox): make scheduled-only retries canonical

* test(outbox): preserve snapshot isolation through list

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* fix(agent): keep storage ACK retries collector-owned

* fix(agent): clear request-owned SLO state

* refactor(agent): make reliable failure policy explicit

* test(devnet): cover StorageACK retry ownership

* refactor(agent): split request-owned reliable sends

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* fix(cli): reject unclaimable async publish jobs

* fix(cli): classify publisher readiness recovery

* fix(cli): centralize async publisher readiness

* test(devnet): cover async publisher readiness gate

* test(cli): pin publisher availability lifecycle states

* test(cli): provision async publisher in live daemon e2e

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* fix(chain): retry reads after full-pool throttling

* fix(chain): scope throttle retries to critical reads

* fix(chain): retry only all-throttled receipt block reads

* refactor(chain): contain full-pool throttle retry policy

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* fix(cli): verify and roll back torn global updates

* fix(update): verify exact CLI version before commit

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* refactor(storage): retire oxigraph-worker backend

* fix(cli): close store migration review gaps

* test(storage): align main coverage with worker retirement

* test(cli): isolate lifecycle wiring from store startup

* fix(cli): address store retirement review

* test(cli): make admission saturation deterministic

* refactor(storage): centralize backend taxonomy

* refactor(storage): clarify backend boundaries

* test(cli): materialize managed store in startup mocks

* fix(storage): separate adapter and daemon policy

* fix(storage): enforce runtime backend capabilities

* test(cli): pin operator config at request boundary

* test(cli): wait for publisher readiness

* refactor(cli): expose canonical route store views

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
This reverts commit 62ab623.

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* fix(agent): bound periodic outbox drains

* fix(agent): prevent bounded outbox page starvation

* fix(outbox): formalize bounded drain contract

* test(devnet): cover bounded outbox scheduling

* refactor(outbox): isolate shutdown-safe bounded drainer

* test(outbox): assert lexical SQLite tie ordering

* test(outbox): close shutdown and paging regressions

* test(outbox): pin production paging boundary

* fix(outbox): preserve store compatibility

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
* fix(publish): isolate named KA shared-memory scope

* fix(storage): expose exact named KA SWM reads

* fix(publisher): reconcile exact SWM cleanup ownership

* fix(swm): enforce exact named knowledge asset scope

* fix(storage): preserve checksum casing in exact SWM lookup

* fix(storage): isolate named SWM lifecycle scopes

* fix(storage): close named lifecycle clear boundary

* fix(storage): preserve finalized cleanup contracts

* fix(agent): retain legacy queued KA scope fallback

---------

Co-authored-by: Branimir Rakic <aleatoric@Branimirs-MacBook-Pro.local>
@branarakic
branarakic merged commit dbc0b3a into testnet-canary Jul 13, 2026
16 checks passed
// on-chain call plus an estimateGas round-trip, and serializer occupancy,
// on EVERY confirmed publish (#1575). Registration is already done, so
// skip the verify attempt entirely and proceed to the data promotion.
this.log.debug(ctx, `V10 auto-registered KC to context graph ${targetCgId}; explicit verify skipped`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug: Context-graph registration is skipped for every chain adapter, not just V10

What's wrong
The new code removes the only explicit ContextGraphs registration call and assumes every confirmed publish has already registered the KA. That is true for the described V10 publishDirect path, but the public ChainAdapter contract still includes verify and existing adapters may rely on that fallback.

Example
A ChainAdapter whose publish path returns a confirmed onChainResult but relies on verify({ contextGraphId, batchId, merkleRoot, signerSignatures }) to register the KA to the context graph will now return a confirmed publish and promote data locally, while the on-chain ContextGraphs registration never happens.

Suggested direction
Gate the skip on an explicit V10 capability/registration signal, or keep calling chain.verify for non-V10/custom adapters and continue treating the known V10 Hub-authorized revert as non-applicable.

Confidence note
This assumes adapters that implement ChainAdapter.verify remain supported. The interface and EVM/mock adapters still expose verify, but if this release is intentionally V10-only then the compatibility concern should be confirmed by maintainers.

For Agents
In packages/publisher/src/dkg-publisher.ts, keep the V10 fast path only when the adapter/result proves publishDirect already registered the KA. Preserve the old chain.verify fallback for adapters that still need explicit ContextGraphs registration, and add a test with a fake adapter whose verify call is required.

hasEntry(peer: string, protocol: string, messageId: string): boolean;

/** Whether this peer still has any durable row (DHT recovery bookkeeping). */
hasPendingFor(peer: string): boolean;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug: The outbox store interface now requires a new method, breaking existing custom stores

What's wrong
The comment says due(now) remains the required public store contract for compatibility, but hasPendingFor was added as another required method. Existing/custom ProtocolOutboxStore implementations will no longer satisfy the exported interface and can also fail at runtime if consumed from JavaScript.

Example
An out-of-tree store built against the previous ProtocolOutboxStore contract implements enqueue, markDelivered, hasEntry, due, list, and getEntry, but not hasPendingFor. After a send/retry drains the last row, Messenger.clearDhtWalkRateLimitIfDrained calls ProtocolOutbox.hasPendingFor, which calls undefined and crashes at runtime for JavaScript consumers, or fails TypeScript builds for TS consumers.

Suggested direction
Keep the public store contract backward-compatible by making hasPendingFor optional and falling back to list().some(entry => entry.peer === peer), or provide a legacy pendingFor fallback during a deprecation window.

For Agents
In packages/core/src/messenger-types.ts and packages/core/src/protocol-outbox.ts, make hasPendingFor optional or derive it from the required list() method when absent. Add a compatibility test using a legacy due/list-only store to prove Messenger/ProtocolOutbox still work.

admission: AdmissionStatsView,
emitMemoryGraphChanged?: (event: MemoryGraphChangedEvent) => void,
emitNotification?: (event: NotificationSseEvent) => void,
publisherAvailability?: AsyncPublisherAvailability,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Stop growing the daemon request boundary with optional positional state

What's wrong
This makes an already fragile API boundary more brittle. handleRequest is not really a function with 30 independent arguments; it is a request-context factory. Adding another optional tail parameter normalizes positional plumbing and makes future changes harder to review because the important state is hidden in a long call list.

Example
The lifecycle call now passes publisherAvailability after two optional emit callbacks; the callee then stores it as ctx.publisherAvailability. The next daemon-scoped dependency will repeat the same widening pattern.

Suggested direction
Introduce a named dependency object, ideally assembled once in lifecycle and reused per request, so publisher availability is a named field instead of the latest optional positional argument. This also gives future route-wide state a canonical home.

For Agents
Refactor packages/cli/src/daemon/handle-request.ts and the single call in packages/cli/src/daemon/lifecycle.ts to pass a named DaemonRequestDeps/context-builder object. Preserve the route behavior and prove the async publisher unavailable responses still include reason/retryable/operatorActionRequired.

* `SELECT *` and caller DISTINCT semantics. Unsupported/colliding query shapes
* return null so the existing generic multi-graph fallback remains available.
*/
function wrapWithDeduplicatedGraphValues(sparql: string, graphUris: string[]): string | null {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Extract the VM dedup SPARQL rewrite out of the monolithic query engine

What's wrong
The change adds a substantial new mini rewrite engine directly to an already oversized query engine. The raw string slicing, sentinel variables, fallback ordering, and shape scanner are all tightly coupled in one place, which raises the cost of safely changing VM query routing later.

Example
Adding another VM graph source or changing graph-wrapping semantics now requires reading wrapVerifiableMemoryGraphSet, wrapWithProjectedGraphSubselect, wrapWithDeduplicatedGraphValues, sentinel collision handling, and isDedupSafeBasicGraphPattern in one large file.

Suggested direction
Keep the behavior, but isolate this as a dedicated rewrite abstraction with a narrow API. The query engine should route to a named VM graph-rewrite policy, not own the raw SPARQL anti-join construction and safety scanner inline.

Confidence note
This is a structural maintainability concern; the current tests may cover behavior, but the implementation still concentrates too much SPARQL-rewrite machinery in the main query engine file.

For Agents
Move the VM graph-set rewrite policy and helper/sentinel machinery from packages/query/src/dkg-query-engine.ts into a focused module, e.g. verifiable-memory-graph-rewrite.ts. Keep the same fallback order and existing read-both dedup tests, with DKGQueryEngine only invoking the named rewrite function.

type ProviderSetExhaustionKind = 'all-throttled' | 'mixed';

/** Internal exhaustion detail used only while deciding whether to retry a pass. */
class ProviderSetExhaustedError extends ChainRpcTransportError {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Avoid a control-flow error subclass for throttle retry state

What's wrong
This adds an internal error class and couples retry orchestration to runtime error identity. The public read path now has two concepts for endpoint exhaustion: the transport error callers care about and a private subclass used to decide whether to retry. That indirection is avoidable and makes the boundary harder to reason about.

Example
An all-throttled pass throws ProviderSetExhaustedError; the retry wrapper catches it for control flow, and exhausted callers receive a subclassed transport error carrying an internal retry classification.

Suggested direction
Model full-pool exhaustion as a typed internal pass result rather than a thrown subclass. That keeps retry policy explicit in read() and avoids making the public transport error hierarchy double as private control-flow state.

For Agents
In packages/chain/src/rpc-failover-client.ts, split provider-pass execution from public error throwing. Have the pass return a typed exhaustion result carrying { error, allThrottled }, let read() own the retry loop, and throw the existing public ChainRpcTransportError after retries are done. Preserve current tests for all-429 retry, mixed exhaustion, and single-provider messages.

// Legacy/mock seals may carry only the low 96-bit KA number. Preserve that
// compatibility by binding a zero packed namespace to the sealed author;
// a real nonzero namespace must still match exactly.
if (packedAuthor !== 0n && ethers.getAddress(unpacked.agentAddress) !== sealedAuthor) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Finalized KA namespace guard lacks a regression test

What's wrong
This guard protects the new named-lifecycle shared-memory scope from using a KA id in another author namespace. The current tests only prove matching author ids are threaded; they would still pass if the mismatch rejection were removed or inverted, leaving a data-isolation regression unverified.

Example
A regression test could persist a finalized seal with authorAddress = AGENT_B and reservedKaId = (BigInt(DEFAULT_AGENT) << 96n) | 1n, then assert publishFromFinalizedAssertion(CG, NAME, { agentAddress: AGENT_B }) rejects with /not in author/ and does not call _loadSelectedSWMQuads or publishFromSharedMemory.

Suggested direction
Add a negative finalized-assertion test for a reservedKaId whose high 160 bits belong to a different author, and assert no publish side effects occur.

Confidence note
The helper is private, so this is based on reachable behavior through publishFromFinalizedAssertion tests rather than direct helper tests.

For Agents
Add coverage in packages/agent/test/publish-finalized-agent-lane.test.ts for sharedMemoryScopeForFinalizedLifecycle behavior through publishFromFinalizedAssertion: mismatched nonzero packed author must reject before loading/publishing; optionally also cover legacy low-96-bit reservedKaId binding to the seal author.

return jsonResponse(res, 503, {
error: "PublisherUnavailable",
message: "Async EPCIS capture requires the publisher runtime to be running with at least one configured publisher wallet",
reason: publisherAvailability.reason,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: EPCIS publisher-readiness diagnostics are not asserted

What's wrong
The route now exposes user-facing readiness diagnostics, but its test only checks the coarse error name. That gives false confidence for clients/operators relying on the new reason and retryability fields.

Example
If the EPCIS unavailable branch returned { error: 'PublisherUnavailable' } without reason/retryable/operatorActionRequired, or returned retryable: true for no_publisher_wallets, the current EPCIS readiness test would still pass.

Suggested direction
Pin the new EPCIS 503 body fields in route tests, mirroring the knowledge-assets publish-async readiness assertions.

For Agents
Extend packages/cli/test/epcis-route-readiness.test.ts to assert the new response fields for at least publisher_startup_failed/runtime-null, no_publisher_wallets/empty runtime, and publisher_starting via ctx.publisherAvailability. Preserve the existing PublisherDisabled mapping separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants