feat(sync): author a repo's memories to a public stream when published (#1157) - #1158
Merged
Conversation
#1157) Groundwork for a dedicated public knowledge-base node (owner writes, anyone reads anonymously). A repo can now be marked public, after which its memories are authored to a public_read `/2` owner stream so the account is servable to anonymous readers; until then nothing changes. - `sync publish` marks the active repo's account as a public knowledge base: it persists a one-way `memory_stream_access_mode` intent and ensures the public_read `/2` stream. It refuses — rather than strand the account — when the account already holds a private stream (publishing an existing private repo is not supported; start a fresh public index), when sealing is intended (a public reader cannot unwrap sealed content), or on a legacy/local-only repo id. - Memory authoring resolves the owner stream's access mode through one seam, `owner_stream_access_mode`: the persisted intent, falling back to the op-log fact (an already-owned public stream) so the one-way ratchet survives loss of the intent row. Every `/2` resolver — live-write, drain, reconcile, catch-up — threads that mode, so they all target the same stream id; the access mode folds into the stream identity, so a public and a private stream are distinct and a mode split would otherwise mix the account permanently. - Publish and sealing are mutually exclusive in both directions. - Consolidation carries a lone public intent onto its fresh target and refuses two disagreeing explicit modes. Serving a published account under public-read admission, and seeding an existing index's memories onto a fresh public node, follow in later changes.
Contributor
|
Bencher Report| Project | rag-rat |
| Branch | feat/public-node |
| Testbed | ubuntu-latest |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
- RAM Hits (hits)
- Instructions (instructions)
- LL Hits (hits)
- L1 Hits (hits)
- Estimated Cycles (cycles)
- Total read+write (reads/writes)
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | Estimated Cycles | cycles x 1e6 | Instructions | instructions x 1e6 | L1 Hits | hits x 1e6 | LL Hits | hits x 1e6 | RAM Hits | hits x 1e3 | Total read+write | reads/writes x 1e6 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| rag_pipeline::pipeline::index cargo_resolver:resolver_config() | 📈 view plot | 3,883.00 x 1e6 | 📈 view plot | 2,580.90 x 1e6 | 📈 view plot | 3,552.18 x 1e6 | 📈 view plot | 56.73 x 1e6 | 📈 view plot | 1,346.97 x 1e3 | 📈 view plot | 3,610.26 x 1e6 |
| rag_pipeline::pipeline::query_cold cargo_resolver:resolver_built_config() | 📈 view plot | 340.40 x 1e6 | 📈 view plot | 226.70 x 1e6 | 📈 view plot | 323.59 x 1e6 | 📈 view plot | 3.14 x 1e6 | 📈 view plot | 31.27 x 1e3 | 📈 view plot | 326.77 x 1e6 |
| rag_pipeline::pipeline::query_warm cargo_resolver:resolver_index() | 📈 view plot | 313.64 x 1e6 | 📈 view plot | 208.39 x 1e6 | 📈 view plot | 297.88 x 1e6 | 📈 view plot | 2.92 x 1e6 | 📈 view plot | 32.51 x 1e3 | 📈 view plot | 300.84 x 1e6 |
Contributor
|
| Project | rag-rat |
| Branch | feat/public-node |
| Testbed | ubuntu-latest |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result seconds (s) (Result Δ%) | Upper Boundary seconds (s) (Limit %) |
|---|---|---|---|
| index_time/full_rebuild_cargo | 📈 view plot 🚷 view threshold | 6.19 s(+12.05%)Baseline: 5.53 s | 6.25 s (99.03%) |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for a dedicated public knowledge-base node (#1157) — the owner writes, anyone reads anonymously. A repo can be marked public, after which its memories are authored to a
public_read/2owner stream so the account is servable to anonymous readers (the transport for that merged in #1150–#1153). Until a repo is published, nothing changes.What changed
rag-rat sync publishmarks the active repo's account as a public knowledge base: persists a one-waymemory_stream_access_modeintent and ensures thepublic_read/2stream. It refuses — rather than strand the account — when the account already holds a private stream (publishing an existing private repo is not supported; start a fresh public index), when sealing is intended (a public reader cannot unwrap sealed content), or on a legacy/local-only repo id.owner_stream_access_mode: the persisted intent, falling back to the op-log fact (an already-owned public stream) so the one-way ratchet survives loss of the intent row. Every/2resolver — live-write, drain, reconcile, catch-up — threads that mode, so they all target the same stream id. The access mode folds into the stream identity, so a public and a private stream are distinct; a mode split would otherwise author a secondStreamOwnand mix the account permanently (an append-only, unservable state).Mechanism only:
owned_stream_v2_id_with_mode/established_owned_stream_v2_with_modedefault toPrivate, so this is a no-op untilsync publishsets the intent.Tests
Publish is idempotent and makes the account fully public + authors public; a repo with prior private memories refuses publish; publish×seal are mutually exclusive both directions; the access-mode ratchet survives a deleted intent row (no second private
StreamOwn); consolidation carries the public intent and rejects an unknown token.Scope
Serving a published account under public-read admission (flip the serve policy), and seeding an existing index's memories onto a fresh public node, follow in later changes.
Verification
cargo +nightly fmt --check; clippy--workspace --all-targetsunder--no-default-featuresand--all-features, plus the--features evalgates — all-D warningsclean;cargo nextest run --workspace(5005) andcargo test -p rag-rat-core -p rag-rat-oplog(both CI runners).