Skip to content

[PR] storage: GC archive as one append-only JSONL log (follow-up to #3911) - #3927

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-20_archive-jsonl
Oct 20, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-20_archive-jsonl

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Follow-up to #3911 (merged without this commit — it was pushed right at the merge boundary): the storage-review change to the GC archive format.

What: archive/GC no longer writes one export file per archived session. Sessions are small and numerous; a file per session would spray tiny files across the archive dir. Archiving now appends ONE JSON line per session to a single append-only archive/archive.jsonl (a crash mid-append leaves at most one truncated line, which the restore reader skips), and restore rewrites the log without the restored session's lines. No per-session files, ever.

  • UnifiedStore.ARCHIVE_LOG_NAME = "archive.jsonl"; archiveSession appends, restoreArchived scans lines (skips malformed) and rewrites without the restored ids, re-archive appends fresh.
  • New test pins: single log file on disk, one line per session, restore removes exactly that session's line, other sessions' lines untouched.
  • Docs (CONFIGURATION en/zh + yaml seeds) and CHANGELOG wording updated to the single-log format.

Everything else in #3911 is untouched. CI runs the full matrix on this PR.

Sessions are small and numerous — one export file per archived session
would spray tiny files across the archive dir. One archive.jsonl holds
every archived session as one JSON line: append on archive (crash =
at most one truncated line, which the restore reader skips), rewrite
without the restored session's lines on restore. Docs + test updated.
@ranxianglei ranxianglei changed the title storage: GC archive as one append-only JSONL log (follow-up to #3911) [PR] storage: GC archive as one append-only JSONL log (follow-up to #3911) Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-20_archive-jsonl (528676d)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4670-pr.3927.0c85568

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3927.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。检索 open+closed issue/PR 均无其他条目覆盖「archive 单文件 JSONL 日志」这一格式变更;本 PR 是 #3911(已合并)在合并边界处漏掉的后续提交,与 #3911 本身不重复——#3911 已落地的是 per-session 导出文件方案,本 PR 将其替换为单一 append-only archive.jsonl。

收到 [PR] 通知,这是已完成的工作,我不重做:正在拉取分支 2026-10-20_archive-jsonl 核对代码、检查 CI,随后按八问体例给出审查结论。

awork added 2 commits October 20, 2026 15:31
…g format (#3927 review)

The JSONL switch updated CHANGELOG/CONFIGURATION(en)/yaml seeds but left
three spots describing the old <archiveDir>/<sessionId>.json layout:
the archiveSession docblock (now also orphaned above the method by the
ARCHIVE_LOG_NAME insertion), process.yaml desc_zh, and the matching
CONFIGURATION.zh-CN.md row. No behavior change.
…estores (#3927 review)

The single-log design promises a crash mid-append leaves at most one
truncated line which the restore reader skips, but only the happy path
was staked. Negative case now pinned: a partial trailing line does not
block restoring another session, the restore rewrite drops it (safe —
the append precedes the DB sweep, so its rows were never released),
and restoring the crashed session itself honestly reports
export-missing.
@ranxianglei
ranxianglei merged commit 8f98078 into master Oct 20, 2026
39 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.652-master.8f98078 (rolling master tag).

npm install -g billion-context@0.0.652-master.8f98078  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant