Skip to content

RFC 0013: SQLite snapshot backup artifacts - #20

Closed
giodl73-repo wants to merge 24 commits into
mainfrom
rfc/cloud-serializable-sqlite-state
Closed

giodl73-repo wants to merge 24 commits into
mainfrom
rfc/cloud-serializable-sqlite-state

Conversation

@giodl73-repo

@giodl73-repo giodl73-repo commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

RFC 0013 defines the SQLite-safe artifact boundary now implemented by
openclaw/openclaw#105718:

one live OpenClaw-owned SQLite database -> one verified, immutable snapshot
directory -> fresh-target restore

The landed command is:

openclaw backup sqlite create
openclaw backup sqlite list
openclaw backup sqlite verify
openclaw backup sqlite restore

OpenClaw owns database roles, SQLite-safe capture, manifest and hash creation,
strict verification, private staging, atomic local publication, and
fresh-target restore. Hosts and operators own upload, encryption, retention,
restore timing, startup orchestration, and failover policy.

What We Learned From Running Snapshots In Lobster

We have learned a lot since we first started running the snapshot boundary in
Lobster:

  • syncing completed manifest.json plus database.sqlite artifacts is the
    right boundary; syncing live SQLite files and sidecars is not;
  • the SQLite owner must author and verify the artifact because a host cannot
    infer database role, WAL state, schema, indexes, ownership, or safe restore
    semantics from paths;
  • a verified local snapshot is intentionally smaller than a complete
    replacement or scale-from-zero transaction;
  • portable recovery still needs explicit treatment for non-SQLite state,
    credentials and external dependencies, durable host acceptance, final
    quiescence, generation fencing, restored readiness, and coordinator replay;
    and
  • those lifecycle concerns should compose above this RFC rather than expanding
    or duplicating the landed SQLite implementation.

OpenClaw main also already has the host-neutral
gateway.suspend.prepare|status|resume contract from
openclaw/openclaw#103618,
with Vincent's validation repair in
openclaw/openclaw#103925.
Follow-on handoff work should reuse that cooperative tracked-work fence rather
than propose another pause API. The host still owns external ingress fencing,
third-party Channel handling, and full process/filesystem consistency.

The result is a simpler follow-on direction: keep RFC 0013 and PR 105718 as the
authoritative owner primitive, then discuss small optional sidecars for
aggregate recovery-point components, portable handoff, and restored admission.
Lobster remains one proving host, not part of the core snapshot contract.

Scope

This RFC remains deliberately narrow:

  • shared OpenClaw state and per-agent SQLite databases;
  • verified local snapshot repositories;
  • committed WAL-state capture through SQLite;
  • strict manifests, hashes, schema/role/owner verification;
  • safe cross-platform staging and publication; and
  • restore only to a fresh target.

It does not define:

  • scheduling, upload, object storage, or retention;
  • credential transfer;
  • aggregate OpenClaw state capture;
  • restore-on-boot;
  • leases, generation fencing, or safeToDestroy;
  • retained ingress, hibernation, wake, or managed failover.

Implementation

Implementation landed in
openclaw/openclaw#105718,
commit
2f25e9cba384acfc06cdf83640f236fdb7c1af33.

The original contributor implementation in
openclaw/openclaw#94805 was
superseded by Vincent's hardened implementation while preserving the
SQLite-safe artifact direction.

Current-Head Validation

  • RFC head:
    864d7dee6ba6b9cf9e2f902f174408e5fbc570db
  • One RFC file, rfcs/0013-cloud-serializable-sqlite-state.md
  • Frontmatter records status: completed and the landed implementation.
  • The RFC matches the final backup sqlite command and responsibility split.
  • PR 105718 records Linux, Windows, and macOS proof for the landed behavior.
  • RFC checks currently pass.

Review Request

Please review RFC 0013 as the completed SQLite owner-artifact contract.

Separately, we would value feedback on whether its completed snapshot
directories and exact manifests are the right substrate for small,
host-neutral follow-on sidecars. Those sidecars would not change this RFC's
snapshot semantics, add Lobster-specific APIs, or duplicate the existing
gateway.suspend.* contract.

@clawsweeper

clawsweeper Bot commented Jun 18, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 21, 2026, 4:55 PM ET / 20:55 UTC.

ClawSweeper review

What this changes

This PR adds a completed RFC documenting OpenClaw’s SQLite-safe snapshot artifact and fresh-target restore contract.

Merge readiness

Blocked by patch quality or review findings - 6 items remain

This RFC is not on current main and remains useful, but it is not merge-ready: its completed status points issue at an implementation pull request rather than the required accepted implementation issue.

Priority: P3
Reviewed head: 864d7dee6ba6b9cf9e2f902f174408e5fbc570db

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The contract text is substantial, but the missing required acceptance record leaves the documentation PR not quality-ready to merge.
Proof confidence 🌊 off-meta tidepool Not applicable: This is a documentation-only RFC change; runtime proof belongs to the separately referenced core implementation.
Patch quality 🦪 silver shellfish (2/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This is a documentation-only RFC change; runtime proof belongs to the separately referenced core implementation.
Evidence reviewed 4 items RFC metadata mismatch: The proposed frontmatter marks the RFC completed but records an OpenClaw pull request in its issue field.
Published RFC lifecycle: The repository requires an accepted RFC to record a GitHub implementation issue before it can later be marked completed.
Current-main check: Current main does not contain this RFC file, so the documentation change has not already landed in this repository.
Findings 1 actionable finding [P1] Link the accepted implementation issue
Security None None.

How this fits together

The RFC repository records OpenClaw design contracts. This document defines how a live OpenClaw-owned SQLite database becomes a verified snapshot directory for host retention and later fresh-target restore.

flowchart LR
  A[Live SQLite database] --> B[SQLite-safe capture]
  B --> C[Manifest and hash verification]
  C --> D[Completed snapshot directory]
  D --> E[Host retention or transfer]
  D --> F[Fresh-target restore]
  F --> G[OpenClaw runtime]
Loading

Before merge

  • Add real behavior proof - Not applicable: This is a documentation-only RFC change; runtime proof belongs to the separately referenced core implementation.
  • Link the accepted implementation issue (P1) - The RFC lifecycle requires an accepted RFC to record a GitHub implementation issue before later becoming completed. This line instead points issue at an implementation pull request, leaving the required acceptance record absent; link the accepted issue and preserve the discussion trace before merging.
  • Resolve merge risk (P1) - Merging a completed RFC whose required implementation issue is not recorded would bypass the repository’s published acceptance trace and make the contract’s approval history ambiguous.
  • Complete next step (P2) - The remaining correction requires the contributor or RFC owners to provide the repository-required acceptance record; it is not an autonomous repair task.
  • Improve patch quality - Record the accepted implementation issue in frontmatter.
  • Improve patch quality - Ensure the required maintainer-discussion acceptance record is traceable.

Findings

  • [P1] Link the accepted implementation issue — rfcs/0013-cloud-serializable-sqlite-state.md:8
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
RFC scope 1 RFC file; 290 added, 0 removed The entire proposed change is one new completed contract, so lifecycle metadata is the key merge gate.

Merge-risk options

Maintainer options:

  1. Restore the RFC acceptance record (recommended)
    Link the accepted GitHub implementation issue in the frontmatter and ensure the required maintainer-discussion acceptance record is available before merge.

Technical review

Best possible solution:

Keep the narrow completed contract, but merge it only after its frontmatter records the accepted implementation issue and the required acceptance discussion is traceable.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR changes RFC documentation rather than executable behavior; the described runtime implementation is outside this repository.

Is this the best way to solve the issue?

No: the completed status and implementation pull-request link do not satisfy this repository’s required accepted implementation-issue record.

Full review comments:

  • [P1] Link the accepted implementation issue — rfcs/0013-cloud-serializable-sqlite-state.md:8
    The RFC lifecycle requires an accepted RFC to record a GitHub implementation issue before later becoming completed. This line instead points issue at an implementation pull request, leaving the required acceptance record absent; link the accepted issue and preserve the discussion trace before merging.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against af708f0ddb6f.

Labels

Label justifications:

  • P3: This is documentation and process cleanup, not a production incident.
  • merge-risk: 🚨 other: The completed metadata would create an ambiguous approval and implementation trail if merged without the required issue record.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🌊 off-meta tidepool and patch quality is 🦪 silver shellfish.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: This is a documentation-only RFC change; runtime proof belongs to the separately referenced core implementation.

Evidence

What I checked:

Likely related people:

  • kevinlin-openai: Authored the current RFC-process guidance that defines the required implementation-issue record. (role: RFC lifecycle author; confidence: high; commits: e366ea9825a4; files: README.md, rfcs/0000-template.md)
  • Dallin Romney: Recent history shows work clarifying RFC layout and supporting-material conventions adjacent to this document. (role: recent RFC structure contributor; confidence: medium; commits: 3aa7d727383f; files: README.md)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (46 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-09T12:42:08.718Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Restore the required RFC acceptance record
  • reviewed 2026-08-09T14:50:52.171Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Record the accepted implementation issue
  • reviewed 2026-08-09T19:49:33.805Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Record the accepted implementation issue
  • reviewed 2026-08-09T22:00:38.165Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Record the accepted implementation issue
  • reviewed 2026-08-09T23:13:46.602Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Record the required accepted implementation issue
  • reviewed 2026-08-11T03:13:24.429Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Link the required implementation issue
  • reviewed 2026-08-12T19:14:17.581Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Record the accepted implementation issue
  • reviewed 2026-08-15T04:02:21.810Z sha 864d7de :: needs real behavior proof before merge. :: [P1] Record the accepted implementation issue

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Jun 18, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 18, 2026
@giodl73-repo giodl73-repo changed the title RFC 0013: Cloud-Serializable SQLite State RFC 0013: SQLite State Snapshot Plugin Jun 18, 2026
@clawsweeper clawsweeper Bot added the feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. label Jun 18, 2026
@giodl73-repo

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Updated the PR body with final-head RFC proof, including head SHA, diff stat, frontmatter, line count, and git diff --check output. The maintainer-discussion thread remains pending maintainer-side Discord action/access, and acceptance metadata is intentionally unchanged while draft.

@clawsweeper

clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 19, 2026
@giodl73-repo
giodl73-repo marked this pull request as ready for review June 19, 2026 01:52
@clawsweeper clawsweeper Bot removed the proof: sufficient Contributor real behavior proof is sufficient. label Jun 19, 2026
@giodl73-repo giodl73-repo changed the title RFC 0013: SQLite State Snapshot Command RFC 0013: SQLite snapshot backup artifacts Jul 7, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 7, 2026
@giodl73-repo

Copy link
Copy Markdown
Contributor Author

Hey @vincentkoc — we have learned a lot since we first started running the
snapshot boundary in Lobster.

Your #105718 implementation gave us the native owner seam we were missing. The
main lesson from the broader continuity proof is that we should not build a
parallel snapshot/archive system. We can keep RFC 0013 and backup sqlite
authoritative, then make any follow-on much smaller: verified owner snapshots
plus explicit non-SQLite components and external obligations, followed by
host-owned acceptance, fenced restore, and restored readiness.

We also confirmed that current main already has the host-neutral
gateway.suspend.prepare|status|resume seam from #103618, including your
#103925 validation repair. Any follow-on handoff should compose that shipped
cooperative fence with backup sqlite, not invent another pause or drain API.

The real process/container proof also clarified where snapshot semantics should
end:

  • local snapshot success is not durable host acceptance or safeToDestroy;
  • portable state needs explicit credential and dependency treatment;
  • final capture should reuse gateway.suspend.* while the host separately
    fences external ingress and untracked work, rather than infer safety from
    process exit;
  • restore success is not readiness or admission; and
  • coordinator replay must reuse the same child/generation without repeating
    preparation or restore.

We have updated this PR body to match the landed implementation and current RFC
head. Before authoring any follow-on sidecars, we would like your read on the
ownership split: does composing completed snapshot directories and exact
manifests above RFC 0013 feel like the right direction, while keeping upload,
retention, lifecycle, and wake host-owned?

@giodl73-repo

Copy link
Copy Markdown
Contributor Author

Follow-on sidecars are now isolated in draft PR #46 rather than expanding this
completed RFC directly:

This keeps PR #20 narrow and lets the lifecycle contracts receive independent
review before any implementation work.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 29, 2026
@clawsweeper clawsweeper Bot removed the merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. label Aug 6, 2026
@steipete

Copy link
Copy Markdown
Contributor

Closing as shipped — thank you for driving this boundary to a clean contract.

The capability this RFC specifies is fully implemented and canonical in the product: openclaw backup sqlite create/list/verify/restore landed via openclaw/openclaw#105718 and is documented at https://docs.openclaw.ai/cli/backup and https://docs.openclaw.ai/install/backups. Verified live against current main (d7c57716841): all four subcommands are present with the one-live-DB → verified immutable snapshot → fresh-target restore semantics this RFC describes.

We're keeping the RFC repository as a decision-record surface rather than maintaining parallel spec text for already-shipped behavior — the product docs and code are canonical for shipped surfaces, so an open spec PR for them closes rather than merges. The still-open design work this thread spawned continues in #46 (application-consistent recovery for scale-to-zero), which is unaffected by this close.

What would reopen this: a concrete divergence between the shipped backup sqlite behavior and a contract consumers depend on — that would come back as an openclaw/openclaw issue against the shipped surface rather than RFC text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants