Skip to content

docs: design bounded hosted live smoke tier - #1195

Open
steipete wants to merge 1 commit into
mainfrom
codex/944-live-tier
Open

docs: design bounded hosted live smoke tier#1195
steipete wants to merge 1 commit into
mainfrom
codex/944-live-tier

Conversation

@steipete

Copy link
Copy Markdown
Contributor

Summary

  • propose a bounded tier-two hosted live-smoke design with a literal first-phase allowlist of DigitalOcean and Scaleway, selected for their small fixed instance shapes, dedicated lifecycle runners, provider-native ownership metadata, and verified cleanup surfaces
  • define the live-smoke protected-environment boundary, schedule/manual-only triggers, main-only checkout guards, least-privilege provisioning and sweeper credentials, rotation policy, and the required-reviewer tradeoff for scheduled reconciliation
  • set explicit USD 0.25 per-provider, USD 0.50 per-scheduled-run, and USD 20 monthly ceilings, enforced through dedicated provider projects, one-resource quotas, hard project credit limits, fixed shapes, TTLs, sequential execution, and workflow timeouts
  • specify targeted always-run cleanup plus an independent 15-minute provider-native sweeper, cancellation/failure classifications, sanitized evidence, and an exact maintainer activation checklist
  • cross-link the proposal from the existing provider live-smoke guide; this PR adds no active workflow and no secrets

This is the design-only tier-two follow-up for #944. It builds on the hermetic lifecycle gate from #948, the secretless hosted end-to-end tier from #981, and the provider-specific live-test documentation work in #979.

Verification

$ scripts/check-docs.sh
checked 56 command docs: command surface ok
checked provider matrix: 79 providers
validated publishable Crabbox Agent Skill: 488 lines, canonical and projection identical
checked 238 markdown files: internal links ok
validated registry-ready Crabbox Zed extension: 10 tasks, 3 snippets
executed 10 Zed tasks through the Crabbox CLI boundary
built docs site: dist/docs-site
tests 14; pass 14; fail 0
normalized navigation counts across 233 pages; 79 providers

$ node scripts/check-docs-links.mjs
checked 238 markdown files: internal links ok

$ node scripts/build-docs-site.mjs
built docs site: dist/docs-site

$ git diff --check origin/main..HEAD
# clean

$ autoreview --mode local --stream-engine-output
autoreview clean: no accepted/actionable findings reported
overall: patch is correct (0.99)

No live provider proof applies: this is a documentation-only proposal, deliberately adds no active workflow, and performs no provider mutation. The generated dist/docs-site artifact was removed after validation.

Security and activation

No repository or environment secret is added or referenced by executable code. The proposal keeps all future funded credentials inside the live-smoke environment and blocks activation until dedicated provider projects, hard caps, separately scoped sweepers, destructive canary proof, shape tests, and the maintainer checklist are complete.

No changelog entry: documentation-only design proposal, as requested.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Jul 28, 2026
@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed August 2, 2026, 3:46 PM ET / 19:46 UTC.

ClawSweeper review

What this changes

This PR adds a design document for a bounded hosted live-smoke tier and links it from the existing provider live-smoke guide.

Merge readiness

⚠️ Ready for maintainer review - 5 items remain

Keep this PR open. The design is thoughtfully bounded and adds no executable secret or provider change, but it still embeds repository-specific history, a hard-coded trusted-repository guard, and repository-specific ownership metadata that conflict with Crabbox’s generic documentation policy. A maintainer must also decide whether the proposed paid hosted-smoke policy should become endorsed repository direction.

Priority: P3
Reviewed head: 5e78d383276c34bb50aefe942411ff7b5ffac73f
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The proposal is detailed and safety-conscious, but unresolved policy fit and three small documentation corrections limit merge readiness.
Proof confidence 🌊 off-meta tidepool Not applicable: Real behavior proof is not required because this PR only changes files under docs/.
Patch quality 🦐 gold shrimp (3/6) 3 actionable review findings remain.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: Real behavior proof is not required because this PR only changes files under docs/.
Evidence reviewed 5 items Current hosted-tier boundary: Current main states that the hosted live matrix is not enabled and requires a separate repository policy covering trusted environments, credentials, spend, cancellation, and orphan auditing.
Repository documentation policy: The repository policy requires new documentation to avoid OpenClaw or other project-specific workflows unless it is legacy compatibility or release history; this proposed feature guide is neither.
Repository-specific content remains: The proposed guide names prior repository issue and PR history, hard-codes openclaw/crabbox in its workflow guard, and stores that identifier in resource ownership metadata.
Findings 3 actionable findings [P3] Remove repository-specific workflow history
[P3] Use a neutral trusted-repository guard
[P3] Make ownership metadata portable
Security None None.

How this fits together

Crabbox’s provider live-smoke documentation distinguishes hermetic checks from opt-in provider validation. This proposed tier would later govern protected scheduled or manually dispatched cloud lifecycle checks, including resource cleanup and evidence reporting.

flowchart LR
A[Main branch or manual dispatch] --> B[Protected live-smoke environment]
B --> C[Fixed provider allowlist]
C --> D[Create, verify, and destroy resource]
D --> E[Sanitized lifecycle evidence]
F[Independent provider sweeper] --> G[Orphan cleanup result]
Loading

Decision needed

Question Recommendation
Should Crabbox adopt a funded, scheduled hosted live-smoke tier as repository policy, including the proposed initial provider allowlist, spend ceilings, and independent sweeper model? Defer policy endorsement: Keep the hosted matrix disabled and resolve the proposed provider, budget, and credential policy in #944 before merging an endorsed design.

Why: The diff is documentation-only, but it specifies a future operational and credential policy that current main deliberately leaves to a separate repository decision; source review cannot determine whether maintainers want to endorse that policy.

Before merge

  • Remove repository-specific workflow history (P3) - Replace the opening chronology of this repository’s issue and pull requests with neutral context about the existing validation tiers. The repository policy prohibits project-specific workflow references in new general documentation; this remains unchanged from the prior review of the same head.
  • Use a neutral trusted-repository guard (P3) - Do not prescribe openclaw/crabbox as the trust boundary in a generic feature guide. Describe a configured trusted repository/default branch guard so adopters can apply the design without copying this repository’s identity; this remains unchanged from the prior review of the same head.
  • Make ownership metadata portable (P3) - Replace the literal repository=openclaw/crabbox ownership field with a neutral trusted-repository identifier. Provider-native cleanup can retain the same safety properties without embedding this repository’s identity; this remains unchanged from the prior review of the same head.
  • Resolve merge risk (P1) - Merging detailed provider, budget, credential, and repository-specific activation rules could be read as an approved operational policy even though current main says hosted live smoke still needs a separate repository-policy decision.

Findings

  • [P3] Remove repository-specific workflow history — docs/features/hosted-live-smoke.md:6-12
  • [P3] Use a neutral trusted-repository guard — docs/features/hosted-live-smoke.md:65-66
  • [P3] Make ownership metadata portable — docs/features/hosted-live-smoke.md:134-142
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Documentation surface 2 files changed; 295 added, 4 removed The patch is documentation-only, but the added design would define a long-lived operational contract for future hosted-provider automation.

Root-cause cluster

Relationship: partial_overlap
Canonical: #944
Summary: This PR proposes one documentation-and-policy slice of the broader open lifecycle-testing and hosted-tier work.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Resolve policy before merging (recommended)
    Keep the proposal as a discussion artifact until a maintainer accepts or narrows the funded hosted-smoke direction in the canonical umbrella issue.
  2. Accept the operational policy
    If maintainers approve the two-provider, budgeted, protected-environment model, revise the document to make all repository identity and ownership examples portable before merge.

Technical review

Best possible solution:

Keep the existing hosted-tier boundary until a maintainer sponsors the paid hosted-smoke direction in #944, then merge a provider-neutral design that uses configurable trusted-repository and ownership-identity placeholders.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this documentation-only PR does not activate a hosted workflow, provision a provider resource, or change current runtime behavior.

Is this the best way to solve the issue?

No. The document should first use neutral, configurable trusted-repository and ownership examples, and maintainers must decide whether its paid hosted-smoke policy is a direction Crabbox should endorse.

Full review comments:

  • [P3] Remove repository-specific workflow history — docs/features/hosted-live-smoke.md:6-12
    Replace the opening chronology of this repository’s issue and pull requests with neutral context about the existing validation tiers. The repository policy prohibits project-specific workflow references in new general documentation; this remains unchanged from the prior review of the same head.
    Confidence: 0.99
  • [P3] Use a neutral trusted-repository guard — docs/features/hosted-live-smoke.md:65-66
    Do not prescribe openclaw/crabbox as the trust boundary in a generic feature guide. Describe a configured trusted repository/default branch guard so adopters can apply the design without copying this repository’s identity; this remains unchanged from the prior review of the same head.
    Confidence: 0.99
  • [P3] Make ownership metadata portable — docs/features/hosted-live-smoke.md:134-142
    Replace the literal repository=openclaw/crabbox ownership field with a neutral trusted-repository identifier. Provider-native cleanup can retain the same safety properties without embedding this repository’s identity; this remains unchanged from the prior review of the same head.
    Confidence: 0.98

Overall correctness: patch is correct
Overall confidence: 0.96

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 0171ae2ee19f.

Labels

Label justifications:

  • P3: This is a low-risk documentation and product-direction proposal with no active workflow or runtime behavior change.
  • merge-risk: 🚨 other: The proposed documentation could prematurely establish a repository-specific paid-operation policy that current main intentionally leaves undecided.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Real behavior proof is not required because this PR only changes files under docs/.

Evidence

What I checked:

  • Current hosted-tier boundary: Current main states that the hosted live matrix is not enabled and requires a separate repository policy covering trusted environments, credentials, spend, cancellation, and orphan auditing. (docs/features/provider-live-smoke.md:31, 0171ae2ee19f)
  • Repository documentation policy: The repository policy requires new documentation to avoid OpenClaw or other project-specific workflows unless it is legacy compatibility or release history; this proposed feature guide is neither. (AGENTS.md:9, 797f20736844)
  • Repository-specific content remains: The proposed guide names prior repository issue and PR history, hard-codes openclaw/crabbox in its workflow guard, and stores that identifier in resource ownership metadata. (docs/features/hosted-live-smoke.md:6, 5e78d383276c)
  • Feature-history provenance: The current guide’s history includes the hermetic lifecycle gate commit, which is a relevant existing owner signal for the validation-tier boundary. (docs/features/provider-live-smoke.md:1, 5b39acb1d8d0)
  • Re-review continuity: The current PR head is the same SHA reviewed previously, so the three previously raised documentation-policy concerns remain unresolved rather than being newly introduced. (5e78d383276c)

Likely related people:

  • Peter Steinberger: Git blame attributes the generic documentation policy to Peter Steinberger, and the provider live-smoke guide history includes the merged hermetic lifecycle-gate work. (role: repository-policy author and lifecycle-guide contributor; confidence: high; commits: 797f20736844, 5b39acb1d8d0; files: AGENTS.md, docs/features/provider-live-smoke.md)
  • Vincent Koc: The provider live-smoke guide history identifies Vincent Koc’s documentation commit defining the live-smoke contract, which is the closest established documentation surface. (role: historical live-smoke documentation contributor; confidence: medium; commits: 47736cf94dfa; files: docs/features/provider-live-smoke.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (18 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-01T18:45:28.841Z sha 5e78d38 :: needs changes before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral repository guard placeholder
  • reviewed 2026-08-02T01:42:08.898Z sha 5e78d38 :: found issues before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral repository guard placeholder
  • reviewed 2026-08-02T05:08:30.067Z sha 5e78d38 :: found issues before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral trusted-repository guard placeholder
  • reviewed 2026-08-02T08:53:43.375Z sha 5e78d38 :: found issues before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral trusted-repository guard placeholder
  • reviewed 2026-08-02T11:54:38.048Z sha 5e78d38 :: needs changes before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral trusted-repository guard
  • reviewed 2026-08-02T13:56:12.528Z sha 5e78d38 :: needs changes before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral trusted-repository guard
  • reviewed 2026-08-02T17:34:59.346Z sha 5e78d38 :: found issues before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral trusted-repository guard | [P3] Make the resource ownership tag portable
  • reviewed 2026-08-02T18:47:05.509Z sha 5e78d38 :: found issues before merge. :: [P3] Remove repository-specific workflow history | [P3] Use a neutral trusted-repository guard | [P3] Make ownership metadata portable

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant