Skip to content

Fix sbx upgrade compatibility and runner resilience - #57

Merged
joecklau merged 7 commits into
developfrom
fix/sbx-integration-resilience
Oct 21, 2026
Merged

joecklau merged 7 commits into
developfrom
fix/sbx-integration-resilience

Conversation

@joecklau

Copy link
Copy Markdown
Contributor

Summary

Upgrading Docker Sandboxes from v0.46.0 to v0.49.1 caused ./start to fail on inventory and diagnostics CLI changes introduced in v0.48. Long-running pools could also exit when a host-trust lease write timed out during runner readiness. This PR restores startup compatibility and keeps recoverable candidates within the existing quarantine, exact-cleanup, and capacity rules.

  • Support legacy and current inventory, policy-log, and diagnostic formats, including workspace-free sandboxes and optional diagnostic presentation fields. Preserve strict identity, policy attribution, freshness, and recovery-census checks.
  • Route readiness lease-write timeouts through bounded candidate recovery, while retaining terminal handling for cancellation and permanent trust failures.
  • Verify stop, deletion, template removal, and policy outcomes through fresh structured readbacks instead of command prose. Finish receipt-owned staging cleanup after deferred deletion or controller restart before tombstoning the instance.
  • Require actual apt/dpkg lock absence before guest readiness, including locks held by unfamiliar processes. Preserve the narrow known-bootstrap termination rules, bounded waiting, and recovery marker; ignore zombies that cannot hold locks.

The changes are split across seven focused commits. The guest lock check requires a rebuilt local template or a newly published prebuilt package containing it. Live Windows relay validation remains outstanding; the macOS smoke does not certify foreign-architecture workloads or other host platforms.

Validation

Validation was performed across the commits and repeated for affected behavior after corrections:

  • Full Go suite (go test -timeout 5m ./...), affected image/provider/pool/startup normal and race checks, and wrapper syntax/command-forwarding checks passed.
  • Windows amd64 provider test binary compiled successfully; legacy and current policy-log formats passed the relay attribution, freshness, route, and rule-proof regressions.
  • On macOS arm64 with sbx v0.49.1, default ./start brought two cached Full-template runners online and idle. Console and guest logs, host-trust lease renewal, and exact shutdown cleanup were checked. A foreign workspace-free sandbox remained untouched by EPAR cleanup.
  • A fresh Act local build passed provenance/SBOM generation, exact template import and activation, guest helper checksum verification, runtime admission, GitHub online/idle, and exact shutdown cleanup.
  • A disposable Linux sandbox verified natural release of an unknown holder of the real dpkg frontend lock, a 185-second bounded timeout with the recovery marker while preserving the holder, and readiness with an unreaped apt zombie.
  • Independent reviews completed with no outstanding correctness blockers. Final live status and sandbox inventory were empty after cleanup.

Checklist

  • I kept credentials, private keys, tokens, and machine-specific configuration out of this pull request.
  • I added or updated tests where behavior changed.
  • I updated relevant documentation.
  • For provider or onboarding changes, I followed the Development and Extension Principles and documented and tested every intentional exception.
  • I read and followed the contributing guide and code of conduct.

@joecklau
joecklau deployed to epar-live-ci October 21, 2026 13:08 — with GitHub Actions Active
@joecklau
joecklau merged commit d5eb24e into develop Oct 21, 2026
@joecklau
joecklau deleted the fix/sbx-integration-resilience branch October 21, 2026 14:35

This branch was successfully deployed

1 active deployment
epar-live-ci — ab7a5a3f Deployed Oct 21, 2026 by joecklau
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants