[JasonBuildAI]

Summary

updateSnapshotStatusFromSucceededCommitJob published a rootfs-v1 SandboxSnapshot as Succeed even when the commit result was unavailable. The result is the only place the per-container image digests are written (the commit Pod's termination message), and since #1584 (1ea4e385) resume restores rootfs-v1 containers exclusively by ImageURI@sha256:<digest>. The consequence was silent and permanent: pause reported success, completePause deleted the runtime Pods, and every later resume failed with ResumeFailed/InvalidSnapshotImage: invalid sha256 digest "".

The commit result can be gone while the Job still reports success: a completed-Pod cleanup policy, node drain/eviction, or a controller restart/leader loss that reads the Job after the Pod was removed leaves Job.Status.Succeeded > 0 with no Pod to read. (The Job's own 300s TTL deletes the Job as well, which makes the controller recreate it and re-commit — that path heals itself.) Nothing retried the digest, because it was never persisted.

This change treats a missing or incomplete commit result as a failed commit for rootfs-v1, mirroring what the qemu-v1 branch already does:

• no result (found == false) → Failed / CommitResultUnavailable
• result without a digest for every snapshot container → Failed / CommitResultIncomplete, naming the containers in the message

found == false means the result is gone from the API server, not only from the informer cache. The commit Pod is read through the uncached APIReader whenever the cached lookup comes back empty, because the Job controller writes Job.status.succeeded only after the commit Pod reached a terminal state — the Pod status carrying the termination message is persisted before the Job status is — so the result already exists in the API server when the Job reports success, and only the cache may lag behind it. Without that read, a lagging cache would fail the pause of a healthy sandbox.

syncPauseOrClear then reports PauseFailed on the BatchSandbox, the Pods are not deleted, and the sandbox stays running so pause can be retried. Legacy snapshots (Format == "") still restore by URI and keep their current behavior; an already Succeed snapshot is never downgraded. docs/guides/pause-resume.md now documents when Succeed is published and the two failure reasons.

Fixes #2481.

Testing

New controller tests in kubernetes/internal/controller/sandboxsnapshot_controller_test.go:

• TestSandboxSnapshotHandleCommitting_RootfsCommitWithoutResultFails — rootfs-v1, Succeeded Job, no Pod → Failed / CommitResultUnavailable
• TestSandboxSnapshotHandleCommitting_RootfsCommitWithIncompleteResultFails — termination message covers main but not sidecar → Failed / CommitResultIncomplete
• TestSandboxSnapshotHandleCommitting_RootfsCommitWithAllDigestsSucceeds — positive control; still Succeed with digests recorded
• TestSandboxSnapshotHandleCommitting_StaleCacheReadsResultFromAPIServer — the informer cache returns an empty PodList while the API server still holds the commit Pod → Succeed with the digest, not a terminal Failed

The two rejection tests fail on unmodified 97031ea8 (the production file replaced through go test -overlay, tree untouched) and pass at the PR head:

--- FAIL: TestSandboxSnapshotHandleCommitting_RootfsCommitWithoutResultFails
    expected: "Failed"
    actual  : "Succeed"

--- FAIL: TestSandboxSnapshotHandleCommitting_RootfsCommitWithIncompleteResultFails
    expected: "Failed"
    actual  : "Succeed"

--- PASS: TestSandboxSnapshotHandleCommitting_RootfsCommitWithAllDigestsSucceeds

At the PR head:

--- PASS: TestSandboxSnapshotHandleCommitting_SetsSucceedReadyCondition
--- PASS: TestSandboxSnapshotHandleCommitting_PersistsImageDigestsFromTerminationMessage
--- PASS: TestSandboxSnapshotHandleCommitting_RootfsCommitWithoutResultFails
--- PASS: TestSandboxSnapshotHandleCommitting_RootfsCommitWithIncompleteResultFails
--- PASS: TestSandboxSnapshotHandleCommitting_RootfsCommitWithAllDigestsSucceeds
--- PASS: TestSandboxSnapshotHandleCommitting_StaleCacheReadsResultFromAPIServer
--- PASS: TestSandboxSnapshotHandleCommitting_KeepsRetryingWhenJobHasNotTerminallyFailed
--- PASS: TestSandboxSnapshotHandleCommitting_CreatesUnpauseJobWhenCommitJobFailed

Also run (Windows, Go 1.25.9; the package has pre-existing platform failures):

• go test ./internal/controller/ — failure set identical to the 97031ea8 baseline (14 lines: TestControllers, TestReconcileDeletingPod, TestReconcileDeletingTaskSandbox, TestReconcilePodRecovery, and the QEMU cases of TestSandboxSnapshotHandlePending_ImageURITemplate)
• go test ./internal/snapshot/ — same single pre-existing TestContractFromPodParsesQEMU failure as baseline
• go vet ./internal/controller/ ./cmd/controller/ — clean
• GOOS=linux GOARCH=amd64 go build ./internal/controller/ ./cmd/controller/ — clean
• gofmt -d on the changed Go files — empty; git diff --check — clean
• cd docs && pnpm docs:build — build complete in 13.80s, zero errors

Breaking Changes

• None
• Yes (describe impact and migration path)

Behavior change is limited to the failure path: a pause whose commit result was lost now reports PauseFailed and keeps the Pods instead of reporting success and making resume impossible.

Checklist

• Linked Issue or clearly described motivation
• Added/updated docs (if needed) — docs/guides/pause-resume.md states when Succeed is published and the two failure reasons.
• Added/updated tests (if needed)
• Security impact considered — no new input surface; the change avoids publishing an unusable snapshot.
• Backward compatibility considered — legacy Format == "" snapshots and existing Succeed statuses are untouched.

---

This change was drafted with AI assistance at my request, under my direction and review.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-30T05:19:18.685184Z	48c17a7	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 162e44cf63

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[JasonBuildAI]

Addressed the P1 finding in 48c17a73: the commit Pod is now read through the uncached APIReader when the informer cache has not observed it yet, so a lagging cache can no longer turn a healthy pause into a terminal Failed. The review thread has the reasoning and the new test (TestSandboxSnapshotHandleCommitting_StaleCacheReadsResultFromAPIServer, which fails without this commit and passes with it).

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 48c17a73d9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Pangjiping]

LGTM