Fail closed on hook failures - #1097
shunichironomura wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1097 +/- ##
==========================================
+ Coverage 62.49% 63.31% +0.81%
==========================================
Files 45 44 -1
Lines 4994 5097 +103
==========================================
+ Hits 3121 3227 +106
+ Misses 1873 1870 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Heads-up on the server side:
Suggest adding: pub struct HookMeta {
pub id: String,
pub config: Option<JsonValue>,
pub success: bool,
pub error: Option<String>,
pub failure_reason: Option<String>, // new
}and threading it through |
93a5223 to
085d153
Compare
e36bd14 to
6a82a85
Compare
985d14e to
7d291d9
Compare
0df4332 to
709e249
Compare
709e249 to
0d7ac32
Compare
Warning
We should not merge this PR until #1099 is merged.
Summary
This PR changes hook execution from a "best effort / fail open" model to a "fail closed after collecting diagnostics" model.
Internally, it introduces
HookOutcome<T>so hooks can distinguish:Ok(HookOutcome::Succeeded(output)): the hook ran and its configured success condition passedOk(HookOutcome::Failed { output, reason }): the hook ran and captured useful output, but its configured success condition failedErr(error): the hook could not complete because of an operational errorHook::runnow returnsCapsulaResult<HookOutcome<Self::Output>>, whileCapturedis only responsible for JSON serialization. All in-tree hooks, including the newly addedcapture-jsonandcapture-tomlhooks, use the new outcome API. The server preserves policy-failure reasons through upload, database storage, API responses, and the run-detail UI.AI assistance
Breaking changes
breaking changelabel is appliedUser-facing behavior changes
Pre-run hook failures now stop the wrapped command
Before this PR, hook errors were recorded in
pre-run.jsonassuccess: false, but the wrapped command still ran. That meant safety gates could fail open.After this PR:
pre-run.jsonwith every hook result.This affects both policy failures and operational errors:
capture-git-repo.allow_dirty = falsecapture-git-repo.require_pushed = truePost-run hook failures now make Capsula fail after recording results
Post-run hooks cannot stop the wrapped command because it has already run. They now fail closed at the Capsula layer instead:
post-run.jsonwith every hook result.capsula runexits with the hook-failure exit code when the wrapped command succeeded.capsula runpreserves the wrapped command's non-zero exit code when the wrapped command already failed.capsula run-endexits with the hook-failure exit code after writingpost-run.json.This makes post-run capture/notification failures visible to automation without masking an already-failing wrapped command.
Hook policy failures are now visible in hook metadata
When a hook completes but its configured condition fails, the hook output is still recorded, but
__meta.successis nowfalseand__meta.failure_reasonexplains why.For example, a dirty repo with
allow_dirty = falseis recorded as a failed hook outcome rather than a successful capture with an out-of-band abort flag.Operational hook errors continue to be recorded with
__meta.error.capture-commandnow treats non-zero exit status as failure by defaultBefore this PR,
capture-commandonly requested abort on non-zero status whenabort_on_failure = true.After this PR, the default successful status set is
[0]. A non-zero status is therefore a hook failure unless configured otherwise.To intentionally check for a failing command, configure the expected status explicitly:
For compatibility, existing configs that explicitly set
abort_on_failure = falsestill accept any exit status. New configs should prefersuccess_codes.Notes for hook developers
Hook implementations now need to return
CapsulaResult<HookOutcome<Self::Output>>fromHook::run.Use the three outcome categories intentionally:
Ok(HookOutcome::success(output))when the hook completed and the captured state satisfies the hook's configured policy.Ok(HookOutcome::failure(output, reason))when the hook completed and produced useful output, but the captured state violates the hook's configured policy.capture-git-reposuccessfully captured repo state, butallow_dirty = falseand the repo is dirty.capture-commandsuccessfully captured stdout/stderr/status, but the exit status is not insuccess_codes.Err(error)when the hook itself could not complete the operation it was asked to perform.Capturedno longer hasabort_requested(). Abort/failure policy should be expressed throughHookOutcome::failure(...), not hidden inside captured output.The orchestrator will still attempt every hook in the phase. Hook developers do not need to short-circuit other hooks; they only need to classify their own result correctly.
For hook configuration design, prefer explicit success criteria over ad-hoc abort booleans. For example,
capture-commandnow usessuccess_codesso expected non-zero statuses can be represented directly.Notes for reviewers
capsula rundoes not mask a wrapped command's non-zero exit code with a post-run hook failure code.Tests
just lintjust testCloses #1062