Repository navigation
feat(workflow-recipes): size output PVCs from operator config and record the step change policy - #1452
feat(workflow-recipes): size output PVCs from operator config and record the step change policy#1452alfredolopez80 wants to merge 8 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C3LgMsRXRDCMoHU1pkxRwh
…llowlists - State that the four step flags are proposed: none exists in the code at the cited dev commit. Spillover "would be" on by default and the 32 KiB step threshold is labelled as proposed. Table rows are unchanged. - Replace "the reviewed allowlist" with the three per-pod-type lists in podFactory.ts (mcp-host, coordinator, snippet runner) and say that a flag read by mcp-host must be on MCP_HOST_ENV_PASSTHROUGH. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Went through this one and checked every 🟡 Approve with nits
This is my read from the review, not a formal GH approval. Optional (non-blocking)These can merge as is. Treat them as follow-ups or as your call, though I'd fold the first two into the record before anyone builds spillover from it.
What checked out clean
|
…nge-policy * origin/dev: (80 commits) fix(workflow-recipes): never report a pod that is being deleted as ready test(workflow-recipes): return the PodReadiness kind from readiness mocks fix(e2e): keep every kill attempt's node log and report a watcher that ended early fix(hcc): reach the overrun and call-failure counters only through typed helpers docs(hcc): correct claims about the request timeouts and the hard limit fix(workflow-recipes): fail the pass when patchStatus cannot read the live recipe fix(workflow-recipes): give no readiness verdict while the mcp-host pod is absent test(workflow-runtime-core): cover byte-heavy artifact results through the production callers docs(hcc): say what the pass hard limit does not do fix(e2e): continuity lane retries after NO_PASS_START and arms its watcher without a gap fix(workflow-runtime-core): bound status tool-call previews by serialized bytes fix(mcp-host): accept plain digits only for HCC_AUTHORITY_MAX_STALENESS_MS fix(hcc): expose every pass-overrun and call-failure series at zero fix(hcc): refuse a request timeout that is not below the pass hard limit fix(hcc): accept plain digits or an empty value for the request timeouts and the pass hard limit test(hcc): pin the write-attempt signal of the Service and ConfigMap sync fix(workflow-recipes): start the mcp-host readiness window at the later of run or pod start docs(crds): state that backoffSeconds retry waits are capped at 300 s test(workflow-recipes): probe healthCheck against the real runner server, drop the abort pin test(workflow-recipes): cover resume with a snippet step persisted as running ...
…decision Answer the round-1 review of the step change policy against dev 1b1b113: - step tool-result spillover lives in mcp-host memory for one step execution (#1454), released when the step ends; the four spillover names are already on MCP_HOST_ENV_PASSTHROUGH; - no sanitizer runs on step tool results; only the <tool-result> delimiters wrap them; - name both coordinator parsers (ConfigLoader, sdk.config.getSpec()) in layer 3 and the custom-image hop; - state the snippet worker env filter; - principle 5 lists the preview exception, and a new open decision 8 covers reducing status.steps[].toolsCalled[].args and .result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and forward the quota A WRC-managed output PVC for a recipe without spec.output.storageSize now requests CLERUM_WORKFLOW_OUTPUT_STORAGE_SIZE from the WRC Deployment, or 1Gi when unset (was a hard-coded 256Mi). The value must match ^[1-9][0-9]*(Mi|Gi)$; anything else stops WRC at startup. spec.output. storageSize still wins, and WRC still never resizes an existing claim. deploy/base sets the variable to 1Gi explicitly. CLERUM_WORKFLOW_OUTPUT_QUOTA_MB joins MCP_HOST_ENV_PASSTHROUGH, so a value set on WRC reaches recipe mcp-host pods; unset keeps the mcp-host default and an identical pod spec. Tests: config parsing and the deploy/base default, the WorkflowConfig wiring, PVC size precedence, the existing-PVC 409 skip, and the passthrough forwarding with a witness for the absent case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… operator default The six pvc templates in the recipe editor no longer write spec.output.storageSize, so recipes created from them get the size WRC configures (CLERUM_WORKFLOW_OUTPUT_STORAGE_SIZE, else 1Gi). Saved recipes keep their explicit value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the changed default Update the WorkflowRecipe storageSize description (text only) and the CRD reference: spec.output.storageSize, else CLERUM_WORKFLOW_OUTPUT_STORAGE_SIZE on WRC, else 1Gi, applied when WRC creates the PVC. Document CLERUM_WORKFLOW_OUTPUT_QUOTA_MB and add the upgrade note for the first release after v0.12.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the review. All six points are addressed in the design record; the PR also picks up three small code changes the owners asked for while answering R1-M1.
On sanitizing: no sanitizer runs on step tool results today, inline or read back; only the Code in this PR:
|
#1454 raised the step spillover threshold default to 131072 bytes (stepSpillover.ts, b93220e). The flag table and the rationale now say 128 KiB. Open decision 3 stays open: it is settled by the off / 128 KiB / 32 KiB comparison that #1454 requires before merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Summary
Two parts.
Design record (
docs/design/workflow-step-change-policy.md, status Proposed):ConfigLoaderwithparseRuntimeWorkflowSpec, andsdk.config.getSpec()for custom images);/output; the read tool only sees its own step execution's store;status.steps[].toolsCalled[].argsand.resultto names, sizes and an artifact reference, after moving their current readers. It stays open until feat(mcp-host): spill oversized tool results of workflow steps out of the prompt #1454 lands.Code (requested by the owners while answering review round 1):
spec.output.storageSize, else from the new WRC envCLERUM_WORKFLOW_OUTPUT_STORAGE_SIZE, else1Gi. The env accepts^[1-9][0-9]*(Mi|Gi)$; any other value stops WRC at startup.deploy/basesets it to1Gi.spec.output.storageSizegets a1GiPVC (was256Mi). The size applies only when WRC creates the PVC; WRC never resizes an existing claim.CLERUM_WORKFLOW_OUTPUT_QUOTA_MBis added toMCP_HOST_ENV_PASSTHROUGH, so the artifact quota set on WRC reaches recipe mcp-host pods.storageSize, so new recipes use the operator default.storageSizedescription anddocs/crds/workflowrecipe.mddescribe the precedence; the upgrade note is indocs/deploy/releases-and-upgrades.md.Rollout
crds/: applycharts/clerum-crds/crds/workflowrecipe.yamlon each cluster.deploy/basethat set their own WRC env get the1Gibuilt-in default until they setCLERUM_WORKFLOW_OUTPUT_STORAGE_SIZE. Not verified: the infra repository wiring andcrd-guard.sh.Testing
Head
372ce8363, Node 24.22.1.tsc -bEXIT=0;npm testEXIT=0, 154 files, 3371 passed, 29 skippedtscEXIT=0; tests EXIT=0, 261 files, 3106 passedstorageSizerestored, local templates offbash scripts/minikube/t0.sh:T0_EXIT=0, affected checks PASS (workflow-recipes 3371 passed, control-ui 3106 passed)bash scripts/ci/check-no-infra-identifiers.shChecklist
🤖 Generated with Claude Code