diff --git a/CHANGELOG.md b/CHANGELOG.md index 50c87de3..0635b431 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- Added a non-secret Clearfolio capability-readiness record at server startup so + operators can distinguish configured provider, explicit development mock, and + unavailable/invalid configuration without coupling optional document-viewer + readiness to whole-process `/api/health` liveness. - Added deterministic PM analysis for requirements/RFI/RFP readiness, WBS estimation coverage, dependency risk, and procurement package section checks. - Preserved PM-analysis research papers, NASA WBS handbook, BCP 14, and JSON diff --git a/docs/deploy.md b/docs/deploy.md index 690ee4f3..74f653d4 100644 --- a/docs/deploy.md +++ b/docs/deploy.md @@ -57,6 +57,28 @@ ScopeWeave planning capabilities remain available. For local integration work, permits HTTP only for `localhost`, `127.0.0.1`, or `::1`; remote HTTP endpoints are rejected. +At process startup ScopeWeave emits one structured, non-secret readiness record: + +```json +{"event":"capability.readiness","capability":"clearfolio","ready":false,"mode":"unavailable","reason":"clearfolio_not_configured","action":"Set CLEARFOLIO_URL and CLEARFOLIO_HMAC_SECRET, or use SCOPEWEAVE_DEV=1 only for local development."} +``` + +Use `ready`, `mode`, `reason`, and `action` to decide the next operator step. The +record validates local provider and artifact-origin configuration only; it does +**not** make a DNS or HTTP call and therefore does not claim that Clearfolio is +reachable. `mode=development_mock` is deliberately distinct from +`mode=provider`. Invalid provider transport, weak HMAC configuration, and an +invalid artifact-origin allowlist report `ready=false` with a stable reason and +a safe remediation instruction. + +`GET /api/health` remains liveness-only and returns `{"ok":true}` even when the +optional Clearfolio capability is unavailable. This separation prevents an +optional document-viewer dependency from causing the planner process to be +restarted or removed from service. Kubernetes documents liveness as the signal +for restarting unhealthy containers and readiness as the signal for whether a +container should receive traffic; ScopeWeave keeps the whole application live +while reporting the optional capability independently. + Provider URLs are treated as service origins, not arbitrary request prefixes. Keep credentials in the dedicated HMAC secret setting rather than URL userinfo, and do not configure a path, query string, or fragment. The adapter constructs @@ -184,4 +206,6 @@ stays focused on the container + compose path.) ## Health `GET /api/health` → `{"ok":true}`. Wired as the container `HEALTHCHECK` and the -compose healthcheck. +compose healthcheck. Optional dependency readiness is reported separately in the +startup `capability.readiness` record so Clearfolio configuration never makes +whole-process liveness fail. diff --git a/docs/doctoring/clearfolio-capability-readiness.md b/docs/doctoring/clearfolio-capability-readiness.md new file mode 100644 index 00000000..9253ddf7 --- /dev/null +++ b/docs/doctoring/clearfolio-capability-readiness.md @@ -0,0 +1,83 @@ +# Clearfolio capability readiness and liveness separation + +## Decision + +Clearfolio is an optional ScopeWeave MSA capability. Its local configuration state must be visible to an operator without turning the whole planner process unhealthy and without making a provider network request merely to answer a health question. + +ScopeWeave therefore keeps `GET /api/health` as whole-process liveness and publishes one non-secret structured `capability.readiness` record for Clearfolio at server startup. The readiness record is produced by the same configuration validator used by production Clearfolio operations and returns only four bounded fields: `ready`, `mode`, `reason`, and `action`. + +This is a bounded follow-up slice of issue #489. It does not claim remote Clearfolio reachability, latency, authentication success, artifact availability, or end-to-end readiness. Those require operational evidence from real provider calls and the attachment status path; the startup record proves configuration readiness only. + +## States + +### Provider + +A valid root Clearfolio origin, HMAC secret, and optional artifact-origin policy returns: + +```json +{"ready":true,"mode":"provider","reason":null,"action":null} +``` + +No DNS lookup or HTTP request occurs while deriving this state. + +### Explicit development mock + +`SCOPEWEAVE_DEV=1` with no provider URL returns: + +```json +{"ready":true,"mode":"development_mock","reason":null,"action":"Configure a Clearfolio provider before using this deployment for production document conversion."} +``` + +The mode name deliberately prevents the mock from being presented as production-provider readiness. + +### Unavailable or invalid production configuration + +Missing or invalid production configuration returns `ready=false`, `mode=unavailable`, a stable configuration reason, and an action that tells the operator what to change without echoing a URL, shared secret, provider body, network address, or tenant claim. + +Examples include: + +- `clearfolio_not_configured` -> configure the provider URL and HMAC secret, or use the development flag only for local work; +- `clearfolio_hmac_secret_invalid` -> provide at least 32 non-whitespace characters; +- URL component or transport failures -> use a root HTTPS origin without credentials, path, query, or fragment; +- `clearfolio_artifact_origins_invalid` -> provide only comma-separated HTTPS origins or remove the optional setting. + +## Why liveness remains independent + +Kubernetes distinguishes liveness from readiness: a failed liveness probe can trigger container restart, while readiness controls whether a workload should receive service traffic. Clearfolio is not required for planning, authentication, project CRUD, or the static client, so treating its configuration as whole-process liveness would turn an optional dependency failure into an unnecessary planner outage. + +The existing `/api/health` response remains `{"ok":true}` while the Clearfolio capability is unavailable. Operators inspect the startup readiness record for the optional integration and continue to use attachment failure/status evidence for remote operational diagnosis. + +RFC 9110 defines a successful GET response as a representation of the target resource state. ScopeWeave keeps the `/api/health` resource narrowly defined as process liveness rather than silently changing its semantics to aggregate every optional dependency. + +## Security and privacy boundary + +The readiness function calls only local configuration validators. It never: + +- calls `fetch`, resolves DNS, follows redirects, or contacts Clearfolio; +- includes `CLEARFOLIO_HMAC_SECRET`, tenant claims, provider response text, job IDs, artifact tokens, or configured URLs in output; +- changes a capability from unavailable to a successful mock outside explicit development mode; +- weakens the provider URL or artifact-origin allowlist checks established by the parent stack. + +Unknown non-configuration exceptions are rethrown instead of being silently misclassified as a configuration state. + +## Verification contract + +`tests/unit/clearfolio-capability-readiness.test.mjs` launches fresh processes so module-import configuration cannot leak between cases. It replaces global `fetch` with a throwing function and proves that readiness evaluation performs no provider transport. The cases cover: + +- unconfigured production with live `/api/health` and unavailable Clearfolio; +- explicit development mock with a production-configuration action; +- valid production provider configuration; +- insecure production HTTP configuration; +- malformed artifact-origin policy detected before provider transport. + +The regression executes in both `test:unit` and `test:coverage:cases`; `server/clearfolio.mjs` remains in the canonical owned-production c8 target set. + +## Rollback + +Rollback removes the startup capability record and exported readiness function together. It must not restore implicit production mocks or make `/api/health` fail because Clearfolio is optional. If operators require a remote dependency probe later, add it as a separately named operational signal with bounded timeout and explicit failure semantics rather than expanding liveness implicitly. + +## References + +Fielding, R., Nottingham, M., & Reschke, J. (2022). *HTTP semantics* (RFC 9110; STD 97). Internet Engineering Task Force. https://doi.org/10.17487/RFC9110 + +The Kubernetes Authors. (2026). *Liveness, readiness, and startup probes*. Kubernetes Documentation. https://kubernetes.io/docs/concepts/workloads/pods/probes/ diff --git a/package.json b/package.json index b1fb5bfd..80bbedc1 100644 --- a/package.json +++ b/package.json @@ -13,9 +13,9 @@ "coverage": "npm run test:coverage", "server": "node server/server.mjs", "test:api": "node tests/api/auth-secret.test.mjs && node tests/api/smoke.mjs && node tests/api/ratelimit.test.mjs && node tests/api/attachment-status.test.mjs && node tests/api/session-revocation.test.mjs && node tests/api/orchestrator-attribution.test.mjs", - "test:unit": "node tests/unit/opencode-config.test.mjs && node tests/unit/changelog-release-notes.test.mjs && node tests/unit/analytics.test.mjs && node tests/unit/cpm.test.mjs && node tests/unit/baseline-compare.test.mjs && node tests/unit/workload.test.mjs && node tests/unit/cost-evm.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/dep-types.test.mjs && node tests/unit/weekly-report.test.mjs && node tests/unit/clearfolio.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/clearfolio-provider-boundary.test.mjs && node tests/unit/clearfolio-artifact-origin.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/sprint-stats.test.mjs && node tests/unit/burndown.test.mjs && node tests/unit/pm-analysis.test.mjs && node tests/unit/cloud-sync-security.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/coverage-script-contract.test.mjs && node tests/unit/toast-accessibility.test.mjs", + "test:unit": "node tests/unit/opencode-config.test.mjs && node tests/unit/changelog-release-notes.test.mjs && node tests/unit/analytics.test.mjs && node tests/unit/cpm.test.mjs && node tests/unit/baseline-compare.test.mjs && node tests/unit/workload.test.mjs && node tests/unit/cost-evm.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && node tests/unit/dep-types.test.mjs && node tests/unit/weekly-report.test.mjs && node tests/unit/clearfolio.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/clearfolio-provider-boundary.test.mjs && node tests/unit/clearfolio-artifact-origin.test.mjs && node tests/unit/clearfolio-capability-readiness.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/sprint-stats.test.mjs && node tests/unit/burndown.test.mjs && node tests/unit/pm-analysis.test.mjs && node tests/unit/cloud-sync-security.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/coverage-script-contract.test.mjs && node tests/unit/toast-accessibility.test.mjs", "test:coverage": "c8 --all --include=app.js --include=cloud-sync.js --include=scripts/ci/static_coverage_evidence.mjs --include=server/attachment_status.mjs --include=server/app.mjs --include=server/auth.mjs --include=server/clearfolio.mjs --include=server/orchestrator.mjs --reporter=json --reporter=json-summary npm run test:coverage:cases", - "test:coverage:cases": "node tests/unit/coverage-script-contract.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/clearfolio-provider-boundary.test.mjs && node tests/unit/clearfolio-artifact-origin.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && npm run test:api", + "test:coverage:cases": "node tests/unit/coverage-script-contract.test.mjs && node tests/unit/attachment-status.test.mjs && node tests/unit/clearfolio-status-signal.test.mjs && node tests/unit/clearfolio-adapter-mock-hmac.test.mjs && node tests/unit/clearfolio-provider-boundary.test.mjs && node tests/unit/clearfolio-artifact-origin.test.mjs && node tests/unit/clearfolio-capability-readiness.test.mjs && node tests/unit/orchestrator.test.mjs && node tests/unit/orchestrator-coverage.test.mjs && node tests/unit/orchestrator-attribution.test.mjs && node tests/unit/msproject.test.mjs && node tests/unit/auth-password.test.mjs && node tests/unit/editor-unsaved.test.mjs && node tests/unit/static-coverage-evidence.test.mjs && npm run test:api", "test:e2e": "playwright test", "test:e2e:headed": "playwright test --headed", "test:e2e:cloud": "playwright install chromium && playwright test tests/e2e/cloud.spec.js tests/e2e/toast-accessibility.spec.js", diff --git a/server/clearfolio.mjs b/server/clearfolio.mjs index f7e3b5cb..60c18ff2 100644 --- a/server/clearfolio.mjs +++ b/server/clearfolio.mjs @@ -176,6 +176,59 @@ function clearfolioArtifactOrigins(baseUrl) { return trustedOrigins; } +/** + * Return a non-secret operator action for one Clearfolio configuration failure. + * + * @param {string} code - Stable `ClearfolioConfigurationError` code. + * @returns {string} A concrete remediation instruction containing no secret values. + */ +function clearfolioConfigurationAction(code) { + if (code === 'clearfolio_not_configured') { + return 'Set CLEARFOLIO_URL and CLEARFOLIO_HMAC_SECRET, or use SCOPEWEAVE_DEV=1 only for local development.'; + } + if (code === 'clearfolio_hmac_secret_invalid') { + return `Set CLEARFOLIO_HMAC_SECRET to at least ${MIN_HMAC_SECRET_LENGTH} non-whitespace characters.`; + } + if (code === 'clearfolio_artifact_origins_invalid') { + return 'Set CLEARFOLIO_ARTIFACT_ORIGINS to comma-separated HTTPS origins without credentials, path, query, or fragment, or unset it.'; + } + return 'Set CLEARFOLIO_URL to a root HTTPS origin without credentials, path, query, or fragment.'; +} + +/** + * Describe whether the optional Clearfolio capability is locally ready to serve. + * + * This is configuration readiness only: it intentionally performs no DNS, HTTP, + * authentication, or provider-health request, so ScopeWeave liveness cannot be + * coupled to an optional downstream dependency. The development mock is reported + * explicitly and never masquerades as production-provider readiness. + * + * @returns {{ready:boolean,mode:'provider'|'development_mock'|'unavailable',reason:string|null,action:string|null}} Safe capability state. + */ +export function clearfolioCapabilityStatus() { + if (clearfolioMock) { + return { + ready: true, + mode: 'development_mock', + reason: null, + action: 'Configure a Clearfolio provider before using this deployment for production document conversion.', + }; + } + try { + const configuration = clearfolioConfiguration(); + clearfolioArtifactOrigins(configuration.baseUrl); + return { ready: true, mode: 'provider', reason: null, action: null }; + } catch (error) { + if (!(error instanceof ClearfolioConfigurationError)) throw error; + return { + ready: false, + mode: 'unavailable', + reason: error.code, + action: clearfolioConfigurationAction(error.code), + }; + } +} + /** * Sign tenant claims using the Clearfolio HMAC interoperability contract. * diff --git a/server/server.mjs b/server/server.mjs index c84c2e25..5ff61ad9 100644 --- a/server/server.mjs +++ b/server/server.mjs @@ -1,7 +1,14 @@ import { serve } from '@hono/node-server'; import { app } from './app.mjs'; +import { clearfolioCapabilityStatus } from './clearfolio.mjs'; const port = Number(process.env.PORT) || 8787; +const clearfolioCapability = clearfolioCapabilityStatus(); +console.log(JSON.stringify({ + event: 'capability.readiness', + capability: 'clearfolio', + ...clearfolioCapability, +})); serve({ fetch: app.fetch, port }, (info) => { console.log(`ScopeWeave API listening on http://localhost:${info.port}`); }); diff --git a/tests/unit/clearfolio-capability-readiness.test.mjs b/tests/unit/clearfolio-capability-readiness.test.mjs new file mode 100644 index 00000000..4b5e96fb --- /dev/null +++ b/tests/unit/clearfolio-capability-readiness.test.mjs @@ -0,0 +1,116 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; + +const JWT_SECRET = '0123456789abcdef0123456789abcdef'; +const HMAC_SECRET = 'clearfolio-shared-secret-32-bytes!!'; + +/** + * Read liveness and Clearfolio capability state from a fresh ScopeWeave process. + * + * Each probe uses a new Node process because Clearfolio configuration is bound at + * module import time. The child replaces `fetch` with a throwing function so + * readiness is proven from local configuration only and never turns liveness or + * startup diagnostics into provider traffic. + * + * @param {Record} overrides - Environment values for the child. + * @returns {{health:{status:number,body:Record},capability:Record}} Probe result. + */ +function capabilityProbe(overrides = {}) { + const env = { ...process.env }; + delete env.SCOPEWEAVE_DEV; + delete env.CLEARFOLIO_URL; + delete env.CLEARFOLIO_HMAC_SECRET; + delete env.CLEARFOLIO_ARTIFACT_ORIGINS; + Object.assign(env, overrides, { + SCOPEWEAVE_DB: ':memory:', + SCOPEWEAVE_JWT_SECRET: JWT_SECRET, + }); + + const script = ` + globalThis.fetch = async () => { throw new Error('readiness must not call a provider'); }; + const { clearfolioCapabilityStatus } = await import('./server/clearfolio.mjs?capability=' + Date.now()); + const { app } = await import('./server/app.mjs?capability-health=' + Date.now()); + const response = await app.request('/api/health'); + process.stdout.write(JSON.stringify({ + health: { status: response.status, body: await response.json() }, + capability: clearfolioCapabilityStatus(), + })); + `; + const child = spawnSync(process.execPath, ['--input-type=module', '-e', script], { + cwd: process.cwd(), + env, + encoding: 'utf8', + }); + assert.equal(child.status, 0, child.stderr || child.stdout); + return JSON.parse(child.stdout); +} + +function expectLiveHealth(probe) { + assert.deepEqual(probe.health, { status: 200, body: { ok: true } }); +} + +test('liveness stays healthy while unconfigured production reports Clearfolio unavailable', () => { + const probe = capabilityProbe(); + expectLiveHealth(probe); + assert.deepEqual(probe.capability, { + ready: false, + mode: 'unavailable', + reason: 'clearfolio_not_configured', + action: 'Set CLEARFOLIO_URL and CLEARFOLIO_HMAC_SECRET, or use SCOPEWEAVE_DEV=1 only for local development.', + }); +}); + +test('explicit development mock is visible without masquerading as production provider readiness', () => { + const probe = capabilityProbe({ SCOPEWEAVE_DEV: '1' }); + expectLiveHealth(probe); + assert.deepEqual(probe.capability, { + ready: true, + mode: 'development_mock', + reason: null, + action: 'Configure a Clearfolio provider before using this deployment for production document conversion.', + }); +}); + +test('valid production configuration reports provider readiness without provider traffic', () => { + const probe = capabilityProbe({ + CLEARFOLIO_URL: 'https://clearfolio.example', + CLEARFOLIO_HMAC_SECRET: HMAC_SECRET, + }); + expectLiveHealth(probe); + assert.deepEqual(probe.capability, { + ready: true, + mode: 'provider', + reason: null, + action: null, + }); +}); + +test('invalid production configuration degrades only Clearfolio capability and gives a safe next action', () => { + const probe = capabilityProbe({ + CLEARFOLIO_URL: 'http://clearfolio.example', + CLEARFOLIO_HMAC_SECRET: HMAC_SECRET, + }); + expectLiveHealth(probe); + assert.deepEqual(probe.capability, { + ready: false, + mode: 'unavailable', + reason: 'clearfolio_transport_insecure', + action: 'Set CLEARFOLIO_URL to a root HTTPS origin without credentials, path, query, or fragment.', + }); +}); + +test('invalid artifact-origin policy is readiness-visible before provider transport', () => { + const probe = capabilityProbe({ + CLEARFOLIO_URL: 'https://clearfolio.example', + CLEARFOLIO_HMAC_SECRET: HMAC_SECRET, + CLEARFOLIO_ARTIFACT_ORIGINS: 'https://cdn.example/files', + }); + expectLiveHealth(probe); + assert.deepEqual(probe.capability, { + ready: false, + mode: 'unavailable', + reason: 'clearfolio_artifact_origins_invalid', + action: 'Set CLEARFOLIO_ARTIFACT_ORIGINS to comma-separated HTTPS origins without credentials, path, query, or fragment, or unset it.', + }); +});