feat: executable rules and hierarchy validation#29
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Add two new validation passes that run after JSON schema validation: - **Hierarchy validation** (`validate-hierarchy.ts`): checks parent-child type relationships against the schema's `hierarchy` array, with support for `allowSelfRef` and `allowSkipLevels` metadata flags. - **Executable rules** (`validate-rules.ts`, `evaluate-rule.ts`): evaluates JSONata expressions stored in schema `_metadata.rules` against the node set. Rules are organised into `validation`, `coherence`, and `bestPractice` categories. Violations are grouped by category in the output. Schema changes: - `_strict.json`: coherence rules (active outcome/opportunity count) and a best-practice rule (solution quantity per opportunity) with JSONata checks. - `general.json`: adds `aliases` and `allowSelfRef` metadata. - `strict_ost.json`: removes redundant local `_metadata` (now inherited from `_strict`), tightens `assumption_test` type to `const`. - `loadHierarchy` replaced by `loadMetadata` which returns the full metadata object including aliases, allowSelfRef, and rules. - `resolveNodeType` added for alias resolution; `resolvedType` field added to `SpaceNode` and populated at parse time. Closes #21, #16
| "Only one target opportunity should be active at a time" | ||
| "coherence": [ | ||
| { | ||
| "id": "active-outcome-count", |
There was a problem hiding this comment.
This isn't coherence. Add a workflow category for both of these rules.
Coherence is more for things like "the goal is to make $10M" when "the opportunity is to rebuild the design system". Coherence rules are likely to be more qualitative.
There was a problem hiding this comment.
Also add a workflow check that an active (status) node should have an active (status) parent.
Ideally this goes into _shared but we don't have a way to merge rules yet - now we have a reason to want #17 (comment)
| "id": "solution-quantity", | ||
| "description": "Explore multiple candidate solutions (aim for at least three) for the target opportunity", | ||
| "type": "opportunity", | ||
| "check": "$count(nodes[resolvedParentTitle=$$.current.title and resolvedType='solution']) >= 3" |
There was a problem hiding this comment.
This should only apply when status in ['exploring', 'active']
| } | ||
| } | ||
|
|
||
| for (const node of nodes) { |
There was a problem hiding this comment.
Also check that allowSelfRef is strictly a subset of hierarchy. If not -> violation
…g validation - Add `workflow` rule category for process discipline checks; move active-outcome/opportunity-count rules from coherence to workflow - Add `active-node-parent-active` workflow rule: an active node's parent must also be active - Update `solution-quantity` best-practice to only apply when opportunity status is exploring or active - Check `allowSelfRef ⊆ hierarchy` in validate command, reporting mismatches as config errors with a new configErrors result field - Rename schemaErrors/schemaValidCount → nodeErrors/validCount for clarity - Update docs/rules.md: add workflow category, clarify scope is orthogonal to category, update examples
| const context = buildEvalContext(mockNode, allNodes, mockNodeIndex); | ||
| const result = await evaluateExpression('invalid.syntax.('!, context); |
There was a problem hiding this comment.
The string literal closes earlier than intended. In JavaScript/TypeScript the tokenizer closes the string at the second ' (after (), so this is parsed as:
evaluateExpression('invalid.syntax.(', !context)!context evaluates to false (boolean), which is passed as the second argument instead of the EvalContext object. TypeScript should flag this as a type error. The test passes by coincidence because the function's catch block returns false regardless of whether the failure was an invalid expression or an invalid context argument.
| const context = buildEvalContext(mockNode, allNodes, mockNodeIndex); | |
| const result = await evaluateExpression('invalid.syntax.('!, context); | |
| const result = await evaluateExpression("invalid.syntax.('!", context); |
1 participant
Summary
hierarchyarray in schema metadata, with support forallowSelfRef(e.g. nested opportunities) andallowSkipLevelsflags_metadata.rules, organised intovalidation,coherence, andbestPracticecategoriesloadMetadata) to return aliases, allowSelfRef, and rules alongside the hierarchy — used by all read paths and the validate commandresolvedTypetoSpaceNode, populated at parse time by resolving type aliases from schema metadata_strict.json: active outcome/opportunity count (coherence) and solution quantity per opportunity (best-practice)Closes #23, Closes #16
Test plan
bun run test— all unit tests pass (evaluate-rule, validate-hierarchy, validate-rules, validate-strict)bun run test:smoke— smoke tests pass against configured spacesbun run src/index.ts validate <strict-ost-space>— rule violations and hierarchy violations reported correctly in output