From a8d6a8ca0b99763c0320075b0bf02b86c693774c Mon Sep 17 00:00:00 2001 From: Aaron Stannard Date: Mon, 10 Aug 2026 15:13:32 +0000 Subject: [PATCH] design: simplify v0.3 consumer API --- IMPLEMENTATION_PLAN.md | 10 + PROJECT_CONTEXT.md | 8 +- .../v0-3-structured-shell-analysis/design.md | 701 ++++++++++-------- .../proposal.md | 21 +- .../specs/bounded-shell-analysis/spec.md | 29 +- .../specs/consumer-compatibility/spec.md | 14 +- .../executable-command-projection/spec.md | 78 +- .../specs/explicit-redirect-semantics/spec.md | 42 +- .../specs/structured-shell-syntax/spec.md | 26 +- .../v0-3-structured-shell-analysis/tasks.md | 35 +- 10 files changed, 574 insertions(+), 390 deletions(-) diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md index 34fad25..b95cbf3 100644 --- a/IMPLEMENTATION_PLAN.md +++ b/IMPLEMENTATION_PLAN.md @@ -17,6 +17,16 @@ priorities. > below). What remains is the downstream Netclaw integration, which needs > actions outside this repository. +- [ ] **v0.3 prerelease consumer-API correction.** Preserve the stable v0.2 + API and the `Syntax` / `Commands` / `Clauses` ingestion lanes, but replace + the alpha-only sparse coordinate and public property-bag model before + stable v0.3. Return one parser-owned analyzed argument per authored + non-cwd argument, use closed value-domain and redirect alternatives, + reference actual ancestor nodes, and remove public syntax vocabulary + that stable v0.3 never emits. Update the OpenSpec and source mocks first; + then update implementation, snapshots, corpus DTOs, consumer guide, + README, and Netclaw together. No compatibility shim for 0.3 alphas. + - [x] **v0.3 host-selected grammar and PowerShell dialect — library slice.** The executor selects one top-level parser; Bash never cross-parses `pwsh` payloads and PowerShell never cross-parses `bash -c` payloads. Add the extend-only diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 057bae1..f1ce924 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -152,9 +152,11 @@ and IDE-grade concrete syntax remain non-goals. ## Architectural Constraints - **Public API in `SPEC.md` §2 is the contract.** Everything else is - `internal`. During `0.x`, renaming or removing public fields requires a - deliberate minor version bump and migration notes; after `1.0`, it requires - a major version bump. + `internal`. Stable `0.x` releases use the minor version as their breaking + boundary. Prerelease APIs may change before their corresponding stable + release and receive explicit prerelease migration notes; they do not force a + second minor-version bump. After `1.0`, breaking changes require a major + version bump. - **`IShellParser` is the multi-shell seam.** Additional parsers such as Windows `cmd` must be addable without reshaping consumer code. - **No native dependencies.** AOT-trim friendly; ship a single managed diff --git a/openspec/changes/v0-3-structured-shell-analysis/design.md b/openspec/changes/v0-3-structured-shell-analysis/design.md index 2b1b293..94bede1 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/design.md +++ b/openspec/changes/v0-3-structured-shell-analysis/design.md @@ -80,10 +80,10 @@ what an older consumer can observe. Public syntax nodes derive from one `ShellSyntaxNode` base. The base prevents external derivation so ShellSyntaxTree owns the complete node family. Common execution structure may use shell-neutral nodes such as blocks, simple -commands, pipelines, command lists, groups, foreach-style loops, condition -loops, and branches. Condition and branch nodes are reserved vocabulary in -stable v0.3 rather than parser-emitted grammar. A shell-specific public node is -preferred whenever a shared type would erase material semantics. +commands, pipelines, command lists, groups, foreach-style loops, substitutions, +and execution regions. Condition loops and branches remain future grammar and +do not reserve stable-v0.3 public types or enum members. A shell-specific public +node is preferred whenever a shared type would erase material semantics. Consumers are not required to exhaustively match node types for authorization. They use `Commands`; a consumer that does inspect `Syntax` must fail closed or @@ -426,7 +426,8 @@ delimited. Stable v0.3 adds one `ExecutionRegionSyntax` node to the closed syntax family. It exposes independent origin, phase, timing, and cardinality discriminants, -an optional owning `ClauseElement` coordinate, and the recursively parsed body. +an optional reference to the owning argument `ClauseElement`, and the +recursively parsed body. The region may be attached to `SimpleCommandSyntax.ExecutionRegions` or appear directly as a statement for `& {}` and `. {}`. Direct invocation operators are shell syntax and do not create synthetic command occurrences; a command-owned @@ -488,8 +489,9 @@ concurrency. An explicit false switch value does not change scheduling facts. The analyzer separately applies semantic phase order. PowerShell's binder can assign Begin, Process, and End roles to multiple `ForEach-Object` blocks even when parameter or positional order does not match runtime phase order. The -region's `HostClauseElementIndex` is therefore a correlation coordinate, not -an execution-order index. +region's `HostArgument` therefore references the exact authored +`ClauseElement`; its position in the host's region collection is authored +order, not semantic execution order. The stable-v0.3 PowerShell 7 catalog covers direct call and dot-source blocks; `ForEach-Object` Begin, Process, End, RemainingScripts, and Parallel; @@ -530,15 +532,15 @@ than inventing offsets into escaped or encoded outer text. not one entry per predicted runtime iteration. Each occurrence carries its `Clause`, immediate structural role, compositional ancestry suitable for analysis and diagnostics, and an explicit completeness fact. Immediate roles -include at least ordinary, pipeline stage, condition, iterator, loop body, -branch, substitution, and execution region; ancestry frames retain every outer role, such as a -pipeline stage nested inside a loop body. The final names are locked with the -public API review. +are ordinary, pipeline stage, iterator, loop body, substitution, and execution +region; ancestry frames retain every outer role, such as a pipeline stage +nested inside a loop body. -Condition and iterator commands are never omitted. Mutually exclusive branch -commands all appear because the collection is a may-execute set. Runtime loop -counts do not duplicate occurrences; bounded variable domains describe the -possible effective values at the occurrence. +Iterator commands are never omitted. Runtime loop counts do not duplicate +occurrences; bounded variable domains describe the possible effective values +at the occurrence. A future condition/branch slice must preserve all commands +from mutually exclusive branches in this may-execute set, but it does not +reserve stable-v0.3 roles or ancestry regions. Completeness and value precision are independent. A structurally complete occurrence may conservatively contain an `Unknown` value domain when the @@ -554,9 +556,10 @@ projection exposes a discovered subset. For a fully parseable result, `ParsedCommand.Clauses` contains every authored simple command occurrence in source order, including nested iterator, -condition, branch, body, substitution, and execution-region commands. It does not invent +loop-body, substitution, and execution-region commands. It does not invent compound operators across structural boundaries. Existing `Clause.Operator` -values are retained only for actual authored relationships. +values are retained only for actual authored relationships. A future +condition/branch slice must apply the same conservative flattening rule. Projection order is deterministic. Disjoint executable regions follow authored source order. An enclosed substitution precedes its containing simple command, @@ -586,7 +589,7 @@ The analysis domain distinguishes: - `Exact`: one completely proved shell value; - `FiniteSet`: a bounded, fully enumerated set of values; -- `Pattern`: a shell-specific symbolic pattern plus conservative covering +- `PathPattern`: a shell-specific symbolic pattern plus conservative covering scope when one can be proved without enumeration; - `Unknown`: runtime-produced, mutated, indirectly expanded, unsupported, or above the configured proof bound. @@ -770,7 +773,7 @@ Exact and finite Bash `for ... in` domains are analyzed in authored iteration order within the 32-candidate cap. Each iteration consumes the joined reachable state from the preceding iteration, and the loop exit joins every reachable normal exit with the zero-iteration path when zero iterations remain possible. -Pattern and unknown domains use a bounded fixed point with widening to +Path-pattern and unknown domains use a bounded fixed point with widening to `Unknown`; they are never treated as one representative iteration. Until `break`, `continue`, `return`, `exit`, and `exec` have explicit transfer semantics, a supported loop region containing one of them fails closed instead @@ -1038,11 +1041,14 @@ corpus remains sanitized under the existing PII audit. unaffected compatibility facts. Paired shell-oracle corrections to false path or `DynamicSkip` claims are explicit compatibility notes, not hidden behavior-preserving changes. -5. Migrate Netclaw to `Commands` and explicit redirect facts before enabling - supported control flow for authorization reuse. -6. Add Bash and PowerShell vertical slices behind corpus and integration gates. -7. Promote 0.3.0 only after both shells, old-consumer fail-closed behavior, and - the new Netclaw consumer path pass their acceptance matrices. +5. Before stable v0.3, replace the alpha-only sparse/property-bag result model + with Appendix A's parser-owned joined arguments and closed alternatives. + Do not add alpha compatibility shims. +6. Migrate Netclaw to the corrected `Commands` and redirect shapes, then remove + its coordinate and property-invariant validation. +7. Add Bash and PowerShell vertical slices behind corpus and integration gates. +8. Promote 0.3.0 only after both shells, stable-v0.2 compatibility behavior, + and the corrected Netclaw consumer path pass their acceptance matrices. Before stable 0.3.0, a flawed new surface can be revised with prerelease migration notes. After stable release, removals or renames follow the normal @@ -1054,21 +1060,23 @@ The first paired design-corpus review resolves the original open questions as follows. These decisions are normative for this change and are synchronized into the release specifications before production types are added. -1. Appendix A locks the public type names, members, enum zero values, and - defaults. The syntax hierarchy is a closed record family with an explicit - kind; command discovery remains collection-based so authorization consumers - never need a type switch. +1. Appendix A locks the stable-v0.3 type names and members. The syntax, value- + domain, redirect-source, and redirect-analysis families are closed, + library-owned record hierarchies. Runtime type is the discriminant; a + redundant `Kind` property is not exposed. Command discovery remains + collection-based so authorization consumers never walk syntax to find + executable leaves. 2. A value domain contains at most 32 candidates. Supported structural nesting is at most 16 container nodes. Existing decoded-command wrapper recursion - remains capped at 5. The limits are public static get-only properties, not - caller-configurable parser options or compile-time constants. Candidate - overflow produces `Unknown`; structural or wrapper-depth overflow makes the - whole result unparseable. + remains capped at 5. The limits are fixed implementation contracts, + documented but not exposed as caller-configurable options or public API. + Candidate overflow produces `Unknown`; structural or wrapper-depth overflow + makes the whole result unparseable. 3. Within one successful `ParsedCommand`, a simple-command syntax leaf, its command occurrence, and its compatibility `Clauses` entry reference the identical `Clause` instance. This is an in-memory parser-result guarantee, not a serialization reference-preservation guarantee. -4. The initial `Pattern` domain is limited to a Bash path-shaped glob with no +4. The initial `PathPattern` domain is limited to a Bash path-shaped glob with no dynamic root, substitution, indirect expansion, parent-traversal segment, or glob-bearing dot-prefixed segment that can match `..` when `globskipdots` is disabled. Its `CoveringDirectory` is the exact static directory prefix, @@ -1095,86 +1103,100 @@ into the release specifications before production types are added. proved data. Unknown receivers expose supported non-pipeline body commands with incomplete facts rather than silently treating them as data; unproved interior pipelines fail atomically. +8. Compatibility is measured against stable v0.2. The v0.3 alpha packages are + disposable integration previews: no alias, obsolete member, adapter, or + binary shim preserves their sparse `EffectiveArguments`, property-bag value + domains, redirect discriminant enums, public fixed-limit type, redundant + syntax kind, or never-emitted condition/branch types. On every unparseable result, `Commands` and the v0.2 `Clauses` projection are empty. `Syntax` may contain a partial diagnostic tree, but it cannot be used as authorization evidence. This makes accidental subset authorization harder for both old and new consumers. -## Appendix A: Locked Public API Contract +## Appendix A: Corrected Stable-v0.3 Public API Contract -The following shape is normative for this OpenSpec change. XML documentation -and the public API snapshot must preserve these members and defaults when task -group 1 synchronizes the contract into `SPEC.md` and `SPEC.POWERSHELL.md`. +This shape supersedes the public v0.3 surface shipped in every `0.3.0-alpha.*` +package. Compatibility is required against stable v0.2 only. The existing v0.2 +types and public setters remain unchanged; every new v0.3 result type is +constructed by the library and exposes read-only facts to consumers. Internal +constructors and `internal init` accessors are shown to make ownership explicit +but are not consumer-callable API. -### Structural node family +Every exposed `IReadOnlyList` is backed by a library-owned immutable +collection or a defensive copy. The parser never publishes a mutable array or +list that a consumer can cast and change after projection. + +Runtime type is the discriminant for each closed family. Consumers use an +exhaustive type switch with a default fail-closed arm. The library may add a +derived type in a later stable release, so the default arm remains required +even though external assemblies cannot derive their own implementations. + +### Authored structural node family ```csharp namespace ShellSyntaxTree; public abstract record ShellSyntaxNode { - // Prevent consumers from extending the parser-owned node family. private protected ShellSyntaxNode() { } + private protected abstract object LibraryOwnership { get; } - public abstract ShellSyntaxKind Kind { get; } - public int? SourceStart { get; init; } - public int? SourceLength { get; init; } -} - -public enum ShellSyntaxKind -{ - Unknown, - Block, - SimpleCommand, - Pipeline, - CommandList, - Group, - ForEach, - ConditionLoop, - Conditional, - ConditionalBranch, - CommandSubstitution, - ExecutionRegion, + public int? SourceStart { get; internal init; } + public int? SourceLength { get; internal init; } } public sealed record ShellBlockSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.Block; - public IReadOnlyList Statements { get; init; } = []; + internal ShellBlockSyntax() { } + private protected override object LibraryOwnership => this; + + public IReadOnlyList Statements { get; internal init; } = []; } public sealed record SimpleCommandSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.SimpleCommand; - public Clause Clause { get; init; } = new(); - public IReadOnlyList Substitutions { get; init; } = []; - public IReadOnlyList ExecutionRegions { get; init; } = []; + internal SimpleCommandSyntax() { } + private protected override object LibraryOwnership => this; + + public Clause Clause { get; internal init; } = new(); + public IReadOnlyList Substitutions + { get; internal init; } = []; + public IReadOnlyList ExecutionRegions + { get; internal init; } = []; } public sealed record PipelineSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.Pipeline; - public IReadOnlyList Stages { get; init; } = []; + internal PipelineSyntax() { } + private protected override object LibraryOwnership => this; + + public IReadOnlyList Stages { get; internal init; } = []; } public sealed record CommandListSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.CommandList; - public IReadOnlyList Items { get; init; } = []; + internal CommandListSyntax() { } + private protected override object LibraryOwnership => this; + + public IReadOnlyList Items { get; internal init; } = []; } public sealed record CommandListItemSyntax { - public CompoundOperator Operator { get; init; } - public ShellSyntaxNode Command { get; init; } = new ShellBlockSyntax(); + internal CommandListItemSyntax() { } + + public CompoundOperator Operator { get; internal init; } + public ShellSyntaxNode Command { get; internal init; } = null!; } public sealed record GroupSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.Group; - public ShellGroupKind GroupKind { get; init; } - public ShellBlockSyntax Body { get; init; } = new(); + internal GroupSyntax() { } + private protected override object LibraryOwnership => this; + + public ShellGroupKind GroupKind { get; internal init; } + public ShellBlockSyntax Body { get; internal init; } = null!; } public enum ShellGroupKind @@ -1186,70 +1208,44 @@ public enum ShellGroupKind public sealed record ForEachSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.ForEach; - public LoopBindingSyntax Binding { get; init; } = new(); - public ShellSourceFragment Iterable { get; init; } = new(); - public ShellBlockSyntax IteratorCommands { get; init; } = new(); - public ShellBlockSyntax Body { get; init; } = new(); -} - -public sealed record LoopBindingSyntax -{ - public string Name { get; init; } = ""; - public ShellSourceFragment Source { get; init; } = new(); + internal ForEachSyntax() { } + private protected override object LibraryOwnership => this; + + public string BindingName { get; internal init; } = ""; + public ShellSourceFragment BindingSource { get; internal init; } = null!; + public ShellSourceFragment Iterable { get; internal init; } = null!; + public ShellBlockSyntax IteratorCommands { get; internal init; } = null!; + public ShellBlockSyntax Body { get; internal init; } = null!; } public sealed record ShellSourceFragment { - public string Raw { get; init; } = ""; - public int? SourceStart { get; init; } - public int? SourceLength { get; init; } -} + internal ShellSourceFragment() { } -public sealed record ConditionLoopSyntax : ShellSyntaxNode -{ - public override ShellSyntaxKind Kind => ShellSyntaxKind.ConditionLoop; - public ConditionLoopKind LoopKind { get; init; } - public ShellBlockSyntax Condition { get; init; } = new(); - public ShellBlockSyntax Body { get; init; } = new(); -} - -public enum ConditionLoopKind -{ - Unknown, - While, - Until, -} - -public sealed record ConditionalSyntax : ShellSyntaxNode -{ - public override ShellSyntaxKind Kind => ShellSyntaxKind.Conditional; - public IReadOnlyList Branches { get; init; } = []; - public ShellBlockSyntax? Else { get; init; } -} - -public sealed record ConditionalBranchSyntax : ShellSyntaxNode -{ - public override ShellSyntaxKind Kind => ShellSyntaxKind.ConditionalBranch; - public ShellBlockSyntax Condition { get; init; } = new(); - public ShellBlockSyntax Body { get; init; } = new(); + public string Raw { get; internal init; } = ""; + public int? SourceStart { get; internal init; } + public int? SourceLength { get; internal init; } } public sealed record CommandSubstitutionSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.CommandSubstitution; - public ShellBlockSyntax Body { get; init; } = new(); + internal CommandSubstitutionSyntax() { } + private protected override object LibraryOwnership => this; + + public ShellBlockSyntax Body { get; internal init; } = null!; } public sealed record ExecutionRegionSyntax : ShellSyntaxNode { - public override ShellSyntaxKind Kind => ShellSyntaxKind.ExecutionRegion; - public ExecutionRegionOrigin Origin { get; init; } - public int? HostClauseElementIndex { get; init; } - public ExecutionRegionPhase Phase { get; init; } - public ExecutionRegionTiming Timing { get; init; } - public ExecutionRegionCardinality Cardinality { get; init; } - public ShellBlockSyntax Body { get; init; } = new(); + internal ExecutionRegionSyntax() { } + private protected override object LibraryOwnership => this; + + public ExecutionRegionOrigin Origin { get; internal init; } + public ClauseElement? HostArgument { get; internal init; } + public ExecutionRegionPhase Phase { get; internal init; } + public ExecutionRegionTiming Timing { get; internal init; } + public ExecutionRegionCardinality Cardinality { get; internal init; } + public ShellBlockSyntax Body { get; internal init; } = null!; } public enum ExecutionRegionOrigin @@ -1290,55 +1286,30 @@ public enum ExecutionRegionCardinality } ``` -`ForEachSyntax` shares only proved execution structure. `Iterable.Raw` preserves -the shell-specific authored expression without claiming that Bash words and -PowerShell expressions share a grammar; `IteratorCommands` separately exposes -commands discovered inside that expression. Bounded values live on command -occurrences, not on this display tree. Internal parse nodes and expression -adapters remain shell-specific. - -Every enum introduced in v0.3 reserves zero as `Unknown`, except existing v0.2 -enums whose zero values are already locked. Consumers fail closed on `Unknown` -or an unrecognized numeric value. The source implementation pairs the shown -`private protected` ordinary base constructor with an assembly-only abstract -ownership member. Records synthesize a protected copy constructor, so the -ordinary constructor alone would still permit a specially constructed external -derived record. The non-public abstract member makes every external concrete -implementation fail compilation without adding to the public contract; every -library-owned sealed node implements it internally. Later library versions may -add derived records, so authorization code still needs a default fail-closed -type-switch arm. - -`ExecutionRegionSyntax` is the structural relationship between an authored -body and its activation. `HostClauseElementIndex` is null for direct `& {}` or -`. {}` statements and is the non-negative index into the owning -`SimpleCommandSyntax.Clause.Elements` for a proved command argument binding. -The node's source range covers the direct invocation or the bound script-block -token; `Body` covers the recursively parsed interior. Attached regions are -stored in authored source order. `Phase` lets the shell-specific analyzer apply -Begin/Process/End and initialization/main schedules without rewriting the tree. - -`Timing=Synchronous` means the body completes as part of the containing -invocation, `Concurrent` means instances or the containing continuation may -overlap, and `Deferred` means registration and later trigger are distinct. -`Cardinality=Once` means one activation per proved host invocation, -`OncePerInputObject` is the pipeline callback relationship, and `ZeroOrMore` -is an externally triggered callback. These facts do not specify variable, -location, command-resolution, runspace, or process propagation. Consumers use -occurrence analysis for those facts and fail closed on unknown enum values. - -### Command occurrence and bounded values +Stable v0.3 exposes only node types the stable parser can emit. Condition loops +and conditionals remain fail closed and therefore do not reserve public records +or enum members. `ForEachSyntax` inlines its one-use binding record. +`ExecutionRegionSyntax.HostArgument` is null for direct call or dot-source +regions and otherwise references the exact argument `ClauseElement` owned by +the containing simple command. Source ranges remain exact or jointly null. + +### Command occurrences and analyzed arguments ```csharp public sealed record CommandOccurrence { - public Clause Clause { get; init; } = new(); - public CommandOccurrenceRole ImmediateRole { get; init; } - public IReadOnlyList Ancestry { get; init; } = []; - public IReadOnlyList EffectiveArguments { get; init; } = []; - public ShellValueDomain WorkingDirectory { get; init; } = ShellValueDomain.Unknown; - public IReadOnlyList Redirects { get; init; } = []; - public bool IsComplete { get; init; } + internal CommandOccurrence() { } + + public Clause Clause { get; internal init; } = new(); + public CommandOccurrenceRole ImmediateRole { get; internal init; } + public IReadOnlyList Ancestry + { get; internal init; } = []; + public IReadOnlyList Arguments + { get; internal init; } = []; + public ShellValueDomain WorkingDirectory { get; internal init; } = null!; + public IReadOnlyList Redirects + { get; internal init; } = []; + public bool IsComplete { get; internal init; } } public enum CommandOccurrenceRole @@ -1346,21 +1317,19 @@ public enum CommandOccurrenceRole Unknown, Ordinary, PipelineStage, - Condition, Iterator, LoopBody, - Branch, Substitution, ExecutionRegion, } public sealed record CommandAncestryFrame { - public ShellSyntaxKind AncestorKind { get; init; } - public CommandAncestryRegion Region { get; init; } - public int? ChildIndex { get; init; } - public int? SourceStart { get; init; } - public int? SourceLength { get; init; } + internal CommandAncestryFrame() { } + + public ShellSyntaxNode Ancestor { get; internal init; } = null!; + public CommandAncestryRegion Region { get; internal init; } + public int? ChildIndex { get; internal init; } } public enum CommandAncestryRegion @@ -1372,186 +1341,291 @@ public enum CommandAncestryRegion GroupBody, Iterator, LoopBody, - Condition, - Branch, Substitution, ExecutionRegion, } -public sealed record EffectiveArgument +public sealed record AnalyzedArgument +{ + internal AnalyzedArgument() { } + + public Arg Argument { get; internal init; } = null!; + public ClauseElement Element { get; internal init; } = null!; + public ShellValueDomain Value { get; internal init; } = null!; +} +``` + +`Arguments` contains exactly one entry for every non-cwd-attribution +`Clause.Args` entry, in authored argument order. `Argument` and `Element` +reference the corresponding objects already owned by `Clause.Args` and +`Clause.Elements`. The parser performs that correlation once; consumers do not +validate or join a sparse coordinate. `Value` is the effective shell value: +static authored arguments normally produce `Exact`, bounded visits may produce +`FiniteSet` or `PathPattern`, and an unproved value produces `Unknown`. +Synthetic v0.2 cwd-attribution arguments remain available only through +`Clause.Args`; `WorkingDirectory` is their canonical v0.3 replacement. + +The join is not necessarily one-to-one. Inline option spellings such as +`--work-tree=../repo`, PowerShell colon-bound parameters, and equivalent native +equals forms can produce more than one compatibility `Arg` from one authored +token. Each resulting `AnalyzedArgument` references the same authored +`ClauseElement`; argument order and distinct `Arg` references preserve the +consumer-facing binding split. + +`Ancestry` is ordered outermost to innermost. Each frame references the actual +ancestor node, so its runtime type and source range cannot disagree with copied +kind/span fields. The collection excludes the simple-command leaf. +`ImmediateRole` and `Region` contain only relationships emitted by stable v0.3. + +### Closed shell-value domain + +```csharp +public abstract record ShellValueDomain +{ + private protected ShellValueDomain() { } + private protected abstract object LibraryOwnership { get; } + + public sealed record Unknown : ShellValueDomain + { + internal Unknown() { } + private protected override object LibraryOwnership => this; + } + + public sealed record Exact : ShellValueDomain + { + internal Exact(string value) => Value = value; + private protected override object LibraryOwnership => this; + + public string Value { get; } + } + + public sealed record FiniteSet : ShellValueDomain + { + internal FiniteSet(IEnumerable values) + => Values = Array.AsReadOnly(values.ToArray()); + private protected override object LibraryOwnership => this; + + public IReadOnlyList Values { get; } + } + + public sealed record PathPattern : ShellValueDomain + { + internal PathPattern(string pattern, string coveringDirectory) + { + Pattern = pattern; + CoveringDirectory = coveringDirectory; + } + private protected override object LibraryOwnership => this; + + public string Pattern { get; } + public string CoveringDirectory { get; } + } +} +``` + +`Unknown` carries no payload. `Exact.Value` is one proved value. +`FiniteSet.Values` contains 2-32 non-null, distinct values. `PathPattern` has a +non-empty pattern and covering directory and is limited to the locked Bash path +glob proof. Only the library constructs these alternatives, so consumers do +not repeat property-bag invariant checks. A consumer still needs a default +type-switch arm for a future derived alternative. + +The candidate cap remains 32, structural nesting remains 16, and decoded +same-language wrapper recursion remains 5. These are fixed parser contracts, +not consumer tuning knobs, and therefore no public `ShellAnalysisLimits` type +is exposed. + +### Closed redirect analysis + +```csharp +public abstract record RedirectSource { - // A stable authored coordinate is safer than correlating by string value. - public int ClauseElementIndex { get; init; } = -1; - public ShellValueDomain Value { get; init; } = ShellValueDomain.Unknown; + private protected RedirectSource() { } + private protected abstract object LibraryOwnership { get; } + + public sealed record Unknown : RedirectSource + { + internal Unknown() { } + private protected override object LibraryOwnership => this; + } + + public sealed record Default : RedirectSource + { + internal Default() { } + private protected override object LibraryOwnership => this; + } + + public sealed record Descriptor : RedirectSource + { + internal Descriptor(int value) => Value = value; + private protected override object LibraryOwnership => this; + + public int Value { get; } + } + + public sealed record PowerShellAllStreams : RedirectSource + { + internal PowerShellAllStreams() { } + private protected override object LibraryOwnership => this; + } } -public sealed record ShellValueDomain +public abstract record RedirectAnalysis { - public static ShellValueDomain Unknown { get; } = new(); + private protected RedirectAnalysis() { } + private protected abstract object LibraryOwnership { get; } - public ShellValueDomainKind Kind { get; init; } - public IReadOnlyList Values { get; init; } = []; - public string? Pattern { get; init; } - public string? CoveringDirectory { get; init; } + public Redirect Authored { get; internal init; } = null!; + public RedirectSource Source { get; internal init; } = null!; + public bool IsComplete { get; internal init; } } -public enum ShellValueDomainKind +public sealed record FileRedirectAnalysis : RedirectAnalysis { - Unknown, - Exact, - FiniteSet, - Pattern, + internal FileRedirectAnalysis(FileRedirectMode mode) => Mode = mode; + private protected override object LibraryOwnership => this; + + public FileRedirectMode Mode { get; } + public ShellValueDomain Target { get; internal init; } = null!; } -public static class ShellAnalysisLimits +public sealed record UnresolvedRedirectAnalysis : RedirectAnalysis { - public static int MaxValueCandidates => 32; - public static int MaxStructuralNesting => 16; - public static int MaxWrapperRecursionDepth => 5; + internal UnresolvedRedirectAnalysis() { } + private protected override object LibraryOwnership => this; } -``` -`Ancestry` is ordered outermost to innermost and contains structural containers, -not the `SimpleCommandSyntax` leaf itself. `ChildIndex` disambiguates repeated -regions such as a pipeline stage or conditional branch. `ImmediateRole` -describes the nearest execution relation; ancestry retains outer relations. -An embedded substitution adds a substitution ancestry frame while its -containing simple-command leaf remains excluded. Iterator or other outer -frames are therefore preserved outside that substitution frame. Its -`ChildIndex` is the authored index within the owning structural collection. - -The contract uses a source-authored element coordinate rather than attaching -derived values directly to `Arg`. This prevents a loop iteration from mutating -the compatibility leaf and provides a place for one authored token to have -multiple possible effective values. Redirect operands use the separate redirect -coordinate below. Inline option bindings retain their one authored -`ClauseElement`; shell expansions that might create more than one argument are -`Unknown` until their boundaries can be proved. An occurrence lifted from a -wrapper may therefore have a valid clause-element index even when that -element's outer source span is unavailable. -An `Unknown` value at one of those coordinates does not by itself make the -occurrence structurally incomplete. - -The parser emits only valid value-domain combinations: - -- `Unknown`: no values, pattern, or covering directory; -- `Exact`: exactly one value and no pattern fields; -- `FiniteSet`: 2–32 distinct values and no pattern fields; -- `Pattern`: no values, a non-empty pattern, and a non-empty covering directory. - -Any internally invalid combination is a parser bug. A consumer reading an -externally persisted or reconstructed instance fails closed rather than trying -to repair it. - -### Explicit redirect facts +public enum FileRedirectMode +{ + Input, + Output, + Append, + CombinedOutput, + CombinedOutputAppend, +} -```csharp -public sealed record RedirectAnalysis +public sealed record DescriptorDuplicateRedirectAnalysis : RedirectAnalysis { - public int RedirectIndex { get; init; } = -1; - public RedirectSource Source { get; init; } = new(); - public RedirectOperation Operation { get; init; } - public int? TargetDescriptor { get; init; } - public ShellValueDomain Target { get; init; } = ShellValueDomain.Unknown; - public HereDocumentAnalysis? HereDocument { get; init; } - public bool IsPathRelevant { get; init; } - public bool IsComplete { get; init; } + internal DescriptorDuplicateRedirectAnalysis() { } + private protected override object LibraryOwnership => this; + + public int TargetDescriptor { get; internal init; } } -public sealed record HereDocumentAnalysis +public sealed record DescriptorMoveRedirectAnalysis : RedirectAnalysis { - public ShellSourceFragment Delimiter { get; init; } = new(); - public ShellSourceFragment Body { get; init; } = new(); - public HereDocumentExpansionMode ExpansionMode { get; init; } - public bool StripLeadingTabs { get; init; } - public bool IsComplete { get; init; } + internal DescriptorMoveRedirectAnalysis() { } + private protected override object LibraryOwnership => this; + + public int TargetDescriptor { get; internal init; } } -public enum HereDocumentExpansionMode +public sealed record DescriptorCloseRedirectAnalysis : RedirectAnalysis { - Unknown, - Literal, - Expand, + internal DescriptorCloseRedirectAnalysis() { } + private protected override object LibraryOwnership => this; } -public sealed record RedirectSource +public sealed record HereDocumentRedirectAnalysis : RedirectAnalysis { - public RedirectSourceKind Kind { get; init; } - public int? Descriptor { get; init; } + internal HereDocumentRedirectAnalysis() { } + private protected override object LibraryOwnership => this; + + public HereDocumentAnalysis Document { get; internal init; } = null!; } -public enum RedirectSourceKind +public sealed record HereStringRedirectAnalysis : RedirectAnalysis { - Unknown, - Default, - Descriptor, - PowerShellAllStreams, + internal HereStringRedirectAnalysis() { } + private protected override object LibraryOwnership => this; + + public ShellValueDomain Data { get; internal init; } = null!; } -public enum RedirectOperation +public sealed record HereDocumentAnalysis +{ + internal HereDocumentAnalysis() { } + + public ShellSourceFragment Delimiter { get; internal init; } = null!; + public ShellSourceFragment Body { get; internal init; } = null!; + public HereDocumentExpansionMode ExpansionMode { get; internal init; } + public bool StripLeadingTabs { get; internal init; } + public bool IsComplete { get; internal init; } +} + +public enum HereDocumentExpansionMode { Unknown, - FileInput, - FileOutput, - FileAppend, - DescriptorDuplicate, - DescriptorClose, - DescriptorMove, - CombinedOutput, - CombinedOutputAppend, - HereDocument, - HereString, + Literal, + Expand, } ``` -Occurrence-specific redirect analysis stays on `CommandOccurrence`; the -existing `Redirect` record remains untouched. This is necessary because a loop -can give one authored redirect target several effective values, while mutating -`Redirect` would also change v0.2 equality and serialization. `RedirectIndex` -correlates to `Clause.Redirects`. `RedirectSource` represents the operator's -default stream, a numeric Bash/PowerShell descriptor, or PowerShell's `*` -selector without losing shell identity. Invalid source-kind/descriptor -combinations are incomplete and fail closed. - -`HereDocument` is non-null only for `HereDocument` operations and preserves the -authored delimiter and body independently. `Literal` means delimiter quoting -disables body expansion; `Expand` means every execution-bearing substitution -must be discovered and surfaced before the redirect can be complete. -`HereString` specifically represents Bash `<<<`; its operand and exact, finite, -or unknown effective data use `Target`. PowerShell `@"..."@` and `@'...'@` -here-strings remain ordinary PowerShell value tokens, not redirects. - -### ParsedCommand composition and consumer entry point +`Authored` references the corresponding v0.2 `Clause.Redirects` leaf. +A `FileRedirectAnalysis` is always path-relevant. Descriptor alternatives are +never paths and their type determines whether a target descriptor exists. +Heredoc and here-string alternatives are stdin data rather than filesystem +targets. `UnresolvedRedirectAnalysis` and `RedirectSource.Unknown` retain a +completely delimited redirect whose operation or source cannot be proved and +are always incomplete. `IsComplete` remains independent because another +structurally valid alternative can still contain unresolved executable +content. This removes +`RedirectIndex`, `RedirectOperation`, `RedirectSourceKind`, +`TargetDescriptor` null conventions, and the independently mutable +`IsPathRelevant` flag. + +Valid source/operation pairs are closed as follows: + +- `RedirectSource.Unknown` is valid only on `UnresolvedRedirectAnalysis`; +- `RedirectSource.Default` may own Bash combined-output, ordinary file, + descriptor, heredoc, or here-string alternatives; +- `RedirectSource.Descriptor` may own ordinary file, descriptor, heredoc, or + here-string alternatives, but never combined-output alternatives; and +- `RedirectSource.PowerShellAllStreams` may own only output/append file + alternatives or a descriptor-duplicate alternative targeting success stream + descriptor `1`. + +Any other internal pair is a projection bug and discards authorization +projections before publication. + +### ParsedCommand composition ```csharp public sealed record ParsedCommand { + // Stable v0.2 member. public string Source { get; init; } = ""; - public ShellBlockSyntax Syntax { get; init; } = new(); - public IReadOnlyList Commands { get; init; } = []; + // New v0.3 parser-owned projections. + public ShellBlockSyntax Syntax { get; internal init; } = new(); + public IReadOnlyList Commands + { get; internal init; } = []; - // v0.2 compatibility projection; not canonical for structured analysis. + // Stable v0.2 compatibility projection. public IReadOnlyList Clauses { get; init; } = []; + // Stable v0.2 members. public bool IsUnparseable { get; init; } public string? UnparseableReason { get; init; } } ``` -For a successful result, each `SimpleCommandSyntax.Clause`, matching -`CommandOccurrence.Clause`, and matching entry in `Clauses` is reference-equal. -For an unparseable result, `Commands` and `Clauses` are empty even when `Syntax` +For a successful result, each `SimpleCommandSyntax.Clause`, +`CommandOccurrence.Clause`, and matching `Clauses` entry is reference-equal. +Each analyzed argument and redirect similarly references its exact v0.2 leaf. +For an unparseable result, `Commands` and `Clauses` are empty even if `Syntax` contains partial diagnostics. -The new records participate in generated record equality, hashing, and -`ToString()`, and the new `ParsedCommand` members change those generated results. -The library does not define a stable JSON wire format and does not add serializer -attributes or a serialization dependency for the polymorphic syntax family. -Consumers that persist parser results must own a versioned DTO or configure -their serializer explicitly; ordinary in-memory consumers use the typed API. +Adding the v0.3 properties to the stable v0.2 `ParsedCommand` record changes +that record's generated equality, hashing, and `ToString()`; this documented +behavioral difference does not alter the stable v0.2 source or binary surface. +The new records retain generated behavior for ordinary in-memory use. The +library does not define a stable serialized wire +format. Consumers that persist analysis own a versioned DTO and serializer +mapping; persisted data does not justify public constructors or mutable result +bags. -The intended security-consumer shape is therefore: +The intended authorization loop is compact: ```csharp var parsed = parser.Parse(source); @@ -1562,24 +1636,29 @@ if (parsed.IsUnparseable || parsed.Commands.Count == 0) foreach (var occurrence in parsed.Commands) { - if (!occurrence.IsComplete - || occurrence.ImmediateRole == CommandOccurrenceRole.Unknown - || occurrence.Clause.Verb.IsDynamic) + if (!occurrence.IsComplete || occurrence.Clause.Verb.IsDynamic) { return Prompt("command execution is not statically bounded"); } - var interpreted = executableGrammar.InterpretAuthoredShellShape(occurrence); - if (!interpreted.IsComplete) + foreach (var argument in occurrence.Arguments) { - return Prompt("executable arguments are ambiguous"); + EvaluateAuthoredArgument( + argument.Argument, + argument.Element, + argument.Value); } - EvaluateEveryCandidate(interpreted); - EvaluateEveryRedirect(occurrence.Redirects); + foreach (var redirect in occurrence.Redirects) + { + EvaluateRedirectByRuntimeType(redirect); + } } ``` +The parser owns shell grammar and the joins above. The consumer still owns +executable-specific option grammar, protected path zones, hard-deny precedence, +approval identity, and the final allow/prompt/deny decision. ## Appendix B: Locked Grammar Boundaries and Non-Normative Parser Mocks The BNF and support matrices in this appendix lock the v0.3 boundary. The C# @@ -1597,8 +1676,6 @@ statement := simple_command | command_list | group | foreach_loop - | condition_loop - | conditional ``` This vocabulary describes output relationships only. Each shell defines its @@ -1651,7 +1728,7 @@ existing lexer values and spans. | `while` / `until` and `if` / `elif` / `else` command lists | Deferred; whole result unparseable until command discovery and state joins are specified | | Completely delimited `$()` substitution in a supported word, redirect value, iterable, or expanding heredoc body | Inner commands visible; produced value `Unknown` | | Legacy backtick command substitution | Whole result unparseable until its distinct escape and nesting rules are modeled | -| Static path-shaped glob in a supported iterable | `Pattern` only under the locked covering-directory rule | +| Static path-shaped glob in a supported iterable | `PathPattern` only under the locked covering-directory rule | | Existing `<<` / `<<-` heredocs | Supported; preserve delimiter, body, expansion mode, and completeness without treating body data as commands | | Bash `<<<` here strings | Supported with explicit here-string redirect facts | | Process substitution and single-`&` background lists | Independently gated; whole result unparseable until supported | diff --git a/openspec/changes/v0-3-structured-shell-analysis/proposal.md b/openspec/changes/v0-3-structured-shell-analysis/proposal.md index 5c7dd45..3c76209 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/proposal.md +++ b/openspec/changes/v0-3-structured-shell-analysis/proposal.md @@ -27,6 +27,11 @@ fail-closed behavior for incomplete analysis. - Add a library-owned command-occurrence projection containing every command that may execute in supported grammar, including iterator, loop-body, wrapped, substitution, and PowerShell script-block execution-region commands. +- Make the unreleased v0.3 result model parser-owned and invalid-state-resistant. + Each authored argument is returned already joined to its v0.2 `Arg` and + `ClauseElement`; value domains, redirect sources, and redirect operations use + closed typed alternatives instead of public property bags. Prune public + condition/branch vocabulary that stable v0.3 never emits. - Define authorization completeness over authored shell syntax. A complete occurrence proves that the parser discovered and classified the submitted executable region; it does not prove the runtime executable selected by @@ -70,6 +75,10 @@ fail-closed behavior for incomplete analysis. that check `IsUnparseable` continue to fail closed and do not silently miss nested executable commands. Unparseable results expose no command or clause authorization projection. +- Preserve compatibility with stable v0.2, not with any v0.3 prerelease. + Every `0.3.0-alpha*` package was an integration preview; + their new public members may be renamed, removed, or reshaped before stable + `0.3.0`. Netclaw migrates in lockstep to the corrected prerelease. - Expand grammar in vertical slices through Bash `for ... in` and PowerShell `foreach`. Preserve the existing Bash heredoc grammar while adding body, delimiter, and expansion facts, and add @@ -119,12 +128,14 @@ into `SPEC.md` and `SPEC.POWERSHELL.md` before implementation. ## Impact -This is an additive but release-shaped public API change affecting -`ParsedCommand`, new syntax and analysis records, redirect modeling, both +This is an additive change from stable v0.2 and a deliberately breaking +correction from the v0.3 prereleases. It affects `ParsedCommand`, new syntax +and analysis records, redirect modeling, both shell parsers, corpus schemas, public API snapshots, the shared and -PowerShell specifications, and `docs/CONSUMER_GUIDE.md`. Adding properties to -public records also changes generated equality, hashing, `ToString()`, and -default serialization and therefore requires explicit migration notes. +PowerShell specifications, and `docs/CONSUMER_GUIDE.md`. Stable v0.2 types and +members remain source and binary compatible. The corrected v0.3 result types +are library-owned in-memory models, not consumer-constructible DTOs or a stable +serialized wire format. This accepted scope supersedes the earlier assumption that every ordinary PowerShell script-block argument is non-executing. Canonical receivers proved diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md index 5ab129e..80d6e16 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md @@ -305,8 +305,8 @@ reserved syntax. - **THEN** the whole result is unparseable until coprocess structure and timing are modeled ### Requirement: PowerShell approvals prove authored commands, not ambient resolution -`PwshParserOptions.InitialStateMode` SHALL default to `Unknown` and SHALL remain -source and binary compatible. Neither that default nor an ambient alias, +`PwshParserOptions.InitialStateMode` SHALL default to `Unknown`. Neither that +default nor an ambient alias, function, module, profile, executable lookup, or inherited variable SHALL by itself make a static authored command occurrence incomplete. The parser SHALL prove that it discovered the executable syntax submitted by the caller; it @@ -434,10 +434,11 @@ The analysis SHALL classify a policy-relevant shell value as exact, finite, bounded symbolic pattern, or unknown, and SHALL NOT present a weaker proof as a stronger domain. -`Unknown` SHALL contain no values or pattern fields. `Exact` SHALL contain one -value. `FiniteSet` SHALL contain 2–32 distinct values. `Pattern` SHALL contain -no values and SHALL contain a non-empty pattern and covering directory. The -parser SHALL NOT emit any other member combination. +The public domain SHALL be a closed, library-constructed hierarchy. +`Unknown` SHALL carry no payload. `Exact` SHALL contain one value. `FiniteSet` +SHALL contain 2–32 distinct values. `PathPattern` SHALL contain a non-empty +pattern and covering directory. No public property bag or kind enum SHALL let +a consumer construct contradictory member combinations. #### Scenario: One literal value - **WHEN** an eligible isolated-mode loop binds a variable from the single literal `a.txt` @@ -479,16 +480,18 @@ becomes unknown rather than being truncated. ### Requirement: Structural analysis has fixed depth limits The parser SHALL support at most 16 nested executable containers and at most 5 decoded command-string wrapper recursions. Structural depth starts at zero for -the root and increments once when entering a foreach loop, condition loop, -conditional, group, command substitution, or execution region. Blocks, conditional-branch -records, command lists, pipelines, and simple-command leaves do not increment -the depth independently. These bounds SHALL NOT be caller-configurable. +the root and increments once when entering a foreach loop, group, command +substitution, or execution region. Blocks, command lists, pipelines, and +simple-command leaves do not increment the depth independently. Future +container types SHALL define their participation when their grammar is added; +they do not reserve stable-v0.3 public vocabulary. These bounds SHALL NOT be +caller-configurable. Exceeding either bound SHALL make the whole result unparseable rather than returning an authorization projection for a subset. -The limits SHALL be exposed as static get-only properties rather than public -compile-time constants so downstream assemblies read the installed parser's -contract instead of inlining stale values. +The limits SHALL be documented fixed parser contracts but SHALL NOT be exposed +as public tuning options, constants, or properties. Consumers react to the +resulting unknown or unparseable fact rather than reimplementing the bounds. #### Scenario: Structural nesting reaches the limit - **WHEN** a supported input enters exactly 16 nested executable containers diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md index 2b6ea6b..22dce1e 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md @@ -157,13 +157,25 @@ finite shell proof SHALL NOT itself grant authorization. ### Requirement: v0.2 consumers have a documented migration path Release notes and the consumer guide SHALL document the new canonical projections, retained compatibility fields, record equality and serialization -effects, and the period during which `Clauses` remains supported. +effects, and the period during which `Clauses` remains supported. Stable v0.2 +source and binary compatibility SHALL remain the binding baseline. Documented +behavioral differences, including `ParsedCommand` record equality, hashing, +`ToString()`, and explicit oracle-proved classification corrections, SHALL NOT +be mislabeled as behavior-preserving. +No compatibility SHALL be promised between v0.3 prereleases; the migration +guide SHALL map the removed alpha-only members directly to the corrected +stable-v0.3 shapes without shims. #### Scenario: Consumer remains on Clauses during alpha - **WHEN** a consumer upgrades to a v0.3 prerelease without adopting `Commands` - **THEN** existing simple-command behavior remains available - **THEN** supported nested constructs expose conservative authored clauses rather than omitting commands +#### Scenario: Alpha consumer adopts the corrected result model +- **WHEN** a consumer used sparse effective-argument coordinates or redirect/value property bags from an earlier v0.3 alpha +- **THEN** it migrates to joined arguments and closed typed alternatives +- **THEN** ShellSyntaxTree does not retain obsolete aliases or adapters for the prerelease surface + #### Scenario: Netclaw adopts v0.3 analysis - **WHEN** Netclaw migrates to the occurrence and explicit redirect APIs - **THEN** integration tests cover ordinary commands, static fd operations, bounded loops, and unknown-value fallback diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md index ae09da3..52c881f 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md @@ -14,7 +14,7 @@ of whether the command is top-level or nested. - **WHEN** a selected Bash or PowerShell corpus entry is marked for structural verification - **THEN** the corpus records the complete syntax tree and command-occurrence collection in authored order - **THEN** every simple-command node and occurrence references the exact compatibility clause by index and object identity -- **THEN** roles, completeness, ancestry coordinates, and exact-or-null source ranges are compared without weakening legacy corpus entries that omit structural expectations +- **THEN** roles, completeness, joined arguments, ancestor identities, child indices, and exact-or-null node source ranges are compared without weakening legacy corpus entries that omit structural expectations ### Requirement: Occurrences identify structural execution roles Each command occurrence SHALL identify its immediate structural execution role @@ -23,23 +23,21 @@ grouping. `CommandOccurrenceRole.Unknown` and `CommandAncestryRegion.Unknown` SHALL be their enum zero values. Ancestry SHALL be ordered outermost to innermost, -exclude the simple-command leaf, and retain child indices and exact-or-null -source ranges for correlation. +exclude the simple-command leaf, and retain the actual ancestor node plus a +child index. A frame SHALL NOT copy a kind or source range that can disagree +with that node. Each frame SHALL describe the relationship from its ancestor to the next node on the path. The root block SHALL use `Root`; non-root blocks and command lists SHALL use `Statement`; pipelines SHALL use `PipelineStage`; groups SHALL use -`GroupBody`; foreach nodes SHALL use `Iterator` or `LoopBody`; condition loops -SHALL use `Condition` or `LoopBody`; conditionals SHALL use `Branch`; -conditional-branch nodes SHALL use `Condition` or `Branch`; and substitutions -SHALL use `Substitution`; execution regions SHALL use `ExecutionRegion`. -Repeated children SHALL use their zero-based authored -index, with an `else` child indexed after all conditional branches. Frame -source ranges SHALL identify the ancestor. Blocks, command lists, and groups -SHALL retain the incoming immediate role; a nearer pipeline, iterator, body, -condition, branch, substitution, or execution-region relation SHALL replace it. - -#### Scenario: Root and nested block coordinates are deterministic +`GroupBody`; foreach nodes SHALL use `Iterator` or `LoopBody`; substitutions +SHALL use `Substitution`; and execution regions SHALL use `ExecutionRegion`. +Condition loops and branches are not part of stable v0.3. Repeated children +SHALL use their zero-based authored index. Blocks, command lists, and groups +SHALL retain the incoming immediate role; a nearer pipeline, iterator, body, substitution, or +execution-region relation SHALL replace it. + +#### Scenario: Root and nested ancestry is deterministic - **WHEN** a root statement contains a loop-body pipeline - **THEN** a stage occurrence has outer-to-inner `Root`, `LoopBody`, `Statement`, and `PipelineStage` ancestry @@ -50,13 +48,12 @@ The projector SHALL accept only a tree with one syntax-node and one `Clause` reference per authored simple-command position. Node and source-fragment spans SHALL be both unavailable or a non-negative start/length pair. Structural enum values consumed by projection SHALL be known. Empty blocks MAY be valid, but -empty pipelines, command lists, and conditionals SHALL be rejected. +empty pipelines and command lists SHALL be rejected. -Value domains, cwd facts, effective-argument coordinates, redirect -coordinates, redirect shapes, and heredoc facts SHALL satisfy their locked -record invariants before projection succeeds. Any repeated identity, malformed -shape, invalid coordinate, cycle, or depth overflow SHALL discard every -partial command and compatibility result. +Value domains, cwd facts, joined argument identities, redirect alternatives, +and heredoc facts SHALL satisfy their locked invariants before projection +succeeds. Any repeated identity, malformed shape, invalid join, cycle, or +depth overflow SHALL discard every partial command and compatibility result. #### Scenario: Shared leaf identity is not counted twice - **WHEN** an internal parser bug places one syntax leaf or `Clause` reference @@ -66,10 +63,49 @@ partial command and compatibility result. #### Scenario: Complete occurrence cannot contain invalid facts - **WHEN** parser-owned analysis supplies an invalid value domain, argument - coordinate, redirect shape, or unknown scope-affecting structural kind + join, redirect alternative, or unknown scope-affecting structural relation - **THEN** projection fails closed - **THEN** the occurrence is not published with `IsComplete=true` +### Requirement: Occurrences expose fully joined authored arguments +Each command occurrence SHALL expose exactly one `AnalyzedArgument` for every +non-cwd-attribution compatibility `Arg`, in authored order. Each entry SHALL +reference that exact `Arg`, its exact argument-role `ClauseElement`, and one +closed shell-value domain. Static authored values SHALL normally be exact; +bounded visits MAY be finite or path-pattern values; unproved values SHALL be +unknown. Consumers SHALL NOT correlate a sparse coordinate back into +`Clause.Elements`. + +The join SHALL permit multiple compatibility arguments to reference the same +authored element. Inline equals-form native options and PowerShell colon-bound +parameters can split one token into multiple `Arg` records; all split entries +SHALL remain present, ordered, and linked to that one `ClauseElement`. + +#### Scenario: Static and loop-derived arguments share one ingestion path +- **WHEN** an occurrence contains a static option and a bounded loop variable +- **THEN** both appear once in `Arguments` in authored order +- **THEN** each entry directly exposes its compatibility argument, authored element, and effective value + +#### Scenario: Synthetic cwd attribution is not an authored argument +- **WHEN** the v0.2 clause carries a synthetic cwd-attribution `Arg` +- **THEN** it remains available through `Clause.Args` +- **THEN** it is absent from `Arguments` because `WorkingDirectory` is the canonical v0.3 cwd fact + +#### Scenario: Inline option creates a many-to-one authored join +- **WHEN** a supported inline option produces an option `Arg` and an operand `Arg` from one authored token +- **THEN** two analyzed arguments reference their two exact compatibility arguments +- **THEN** both analyzed arguments reference the same exact authored `ClauseElement` +- **THEN** each analyzed value describes its corresponding split `Arg`, not the unsplit element text + +### Requirement: Published collections are not consumer-mutable +Every v0.3 `IReadOnlyList` SHALL use an immutable backing collection or a +defensive copy. The library SHALL NOT expose an array or mutable list that a +consumer can cast and mutate after a successful projection. + +#### Scenario: Caller retains a parser input collection +- **WHEN** internal lowering receives a mutable collection and publishes a v0.3 fact +- **THEN** later mutation of the original collection cannot change the parsed result + #### Scenario: Dynamic Bash command string remains incomplete - **WHEN** Bash parses a `bash` or `sh` clause with dynamic wrapper-control input, decoded or combined command-string options, or an expanding quoted body that does not match the complete literal exactly-one wrapper production - **THEN** the existing outer compatibility leaf remains visible with direct source provenance diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/explicit-redirect-semantics/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/explicit-redirect-semantics/spec.md index d707916..f108e59 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/explicit-redirect-semantics/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/explicit-redirect-semantics/spec.md @@ -2,15 +2,25 @@ ### Requirement: Redirect operation is explicit Each parsed redirect SHALL identify its operation independently from its raw -text and compatibility direction. Supported operations SHALL distinguish file -input, file output, append, descriptor duplicate, descriptor close, -descriptor move, combined output, and any separately supported here-document -or here-string form. - -The redirect source SHALL distinguish the shell-default stream, a numeric -descriptor, and PowerShell's all-streams selector. `Unknown` SHALL be the zero -source kind and operation, and invalid source-kind/descriptor combinations -SHALL be incomplete. +text and compatibility direction through one closed, library-constructed +redirect-analysis hierarchy. Distinct alternatives SHALL represent file, +descriptor duplicate, descriptor close, descriptor move, here-document, +here-string, and unresolved redirects. A file-mode enum SHALL distinguish +input, output, append, combined output, and combined-output append. + +The closed redirect-source hierarchy SHALL distinguish unknown, shell-default, +numeric-descriptor, and PowerShell-all-streams sources. A descriptor source +SHALL always contain one non-negative descriptor. An unresolved source or +operation SHALL use its explicit `Unknown` / `UnresolvedRedirectAnalysis` +alternative and SHALL be incomplete. Consumers SHALL NOT validate mutable +kind/descriptor/operation property combinations. + +Valid pairs SHALL be limited to: unknown source with unresolved redirect; +default source with ordinary file, Bash combined-output, descriptor, heredoc, +or here-string alternatives; numeric source with ordinary file, descriptor, +heredoc, or here-string alternatives; and PowerShell-all-streams source with +output/append file alternatives or descriptor duplication to success stream +descriptor `1`. Every other internal pair SHALL fail projection atomically. #### Scenario: PowerShell all-streams redirect - **WHEN** PowerShell parses `Get-ChildItem *> output.txt` @@ -55,7 +65,9 @@ SHALL be incomplete. ### Requirement: Static and computed descriptor targets differ A descriptor operation SHALL expose a static target descriptor only when the complete target is a literal descriptor. A variable, substitution, malformed -suffix, or other computed target SHALL remain dynamic or incomplete. +suffix, or other computed target SHALL use the unresolved redirect alternative +or make the whole result unparseable; it SHALL NOT create a descriptor variant +with a nullable or sentinel target. #### Scenario: Variable descriptor target - **WHEN** Bash parses `command 2>&$FD` @@ -79,11 +91,11 @@ overwritten or appended. - **WHEN** Bash parses `command &>> output.log` - **THEN** one combined-output append redirect targets `output.log` -### Requirement: Redirect path relevance is independent -Redirect details SHALL state whether the target is a filesystem path without -overloading dynamic-value classification. Static descriptor operations SHALL -not be paths; file targets SHALL retain normal literal, pattern, and unknown -value facts. +### Requirement: Redirect path relevance follows the typed alternative +Every file-redirect alternative SHALL be filesystem-path relevant and SHALL +retain normal exact, finite, path-pattern, or unknown target facts. Descriptor, +heredoc, here-string, and unresolved alternatives SHALL NOT expose a +path-relevance flag whose value can contradict their runtime type. #### Scenario: Static error file - **WHEN** Bash parses `command 2> error.log` diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md index 9c54dea..70a9099 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md @@ -90,13 +90,12 @@ preserves the authored nesting and source order of supported command lists, pipelines, groups, simple commands, substitutions, execution regions, and foreach loops. -The public syntax family SHALL be a closed hierarchy of records derived from -`ShellSyntaxNode`. Every node SHALL expose a `ShellSyntaxKind` discriminant and -zero SHALL mean `Unknown`. The locked family SHALL include block, simple -command, pipeline, command list, group, foreach, condition loop, conditional, -conditional branch, command substitution, and execution-region nodes. -Condition-loop and conditional node kinds are reserved structural vocabulary; -stable v0.3 does not emit them because their grammar remains fail closed. +The public syntax family SHALL be a closed, library-constructed hierarchy of +records derived from `ShellSyntaxNode`. Runtime type SHALL be its only +discriminant. Stable v0.3 SHALL expose only block, simple-command, pipeline, +command-list, group, foreach, command-substitution, and execution-region nodes. +It SHALL NOT publish a redundant kind enum or public condition/branch types +that no stable-v0.3 parser can emit. #### Scenario: Existing flat command receives a structural root - **WHEN** either parser parses `git status && dotnet test` @@ -107,8 +106,8 @@ stable v0.3 does not emit them because their grammar remains fail closed. - **WHEN** a consumer references the public syntax-node base type - **THEN** it cannot derive and inject an external node implementation -#### Scenario: Later node kind is not silently authorized -- **WHEN** a later package returns a derived node or kind an older consumer does not recognize +#### Scenario: Later node type is not silently authorized +- **WHEN** a later package returns a derived node an older consumer does not recognize - **THEN** a display visitor may show an unknown node - **THEN** an authorization visitor fails closed @@ -201,7 +200,7 @@ An `ExecutionRegionSyntax` SHALL represent a completely delimited authored body that may execute because of a direct shell invocation operator or a recognized command argument binding. It SHALL expose an execution origin, phase, timing, cardinality, exact-or-null source range, optional host -`ClauseElement` coordinate, and body. Origin, phase, timing, and cardinality +argument `ClauseElement` reference, and body. Origin, phase, timing, and cardinality SHALL be independent enum facts whose zero values are `Unknown`. Origin SHALL have `Unknown`, `DirectCall`, `DotSource`, and `CommandArgument` @@ -219,10 +218,10 @@ The containing command's `ExecutionRegions` collection SHALL preserve authored script-block order. Semantic phase order MAY differ and SHALL be consumed by the shell-specific analyzer rather than by reordering authored syntax. A direct `& {}` or `. {}` region SHALL appear as a statement, SHALL use `DirectCall` or -`DotSource` respectively, and SHALL have no host element coordinate. A region +`DotSource` respectively, and SHALL have a null host argument. A region bound to a command argument SHALL be attached to that `SimpleCommandSyntax` and SHALL identify the exact script-block -`ClauseElement` when the binding is proved. +`ClauseElement` by reference when the binding is proved. #### Scenario: Reordered pipeline phases retain both orders - **WHEN** PowerShell parses `1 | ForEach-Object -End { Write-Output end } -Begin { Write-Output begin } -Process { Write-Output $_ }` @@ -301,6 +300,9 @@ binding source, the raw iterable source fragment, commands discovered in the iterator, and the body. It SHALL NOT normalize Bash words and PowerShell expressions into a false shared expression grammar. +The binding name and source SHALL be direct `ForEachSyntax` properties rather +than a separately constructible one-use binding record. + #### Scenario: Iterator with no exact outer span - **WHEN** a loop is lifted from decoded wrapper content without an exact mapping to the outer source - **THEN** the iterable raw text remains available diff --git a/openspec/changes/v0-3-structured-shell-analysis/tasks.md b/openspec/changes/v0-3-structured-shell-analysis/tasks.md index 9fea6dd..67bad72 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/tasks.md +++ b/openspec/changes/v0-3-structured-shell-analysis/tasks.md @@ -29,6 +29,22 @@ `PwshDialect` contract. Preserve PowerShell 7 as the compatibility default, make Windows PowerShell 5.1 explicit, keep Bash and PowerShell top-level parsing separate, and require dialect-local grammar/catalog proof. +- [ ] 1.14 Correct the unreleased v0.3 consumer API before stable release. + Preserve every stable v0.2 type and the `Syntax` / `Commands` / `Clauses` + lanes, but do not preserve alpha-only v0.3 members. Replace sparse effective- + argument coordinates with one joined analyzed argument per authored non-cwd + argument; make value domains, redirect sources, and redirect operations + closed parser-owned alternatives; reference actual syntax ancestors; remove + redundant kind/source copies and public condition/branch types that stable + v0.3 never emits; keep executable policy consumer-owned. +- [ ] 1.15 Synchronize the corrected API into `SPEC.md`, implementation, + public-API snapshots, corpus DTOs, README, and consumer guide in one vertical + slice. Add prerelease migration notes for alpha consumers without an adapter + or obsolete compatibility layer. Defensively copy or immutably back every + published `IReadOnlyList` and pin many-to-one inline-option argument joins. +- [ ] 1.16 Migrate Netclaw from alpha.5 to the corrected prerelease, delete its + coordinate/property-bag validation, and rerun the full Linux plus native- + Windows approval matrices before stable v0.3. ## 2. Resolver Provenance Correction and Shared Preparation @@ -41,8 +57,8 @@ ## 3. Structural and Projection Skeleton -- [x] 3.1 Add the locked public syntax-node hierarchy and defaults to the public API snapshot. -- [x] 3.2 Add the locked command-occurrence, role, ancestry, completeness, and analysis records to the public API snapshot. +- [x] 3.1 Add the alpha public syntax-node hierarchy and defaults to the public API snapshot. Superseded for stable v0.3 by tasks 1.14-1.15. +- [x] 3.2 Add the alpha command-occurrence, role, ancestry, completeness, and analysis records to the public API snapshot. Superseded for stable v0.3 by tasks 1.14-1.15. - [x] 3.3 Add `ParsedCommand.Syntax` and `ParsedCommand.Commands` while retaining all v0.2 members. - [x] 3.4 Build a library-owned traversal that emits each simple command occurrence exactly once in deterministic source order. - [x] 3.5 Build the conservative `Clauses` compatibility flattener without inventing cross-structure compound operators. @@ -332,17 +348,20 @@ ## 11. Verification and Release -- [x] 11.1 Add public API default-value, equality, serialization, and unknown-enum compatibility tests. - - `V03PublicApiSnapshotTests` pins every additive record default and enum +- [ ] 11.1 Replace the alpha public-API default-value, equality, serialization, + immutability, and unknown-case tests with the corrected stable-v0.3 contract. + - Alpha evidence: `V03PublicApiSnapshotTests` currently pins every additive record default and enum zero value, proves `Syntax` and `Commands` participate in generated record equality and `ToString()` plus equal-record hash consistency, demonstrates that default JSON is not a polymorphic round-trip contract, and makes every policy-sensitive unknown numeric enum value detectable for consumer rejection. - [ ] 11.2 Assert every supported executable region appears exactly once and every unsupported executable region makes the result unparseable. -- [x] 11.3 Run the complete Bash and PowerShell corpus suites plus the PII audit. -- [x] 11.4 Run `dotnet build -c Release`, `dotnet test -c Release`, `dotnet pack -c Release`, and header verification. -- [x] 11.5 Validate the public API field-for-field against the synchronized shared and PowerShell specifications. - - `PublicApiSnapshotTests` and `V03PublicApiSnapshotTests` enumerate the exact +- [ ] 11.3 Run the complete Bash and PowerShell corpus suites plus the PII audit against the corrected implementation. + - Alpha evidence: the complete suites and PII audit pass on the superseded alpha surface. +- [ ] 11.4 Run `dotnet build -c Release`, `dotnet test -c Release`, `dotnet pack -c Release`, and header verification against the corrected implementation. + - Alpha evidence: these gates pass on the superseded alpha implementation. +- [ ] 11.5 Validate the corrected public API field-for-field against the synchronized shared and PowerShell specifications. + - Alpha evidence: `PublicApiSnapshotTests` and `V03PublicApiSnapshotTests` currently enumerate the exact exported namespace, type family, exact property sets, parser constructors and entry points, enum ordering, reference nullability, defaults, and fixed limits synchronized into `SPEC.md` and `SPEC.POWERSHELL.md`.