test: add v0.3 shell analysis design corpus

Author: Aaronontheweb

IMPLEMENTATION_PLAN.md

@@ -133,6 +133,14 @@ priorities.
       as the v0.3 roadmap and cross-link issue #71 control flow and issue #69
       shared native argument-fragment classification without merging their
       scopes.
+- [x] Add a versioned pre-implementation design corpus with 15 Bash and 15
+      PowerShell representative, boundary, and adversarial cases. The validator
+      rejects schema drift, checks references and command ordering, confirms
+      every `current` expectation against the v0.2 parsers, and includes the
+      files in the PII audit. Corpus review established that command role must
+      be immediate while ancestry remains compositional, occurrence
+      completeness is independent of value precision, and PowerShell authored
+      parameter classification must remain distinct from effective values.
 - [ ] Complete OpenSpec task group 1: lock the additive public type names,
       compatibility projection, fixed analysis bounds, and supported-construct
       matrix before production implementation.

TOOLING.md

@@ -30,6 +30,13 @@ directory, routes each entry to the matching parser (`bash/` → `BashParser`,
 gate — it feeds every PowerShell corpus input to real `pwsh` and enforces
 the oracle matrix.
 
+`tests/ShellSyntaxTree.Tests/DesignCorpus/v0.3/` is a separate,
+pre-implementation contract corpus. Its focused validator rejects unknown JSON
+members, checks syntax/occurrence references and security invariants, and
+compares each recorded `current` result with the real v0.2 parser. Design cases
+move into the executable `Corpus/<shell>/` only when the corresponding v0.3 API
+and parser slice exists. The PII audit scans both corpus trees.
+
 ### PwshCorpusTool
 
 `tools/PwshCorpusTool` is the PowerShell corpus authoring aid

openspec/changes/v0-3-structured-shell-analysis/design.md

@@ -123,16 +123,23 @@ than inventing offsets into escaped or encoded outer text.
 
 `Commands` contains one entry per authored simple command that may execute,
 not one entry per predicted runtime iteration. Each occurrence carries its
-`Clause`, structural role, ancestry suitable for diagnostics, and an explicit
-completeness fact. Roles include at least ordinary, pipeline stage, condition,
-iterator, loop body, branch, and substitution; the final names are locked with
-the public API review.
+`Clause`, immediate structural role, compositional ancestry suitable for
+analysis and diagnostics, and an explicit completeness fact. Immediate roles
+include at least ordinary, pipeline stage, condition, iterator, loop body,
+branch, and substitution; ancestry frames retain every outer role, such as a
+pipeline stage nested inside a loop body. The final names are locked with the
+public API review.
 
 Condition and iterator commands are never omitted. Mutually exclusive branch
 commands all appear because the collection is a may-execute set. Runtime loop
 counts do not duplicate occurrences; bounded variable domains describe the
 possible effective values at the occurrence.
 
+Completeness and value precision are independent. A structurally complete
+occurrence may conservatively contain an `Unknown` value domain when the
+command and its ancestry are fully discovered but a runtime value cannot be
+proved.
+
 If any executable region cannot be discovered completely, the containing
 `ParsedCommand` remains `IsUnparseable=true`. Partial syntax and occurrences
 may be returned for diagnostics but MUST NOT be used to authorize execution.
@@ -171,10 +178,13 @@ shell rules prove the resulting argument boundary. Unquoted Bash expansion,
 PowerShell object-valued pipelines, indirect expansion, mutation, and
 cross-product explosion remain unknown until separately specified.
 
-Effective values are shell facts, not executable semantics. A consumer must
-re-run its executable-aware option grammar for every exact or finite candidate;
-for example, a loop value beginning with `-` may inject an option even if the
-authored `$variable` token was not option-shaped.
+Effective values are shell facts, not executable semantics. The analysis must
+preserve both the authored shell classification and each proved effective
+value. PowerShell does not retroactively turn a string value such as `-Force`
+into a cmdlet parameter token, while the same value passed to a native
+executable may participate in that executable's option grammar. A consumer
+must therefore apply the relevant shell binding rules and re-run its complete
+executable-aware grammar for every exact or finite candidate.
 
 ### Join state rather than selecting a path
 
@@ -393,7 +403,7 @@ remain shell-specific regardless of the eventual public choice.
 public sealed record CommandOccurrence
 {
     public Clause Clause { get; init; } = new();
-    public CommandOccurrenceRole Role { get; init; }
+    public CommandOccurrenceRole ImmediateRole { get; init; }
     public IReadOnlyList<CommandAncestryFrame> Ancestry { get; init; } = [];
     public IReadOnlyList<EffectiveArgument> EffectiveArguments { get; init; } = [];
     public ShellValueDomain WorkingDirectory { get; init; } = ShellValueDomain.Unknown;
@@ -444,6 +454,8 @@ the compatibility leaf and provides a place for one authored token to have
 multiple possible effective values. Contract review must still account for
 redirect operands, inline option bindings, shell expansions that create more
 than one argument, and occurrences that do not have an exact outer source span.
+An `Unknown` value at one of those coordinates does not by itself make the
+occurrence structurally incomplete.
 
 ### Explicit redirect facts
 

openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md

@@ -48,10 +48,11 @@ shell's quoting and expansion rules prove the resulting argument boundaries.
 - **WHEN** a PowerShell `foreach` variable may hold objects emitted by a pipeline
 - **THEN** its effective string or path value is unknown
 
-### Requirement: Executable semantics are reapplied after substitution
-ShellSyntaxTree SHALL preserve effective candidate values without claiming
-whether they are options, operands, subcommands, revisions, or paths for a
-particular executable. Consumers SHALL interpret every candidate through a
+### Requirement: Shell and executable semantics are reapplied after substitution
+ShellSyntaxTree SHALL preserve the authored shell classification together with
+effective candidate values without claiming whether native-command candidates
+are options, operands, subcommands, revisions, or paths. Consumers SHALL apply
+the shell's binding rules and interpret every native candidate through a
 complete executable-aware grammar before reusing authorization.
 
 #### Scenario: Finite value injects an rm option
@@ -64,6 +65,16 @@ complete executable-aware grammar before reusing authorization.
 - **THEN** the authored `--` remains visible before the effective candidate
 - **THEN** the consumer may account for it using rm semantics
 
+#### Scenario: PowerShell cmdlet parameter-like value
+- **WHEN** PowerShell parses `foreach ($value in '-Force') { Write-Output $value }`
+- **THEN** the authored variable argument remains a positional expression
+- **THEN** the effective string `-Force` is not retroactively classified as a cmdlet parameter token
+
+#### Scenario: PowerShell native option-like value
+- **WHEN** PowerShell parses `foreach ($value in '--force') { git clean $value }`
+- **THEN** the authored variable argument remains distinct from its effective value
+- **THEN** the consumer applies the native executable grammar to `--force`
+
 ### Requirement: Control-flow state joins conservatively
 Working-directory and supported variable state SHALL be propagated through
 sequential regions and joined across branches and loop exits. Disagreement

openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md

@@ -16,8 +16,9 @@ of whether the command is top-level or nested.
 - **THEN** its possible effective values are represented by analysis facts rather than three duplicated occurrences
 
 ### Requirement: Occurrences identify structural execution roles
-Each command occurrence SHALL identify the structural role by which it may
-execute and SHALL retain enough ancestry for diagnostics and UI grouping.
+Each command occurrence SHALL identify its immediate structural execution role
+and SHALL retain compositional ancestry for analysis, diagnostics, and UI
+grouping.
 
 #### Scenario: While condition and body roles
 - **WHEN** Bash parses `while curl URL; do sleep 1; done`
@@ -29,6 +30,11 @@ execute and SHALL retain enough ancestry for diagnostics and UI grouping.
 - **THEN** `Get-ChildItem` is identified as an iterator occurrence
 - **THEN** `Remove-Item` is identified as a loop-body occurrence
 
+#### Scenario: Pipeline stage nested in a loop body
+- **WHEN** Bash parses `for f in a b; do printf '%s\n' "$f" | sort; done`
+- **THEN** `printf` and `sort` have the immediate role pipeline stage
+- **THEN** their ancestry also identifies the enclosing loop body
+
 ### Requirement: Iterator and substitution commands remain visible
 The parser SHALL include every inner command from a supported executable
 iterator, substitution, or nested command in the occurrence collection even
@@ -58,6 +64,11 @@ be sufficient authorization evidence.
 - **THEN** the occurrence is structurally complete
 - **THEN** completeness does not imply that `echo` is authorized
 
+#### Scenario: Complete occurrence with unknown value
+- **WHEN** PowerShell parses `foreach ($item in Get-ChildItem) { Write-Output $item }`
+- **THEN** the `Write-Output` occurrence may be structurally complete
+- **THEN** its effective `$item` value remains unknown because pipeline objects are not evaluated
+
 ### Requirement: Source order is deterministic
 The occurrence collection SHALL be ordered by authored command occurrence,
 including commands nested in headers and bodies, with a documented tie-breaker

openspec/changes/v0-3-structured-shell-analysis/specs/explicit-redirect-semantics/spec.md

@@ -84,6 +84,11 @@ executable may interpret that data as code.
 - **THEN** the body is preserved as non-expanding authored data
 - **THEN** the parser does not execute or reinterpret the receiving command
 
+#### Scenario: Quoted delimiter adjacent to the operator
+- **WHEN** Bash parses `cat <<'EOF'` followed by a body and the `EOF` delimiter
+- **THEN** the quoted delimiter is recognized without requiring whitespace after `<<`
+- **THEN** the result is not reported as missing a delimiter
+
 #### Scenario: Executable substitution in an expanding body
 - **WHEN** a supported expanding heredoc body contains command substitution
 - **THEN** the inner command is exposed or the result is unparseable

openspec/changes/v0-3-structured-shell-analysis/tasks.md

@@ -8,6 +8,8 @@
 - [ ] 1.6 Synchronize the accepted public API and shared requirements into `SPEC.md`.
 - [ ] 1.7 Synchronize PowerShell grammar and analysis deltas into `SPEC.POWERSHELL.md`.
 - [ ] 1.8 Update `PROJECT_CONTEXT.md` and `IMPLEMENTATION_PLAN.md` with the accepted v0.3 scope and delivery slices.
+- [x] 1.9 Add a paired Bash and PowerShell design corpus that records current behavior, desired structure, command occurrences, bounded values, redirect facts, compatibility projections, and security invariants.
+- [ ] 1.10 Promote each design case into the executable corpus as its production parser slice lands.
 
 ## 2. Behavior-Preserving Shared Preparation
 
@@ -90,7 +92,7 @@
 
 ## 10. Separately Gated Syntax Concerns
 
-- [ ] 10.1 Specify heredoc delimiter quoting, expansion mode, body provenance, substitutions, and completeness before enabling heredoc bodies.
+- [ ] 10.1 Specify heredoc delimiter adjacency and quoting, expansion mode, body provenance, substitutions, and completeness before enabling heredoc bodies.
 - [ ] 10.2 Specify process-substitution command discovery and the unknown produced descriptor/path value before enabling it.
 - [ ] 10.3 Specify background-list concurrency, ordering, and shell-state boundaries before enabling single `&`.
 - [ ] 10.4 Specify C-style loop and arithmetic hidden-execution behavior before enabling either construct.

tests/ShellSyntaxTree.Tests/Corpus/PiiAuditTests.cs

@@ -16,10 +16,9 @@ namespace ShellSyntaxTree.Tests.Corpus;
 
 /// <summary>
 /// PII audit gate per SPEC §14 / SPEC.POWERSHELL.md §14. Scans every JSON
-/// corpus entry under every <c>tests/ShellSyntaxTree.Tests/Corpus/&lt;shell&gt;/</c>
-/// directory for the forbidden patterns listed in the sanitization table.
-/// The audit reads from the build-output copy of the corpus (the same
-/// location the runner pulls from) so CI runs against the same bytes a
+/// entry under the executable corpus and the pre-implementation design corpus
+/// for the forbidden patterns listed in the sanitization table. The audit
+/// reads from the build-output copies so CI runs against the same bytes a
 /// developer's local <c>dotnet test</c> would.
 /// </summary>
 /// <remarks>
@@ -101,35 +100,29 @@ public class PiiAuditTests
     [Fact]
     public void Corpus_contains_no_pii_per_spec_section_14()
     {
-        var root = Path.Combine(AppContext.BaseDirectory, "Corpus");
-        if (!Directory.Exists(root))
-        {
-            // The audit is vacuous when there's no corpus to audit; the
-            // separate CorpusRunnerTests asserts the corpus is present.
-            return;
-        }
-
         var hits = new List<string>();
-        foreach (var shellDir in Directory.GetDirectories(root).OrderBy(d => d))
+        var roots = new[] { "Corpus", "DesignCorpus" };
+        foreach (var relativeRoot in roots)
         {
-            var shell = Path.GetFileName(shellDir);
-            foreach (var file in Directory.GetFiles(shellDir, "*.json").OrderBy(f => f))
+            var root = Path.Combine(AppContext.BaseDirectory, relativeRoot);
+            if (!Directory.Exists(root))
+            {
+                continue;
+            }
+
+            foreach (var file in Directory.GetFiles(
+                root, "*.json", SearchOption.AllDirectories).OrderBy(f => f))
             {
-                var name = $"{shell}/{Path.GetFileName(file)}";
-                JsonDocument doc;
+                var name = Path.GetRelativePath(AppContext.BaseDirectory, file)
+                    .Replace('\\', '/');
                 try
                 {
-                    doc = JsonDocument.Parse(File.ReadAllText(file));
+                    using var doc = JsonDocument.Parse(File.ReadAllText(file));
+                    Walk(doc.RootElement, name, fieldPath: string.Empty, hits);
                 }
                 catch (JsonException ex)
                 {
                     hits.Add($"{name}: failed to parse JSON for PII audit: {ex.Message}");
-                    continue;
-                }
-
-                using (doc)
-                {
-                    Walk(doc.RootElement, name, fieldPath: string.Empty, hits);
                 }
             }
         }

tests/ShellSyntaxTree.Tests/DesignCorpus/README.md

@@ -0,0 +1,31 @@
+# v0.3 Design Corpus
+
+This directory is a pre-implementation contract corpus for the
+`v0-3-structured-shell-analysis` OpenSpec change. It does not feed the shipped
+v0.2 AST runner because that schema cannot represent nested syntax, command
+occurrences, effective values, or joined state.
+
+Each case records these independent views:
+
+1. `current`: the behavior the released flat parser produces today;
+2. `syntax`: the desired authored structural nodes and their parent slots;
+3. `commands`: every authored command that may execute, exactly once;
+4. `effectiveValues`: bounded shell facts without executable policy claims;
+5. `securityInvariants`: properties every future implementation and consumer
+   must preserve.
+
+The accompanying tests deserialize with unknown-member rejection, validate
+node and command references, compare `current` with the real v0.2 parser, and
+include these JSON files in the repository PII audit. A case moves into
+`Corpus/<shell>/` only after the corresponding public API and parser behavior
+exist and its full expected AST can pass the normal corpus runner.
+
+The design corpus is intentionally shell-specific. Similar Bash and PowerShell
+cases may share structural expectations while retaining different quoting,
+expression, option-binding, object, and scope semantics.
+
+Each command records one `immediateRole` plus its full `ancestry`. These are
+deliberately separate: a pipeline stage nested inside a loop body is immediately
+a pipeline stage and still carries the enclosing loop-body context. Likewise,
+`isComplete` describes command discovery and structure, not value precision; a
+complete occurrence may contain an `Unknown` effective value.