From cd44a21a815750da8b58070809b582d17a9723d0 Mon Sep 17 00:00:00 2001 From: Aaron Stannard Date: Sat, 8 Aug 2026 13:07:37 +0000 Subject: [PATCH] Prune stable v0.3 scope --- IMPLEMENTATION_PLAN.md | 39 ++++---- PROJECT_CONTEXT.md | 23 +++-- SPEC.POWERSHELL.md | 69 ++++++-------- SPEC.md | 45 ++++----- .../v0-3-structured-shell-analysis/design.md | 94 +++++++------------ .../proposal.md | 32 +++++-- .../specs/bounded-shell-analysis/spec.md | 42 +++------ .../specs/consumer-compatibility/spec.md | 13 +-- .../executable-command-projection/spec.md | 20 ---- .../specs/structured-shell-syntax/spec.md | 26 +---- .../v0-3-structured-shell-analysis/tasks.md | 83 ++++++++-------- 11 files changed, 207 insertions(+), 279 deletions(-) diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md index 28411b7..f9ce134 100644 --- a/IMPLEMENTATION_PLAN.md +++ b/IMPLEMENTATION_PLAN.md @@ -378,8 +378,8 @@ priorities. aliases, supported module qualification, exact and abbreviated/inline parameters, positional slots, parameter sets, `ScriptBlock[]`, authored ForEach-Object multi-block coordinates, and semantic Begin/Process/End - phases. The optional Microsoft.PowerShell.ThreadJob 2.2.0 entry remains - incomplete unless its separate module-baseline proof is supplied. Local + phases. The optional Microsoft.PowerShell.ThreadJob entry remains an + unknown incomplete receiver and no longer gates stable v0.3. Local `Invoke-Command -AsJob`, ambiguous prefixes, malformed value binding, unproved identities, and unknown receivers retain unknown/incomplete facts. Supported catalog-owned module qualifications now pass structural @@ -448,27 +448,21 @@ priorities. authored `Set-Alias Env:...` invocation. The generator-owned executable corpus now supports per-entry initial- state mode and includes the promoted Parallel and remote/session cases. - Pinned child runspace jobs remain before deferred - breakpoint/event/completion actions; then unknown - receiver and nested/adversarial matrices. Preserve script blocks proved + Stable v0.3 stops at the delivered Start-Job, Parallel, and remote/session + boundaries. Optional-module Start-ThreadJob and exact deferred + breakpoint/event/completion actions are post-v0.3 catalog work; unknown + receivers continue to expose incomplete bodies. Preserve script blocks proved to be data as opaque values, expose ambiguous bodies with incomplete facts, and fail atomically when any potentially executable interior is unsupported. Local PowerShell 7.6.4 probes pin variable-versus-location independence, semantic phase order, child process/runspace boundaries, - registration-versus-trigger timing, and the fact that the in-process - `Invoke-Command` parameter set does not support `-AsJob`. -- [ ] Deliver Bash `for ... in` and PowerShell `foreach` as the first two - language-specific vertical slices, then extract only the shared analysis - proven by both implementations. + boundaries and the fact that the in-process `Invoke-Command` parameter + set does not support `-AsJob`. +- [ ] Complete the stable-v0.3 Bash `for ... in` and PowerShell `foreach` + vertical slices without gating release on a shared-analysis refactor. - [ ] Build on the delivered bounded Bash heredoc grammar and quoted-delimiter adjacency by exposing public body/delimiter/expansion/completeness facts, then add a separately tested Bash `<<<` here-string redirect slice. -- [ ] Near the end of v0.3 delivery, expand the Web sample with curated complex - Bash and PowerShell inputs and deterministic Mermaid views of syntax, - occurrences, compatibility clauses, ancestry, redirects, and fail-closed - outcomes. Keep visualization downstream of the canonical projection so - it cannot become a second command-discovery implementation; snapshot the - rendering, escape arbitrary shell labels, and emit no raw HTML. --- @@ -498,4 +492,15 @@ priorities. ## Parked -*(empty; move items here when scope changes rather than deleting them)* +- After stable v0.3, consider shared Bash/PowerShell analysis extraction only + where the two delivered implementations prove identical behavior. +- Add Bash and PowerShell condition loops and branches in shell-specific + vertical slices; keep `case`, `switch`, process substitution, background + lists, and arithmetic independently gated. +- Add or expand exact optional-module Start-ThreadJob and deferred breakpoint, + event, and argument-completion receiver semantics only when consumer demand + justifies a pinned contract; existing conservative recognition may remain. +- Expand the Web sample with curated Bash and PowerShell inputs and + snapshot-tested deterministic Mermaid views produced from canonical + projections. This remains part of the broader product follow-up, not the + stable-v0.3 package or Netclaw migration gate. diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index 968fe68..7741982 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -80,23 +80,28 @@ zero-native-deps .NET parser sized to what security gates actually need. - Add a closed, strongly typed syntax-node hierarchy while retaining existing `Clause` leaves. - Add a library-owned command-occurrence projection for security consumers so - every potentially executable iterator, condition, branch, substitution, and - body command is evaluated exactly once. + every potentially executable iterator, substitution, execution-region, and + body command in the supported grammar is evaluated exactly once. - Add typed PowerShell execution regions for direct call/dot-source blocks, - synchronous callbacks, jobs/parallel runspaces, initialization, and deferred - actions. Public origin/phase/timing/cardinality facts remain separate from + synchronous callbacks, jobs/parallel runspaces, and initialization. Public + origin/phase/timing/cardinality facts remain separate from shell-specific variable, location, command-resolution, runspace, and process state analysis; proved script-block data stays opaque and unknown receivers conservatively expose incomplete bodies. - Add fixed, non-executing value and state analysis: at most 32 candidates, at most 16 structural container levels, and the existing wrapper depth of 5. -- Deliver Bash `for ... in` and PowerShell `foreach` first, then the locked - `while` and `if` subsets independently for each shell. Shared lowering and - analysis are extracted only after both front ends prove identical behavior. +- Deliver Bash `for ... in` and PowerShell `foreach` in stable v0.3. Condition + loops, branches, and shared-analysis extraction are post-v0.3 work and do + not gate the validating consumer migration. - Preserve existing Bash heredocs and add explicit body/expansion facts plus Bash `<<<` here strings. Keep process substitution, background lists, Bash - `case`, PowerShell `switch`, arithmetic/C-style loops, and definitions - independently gated. + condition loops and branches, `case`, PowerShell condition loops and + branches, arithmetic/C-style loops, and definitions independently gated. +- Keep optional-module `Start-ThreadJob` and exact deferred breakpoint, event, + and argument-completion receiver semantics outside the stable-v0.3 catalog. + Existing conservative recognition may remain; unproved receivers still + expose completely delimited bodies as incomplete regions and therefore + remain fail closed. - Treat `openspec/changes/v0-3-structured-shell-analysis/` and its paired design corpus as the review authority until the accepted contract is synchronized into `SPEC.md` and `SPEC.POWERSHELL.md` with the production API change. diff --git a/SPEC.POWERSHELL.md b/SPEC.POWERSHELL.md index 45b2220..730a707 100644 --- a/SPEC.POWERSHELL.md +++ b/SPEC.POWERSHELL.md @@ -1,8 +1,8 @@ # ShellSyntaxTree — PowerShell Specification (through v0.3) **Status:** v0.2.0 shipped; the accepted v0.3 contract adds bounded PowerShell -`foreach`, `while`, and `if` structure plus shared command-occurrence and -explicit redirect analysis. +`foreach` structure plus shared command-occurrence and explicit redirect +analysis. **Audience:** Whoever (human or agent) implements, consumes, or maintains the ShellSyntaxTree PowerShell parser. **Read `SPEC.md` (the bash and shared-contract specification) end-to-end @@ -17,7 +17,7 @@ It is **not** a PowerShell interpreter. It does not execute, expand, or evaluate commands. It returns the same structured AST a consumer already walks for bash. The parsing scope is **Pipeline-aware** (§4): linear command pipelines parse; stable v0.3 also supports only the explicitly bounded -`foreach`, `while`, and `if` subsets below. Other script-level constructs mark +`foreach` subset below. Other script-level constructs mark `IsUnparseable`. `SPEC.md` is the canonical home of the shared public API, AST, sanitization @@ -74,13 +74,13 @@ syntax (§5) are all PowerShell 7 semantics. The `pwsh` validation oracle ### v0.3 extension -Stable v0.3 adds structured projection for bounded `foreach`, `while`, and -`if` statements; exposes iterator, condition, branch, and body commands -exactly once; derives exact or finite string values only from proved literal +Stable v0.3 adds structured projection for bounded `foreach` statements; +exposes iterator and body commands exactly once; derives exact or finite string +values only from proved literal iterables; and joins location and supported binding state conservatively. Pipeline-produced objects and unsupported expressions remain unknown without -execution. `do`, `switch`, definitions, and arbitrary script evaluation stay -outside the supported grammar. +execution. `while`, `if`, `elseif`, `else`, `do`, `switch`, definitions, and +arbitrary script evaluation stay outside the supported grammar. --- @@ -389,8 +389,6 @@ terminators. ```text pwsh_script(stop) := pwsh_statement (statement_terminator pwsh_statement)* pwsh_statement := pwsh_foreach - | pwsh_while - | pwsh_if | pwsh_and_or statement_terminator := ";" | NEWLINE @@ -401,14 +399,6 @@ pwsh_pipeline := pipeline_element ("|" pipeline_element)* pwsh_foreach := "foreach" "(" variable "in" foreach_expression ")" script_block_body -pwsh_while := "while" "(" condition_pipeline ")" - script_block_body - -pwsh_if := "if" "(" condition_pipeline ")" script_block_body - pwsh_elseif* pwsh_else? -pwsh_elseif := "elseif" "(" condition_pipeline ")" script_block_body -pwsh_else := "else" script_block_body - foreach_expression := literal_value | literal_array | pipeline_expression @@ -537,8 +527,6 @@ The version-pinned PowerShell 7 catalog covers: | `Start-Job -InitializationScript` | Initialization | Concurrent | Once | child process before Main | | `Start-Job -ScriptBlock` | Main | Concurrent | Once | child process; exit isolated | | `New-Module -ScriptBlock` | Initialization | Synchronous | Once | module state; current-runspace effects analyzed separately | -| `Set-PSBreakpoint -Action`, event `-Action` | Action | Deferred | ZeroOrMore | trigger-time state Unknown without proof | -| `Register-ArgumentCompleter -ScriptBlock` | Completion | Deferred | ZeroOrMore | completion-time state Unknown without proof | Remote `Invoke-Command` bodies begin with Unknown working directory, bindings, aliases, functions, modules, profiles, and command resolution. Local parser @@ -570,10 +558,12 @@ the same conservative invalidation unless its identity is independently proved. An ambiguous changed name, wildcard, or candidate-set overflow collapses to all unproved command names rather than guessing. -The optional inbox `Microsoft.PowerShell.ThreadJob\Start-ThreadJob` follows -the initialization/main child-runspace model only when the caller's pinned -module baseline proves that identity. Otherwise it follows the unknown -receiver rule. +Optional-module `Start-ThreadJob` and deferred breakpoint, event, and argument- +completion receivers are not stable-v0.3 catalog-completeness promises. +Existing conservative recognition may remain, but additional module/version or +trigger-time proof does not gate the release. Every unproved form follows the +unknown-receiver rule: completely parsed bodies remain visible with incomplete +execution and state facts. Aliases, supported module-qualified spellings, static call-operator spellings, parameter abbreviations and inline values, positional binding, parameter-set @@ -585,9 +575,11 @@ runtime order. `ExecutionRegionTiming` and `ExecutionRegionCardinality` are not scope facts. Variable, location, command-resolution, runspace, and process propagation are -analyzed independently. Deferred actions remain authorization-visible at -registration, but relative paths use trigger-time Unknown cwd unless another -proof exists. A constrained canonical `Write-Output { Remove-Item x }` +analyzed independently. Cataloged receivers may retain already-proved +scheduling facts without making further catalog expansion release-gating. +Unproved receiver, binding, or state facts remain Unknown rather than borrowing +registration-time state. A +constrained canonical `Write-Output { Remove-Item x }` remains opaque data and does not invent a `Remove-Item` occurrence. A leading `param(...)` declaration inside any execution region remains outside @@ -595,16 +587,9 @@ the stable-v0.3 body grammar and makes the whole parse unparseable. This deliberately limits realistic argument-completer and directly invoked blocks until parameter declaration and block-argument binding are modeled together. -`condition_pipeline` is limited to a pipeline the existing parser can delimit -completely. A subexpression, member invocation, script block, or other form -that may execute outside complete command discovery makes the whole result -unparseable. Missing delimiters and every unsupported executable region also -produce empty `Commands` and `Clauses`; partial `Syntax` is diagnostic only. - PowerShell scope and location state remain shell-specific. Grouping `( ... )` -does not isolate location. Branch exits retain an exact cwd only when every -supported alternative agrees; disagreement becomes `Unknown`. Loop exits -include the zero-iteration state. The parser does not publish a finite cwd set. +does not isolate location. Foreach exits include the zero-iteration state. The +parser does not publish a finite cwd set. `foreach` assignments use a case-insensitive persistent binding map rather than lexical push/pop restoration. A proved nonempty ordered iterable leaves its @@ -635,9 +620,9 @@ clear those resolutions and retain the `` marker. Decoded child hosts retain inherited invocation-cwd attribution on their compatibility leaves while isolating child exit state. -Stable v0.3 continues to defer `do`, `switch`, functions, definitions, -class/type bodies, and arbitrary execution-bearing expressions outside the -bounded forms above. +Stable v0.3 continues to defer `while`, `if`, `elseif`, `else`, `do`, `switch`, +functions, definitions, class/type bodies, and arbitrary execution-bearing +expressions outside the bounded forms above. --- @@ -1452,7 +1437,7 @@ in **`SPEC.md` §11**. here-string, unterminated `<# ... #>` block comment, unbalanced `{ }` / `$( )` / `@( )` / `@{ }`, unbalanced grouping `( )`. 2. **Unsupported control-flow at statement/verb position** — `switch`, `for`, - `do`, `until`, or an `if`, `elseif`, `else`, `foreach`, or `while` form + `do`, `until`, `if`, `elseif`, `else`, or `while`, plus any `foreach` form outside the bounded stable-v0.3 grammar in §4. 3. **Definition keywords** — `function`, `filter`, `workflow`, `configuration`, `class`, `enum`. @@ -1815,8 +1800,8 @@ v0.2.0 ships when **all** of these hold: ## 18. Out of Scope -- PowerShell control flow outside stable v0.3's bounded `foreach`, `while`, - and `if` subsets — including `do` and `switch`. +- PowerShell control flow outside stable v0.3's bounded `foreach` subset, + including `while`, `if`, `elseif`, `else`, `do`, and `switch`. - `function`/`filter`/`class`/`enum` definitions, `param()`/`begin`/`process`/`end` blocks, `trap`, and `DATA`. - `.ps1` script-file parsing. diff --git a/SPEC.md b/SPEC.md index 2cb6353..2eac353 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,8 +1,9 @@ # ShellSyntaxTree — bash & shared-contract Specification **Status:** v0.2.0 shipped; the accepted v0.3 contract adds structured syntax, -complete command occurrences, explicit redirect analysis, and bounded control -flow while retaining the v0.2 compatibility leaves. +complete command occurrences, explicit redirect analysis, bounded `for` / +`foreach`, substitutions, and execution regions while retaining the v0.2 +compatibility leaves. **Audience:** Whoever (human or agent) works on ShellSyntaxTree. **Read this end-to-end before writing any code.** **PowerShell support is specified separately in `SPEC.POWERSHELL.md` (v0.2.0); @@ -204,6 +205,10 @@ public enum RedirectOperation { ... } public static class ShellAnalysisLimits { ... } ``` +`ConditionLoopSyntax`, `ConditionalSyntax`, and `ConditionalBranchSyntax` are +reserved v0.3 structural vocabulary. The stable-v0.3 parsers do not emit them; +condition-loop and branch grammar remains fail closed until a later release. + That's the entire public API. **Everything else is internal.** The lexer, parser internals, verb tables, resolver — all implementation detail. @@ -1314,10 +1319,11 @@ quoted_string := single-quoted | double-quoted NOT path-resolved. The parser carries the raw token (e.g. `&1`) on `Redirect.Target` and sets `Redirect.IsDynamicSkip = true`. This prevents `2>&1` from being incorrectly resolved to `/&1`. -- Function definitions, assignment-prefix commands, `case`/`esac`, C-style or - implicit loops, arithmetic execution, process substitution, and single-`&` - background lists remain unparseable in stable v0.3 because they can hide - executable regions outside the bounded grammar below. +- Function definitions, assignment-prefix commands, `while` / `until`, `if` / + `elif` / `else`, `case`/`esac`, C-style or implicit loops, arithmetic + execution, process substitution, and single-`&` background lists remain + unparseable in stable v0.3 because they can hide executable regions outside + the bounded grammar below. ### v0.3 structured Bash grammar @@ -1330,8 +1336,6 @@ bash_list_item := bash_and_or bash_and_or := bash_pipeline (("&&" | "||") bash_pipeline)* bash_pipeline := bash_command ("|" bash_command)* bash_command := bash_for_in - | bash_condition_loop - | bash_if | bash_group | bash_subshell | bash_c_wrapper @@ -1342,17 +1346,6 @@ bash_for_in := "for" binding_name "in" iterable_word* bash_script(stop = "done") "done" -bash_condition_loop := ("while" | "until") - bash_script(stop = "do") "do" - bash_script(stop = "done") "done" - -bash_if := "if" bash_script(stop = "then") "then" - bash_script(stop = "elif" | "else" | "fi") - bash_elif* bash_else? "fi" -bash_elif := "elif" bash_script(stop = "then") "then" - bash_script(stop = "elif" | "else" | "fi") -bash_else := "else" bash_script(stop = "fi") - list_sep := ";" | NEWLINE list_terminator := ";" | NEWLINE+ binding_name := supported_scalar_binding @@ -1362,7 +1355,7 @@ iterable_word := word | quoted_string | supported_substitution ``` The supported stable-v0.3 set is the existing simple-command grammar plus -`for name in words`, `while` / `until`, and `if` / `elif` / `else`. Bash +`for name in words`. Bash accepts additional shell identifiers as loop variables, but this bounded grammar fails them closed for the initial-state reasons specified in §2. Every fully @@ -1382,7 +1375,7 @@ rather than changing the v0.2 `VerbChain.IsDynamic` contract, which remains PowerShell-specific. Diagnostic `Syntax` may retain the discovered substitution, but `Commands` and `Clauses` are empty. -Missing `do`, `done`, `then`, or `fi`; an unsupported substitution whose +Missing `do` or `done`; an unsupported substitution whose commands cannot all be discovered; or any skipped executable region makes the entire result unparseable. The parser may preserve a diagnostic syntax tree, but it returns empty `Commands` and `Clauses` so consumers cannot authorize a @@ -2559,10 +2552,12 @@ Stable v0.3 deliberately continues to exclude: live-shell evaluation. - Executable-specific option, operand, object, revision, or subcommand grammars; consumers own those semantics. -- Bash process substitution, single-`&` background lists, `case`, C-style or - implicit positional-parameter loops, arithmetic execution, functions, and - definitions until each hidden-execution and state boundary is specified. -- PowerShell `do`, `switch`, functions, definitions, class/type bodies, +- Bash `while`, `until`, `if`, `elif`, `else`, process substitution, single-`&` + background lists, `case`, C-style or implicit positional-parameter loops, + arithmetic execution, functions, and definitions until each hidden-execution + and state boundary is specified. +- PowerShell `while`, `if`, `elseif`, `else`, `do`, `switch`, functions, + definitions, class/type bodies, arbitrary execution-bearing expressions, and `.ps1` file-content parsing. - A stable serialized wire format for the polymorphic v0.3 records. - Caller-configurable analysis limits, filesystem-dependent pattern diff --git a/openspec/changes/v0-3-structured-shell-analysis/design.md b/openspec/changes/v0-3-structured-shell-analysis/design.md index 30685a4..f967c75 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/design.md +++ b/openspec/changes/v0-3-structured-shell-analysis/design.md @@ -25,7 +25,7 @@ incomplete executable regions never become authorization evidence. - Represent supported nested command structure for Bash and PowerShell. - Expose every command that may execute without requiring consumers to walk an evolving syntax-node hierarchy. -- Represent direct, callback, job, parallel, initialization, and deferred +- Represent direct, callback, job, parallel, initialization, and unknown PowerShell script-block execution without treating proved script-block data as code. - Resolve constrained loop values and shell state only when bounded proof is @@ -81,8 +81,9 @@ Public syntax nodes derive from one `ShellSyntaxNode` base. The base prevents external derivation so ShellSyntaxTree owns the complete node family. Common execution structure may use shell-neutral nodes such as blocks, simple commands, pipelines, command lists, groups, foreach-style loops, condition -loops, and branches. A shell-specific public node is preferred whenever a -shared type would erase material semantics. +loops, and branches. Condition and branch nodes are reserved vocabulary in +stable v0.3 rather than parser-emitted grammar. A shell-specific public node is +preferred whenever a shared type would erase material semantics. Consumers are not required to exhaustively match node types for authorization. They use `Commands`; a consumer that does inspect `Syntax` must fail closed or @@ -347,7 +348,7 @@ PowerShell script blocks are values whose receiver determines whether, when, how often, and where they execute. The former contract treated every ordinary script-block argument as inert. That is correct for `Write-Output { ... }` but incorrect for `ForEach-Object`, `Where-Object`, `Invoke-Command`, jobs, -module initialization, event actions, breakpoints, and argument completers. +module initialization, and potentially custom or deferred receivers. It also made `& { ... }` unparseable even though its body is statically delimited. @@ -376,7 +377,7 @@ PowerShell 7.6.4 probes demonstrate that these are independent dimensions: | `Measure-Command { ... }` / `Trace-Command -Expression { ... }` | shared current scope | shared | synchronous / once | | `Start-Job { ... }` | child process state | inherited initial location; exit isolated | concurrent / once | | `ForEach-Object -Parallel { ... }` | child runspace state; process-wide effects may escape | inherited initial location; runspace-local exit isolated | concurrent / once per input | -| event, breakpoint, completion actions | trigger-time state | trigger-time state | deferred / zero or more | +| unproved receiver or binding | unknown | unknown | unknown | The shell-specific analyzer owns inbound state, exit propagation, phase scheduling, success/failure partitions, and target/runspace differences. @@ -418,15 +419,16 @@ when parameter or positional order does not match runtime phase order. The region's `HostClauseElementIndex` is therefore a correlation coordinate, not an execution-order index. -The pinned PowerShell 7 catalog covers direct call and dot-source blocks; +The stable-v0.3 PowerShell 7 catalog covers direct call and dot-source blocks; `ForEach-Object` Begin, Process, End, RemainingScripts, and Parallel; `Where-Object -FilterScript`; `Invoke-Command -ScriptBlock`; `Measure-Command -Expression`; `Trace-Command -Expression`; `Start-Job` -ScriptBlock and InitializationScript; `New-Module -ScriptBlock`; -`Set-PSBreakpoint -Action`; `Register-ObjectEvent -Action`; -`Register-EngineEvent -Action`; and `Register-ArgumentCompleter -ScriptBlock`. -The optional inbox `Start-ThreadJob` command is complete only under a pinned -module baseline. Aliases, supported module-qualified spellings, static call +ScriptBlock and InitializationScript; and `New-Module -ScriptBlock`. +Optional-module `Start-ThreadJob` and deferred breakpoint, event, and argument- +completion bindings are not stable-v0.3 catalog-completeness promises. Existing +conservative recognition may remain without further expansion. Unproved forms +follow the unknown-receiver rule and retain visible bodies with incomplete +facts. Aliases, supported module-qualified spellings, static call operator spellings, parameter abbreviations/inline values, positional binding, parameter sets, and `ScriptBlock[]` binding use the same static catalog. @@ -727,8 +729,9 @@ unknown occurrence cwd clears those resolutions and retains the dynamic-cwd marker, so a parse location taken from the success partition cannot leak into an exact failure continuation. Decoded child-host compatibility leaves carry the inherited invocation-cwd attribution needed by that projection, while the -child's exit state remains isolated. General extraction of state primitives -waits until both language passes are complete and compared under task 8.1. +child's exit state remains isolated. General extraction of state primitives is +a post-v0.3 refactor and proceeds only after both language passes demonstrate +identical behavior. Parser-owned side facts retain each argument's complete `ShellValue` fragment sequence. For every concrete visit, the analyzer re-evaluates all arguments @@ -820,10 +823,14 @@ The implementation order is: 5. Validate the new consumer path on existing Netclaw cases. 6. Add Bash `for ... in` with literal values first. 7. Add PowerShell `foreach` with literal arrays next. -8. Extract shared occurrence/value/state machinery proven by both slices. -9. Add bounded patterns and iterator/substitution command discovery. -10. Add condition loops and branches in separately testable shell-specific - slices. +8. Add bounded patterns and iterator/substitution command discovery. +9. Complete explicit redirects, heredoc/here-string facts, consumer guidance, + Netclaw migration, and release verification. + +Shared occurrence/value/state extraction, condition loops and branches, exact +optional/deferred PowerShell receivers, and the Web/Mermaid showcase are +post-v0.3 follow-ups. They proceed only from demonstrated duplication or +consumer demand and do not gate the stable package. This order prevents a complete Bash implementation from hardening a Bash-shaped public abstraction before PowerShell exercises it. @@ -922,18 +929,18 @@ into the release specifications before production types are added. filesystem. Bash unmatched-glob settings can change whether the loop has zero iterations or yields the literal pattern, but neither outcome escapes that lexical cover. Every other pattern becomes `Unknown`. -5. v0.3 does not publish a finite cwd domain. Identical branch exits retain an - exact cwd; the first disagreement, unknown mutation, or loop-exit ambiguity - produces `Unknown`. +5. v0.3 does not publish a finite cwd domain. Supported reachable exits retain + an exact cwd only when they agree; the first disagreement, unknown mutation, + or loop-exit ambiguity produces `Unknown`. 6. The stable v0.3 grammar includes the existing simple-command grammar, - structural projection, Bash `for ... in`, `while` / `until`, and - `if` / `elif` / `else`, plus PowerShell `foreach`, `while`, and - `if` / `elseif` / `else` within the bounded subsets below. It also preserves + structural projection, Bash `for ... in`, and PowerShell `foreach` within + the bounded subsets below. It also preserves the existing Bash heredoc grammar while adding explicit body, delimiter, expansion, and completeness facts, and adds Bash `<<<` here strings. - Process substitution, single-`&` background lists, Bash `case`, PowerShell - `switch`, arithmetic/C-style loops, implicit Bash positional-parameter - loops, and function/definition bodies remain independently gated. + Condition loops and branches, process substitution, single-`&` background + lists, Bash `case`, PowerShell `switch`, arithmetic/C-style loops, implicit + Bash positional-parameter loops, and function/definition bodies remain + independently gated. 7. PowerShell direct and command-bound script blocks use the additive `ExecutionRegionSyntax` contract. Origin, phase, timing, and cardinality are public; state propagation remains shell-specific and multi-dimensional. The pinned @@ -1463,8 +1470,6 @@ bash_list_item := bash_and_or bash_and_or := bash_pipeline (("&&" | "||") bash_pipeline)* bash_pipeline := bash_command ("|" bash_command)* bash_command := bash_for_in - | bash_condition_loop - | bash_if | bash_group | bash_subshell | bash_c_wrapper @@ -1476,17 +1481,6 @@ bash_for_in := "for" binding_name "in" iterable_word* bash_script(stop = "done") "done" -bash_condition_loop := ("while" | "until") - bash_script(stop = "do") "do" - bash_script(stop = "done") "done" - -bash_if := "if" bash_script(stop = "then") "then" - bash_script(stop = "elif" | "else" | "fi") - bash_elif* bash_else? "fi" -bash_elif := "elif" bash_script(stop = "then") "then" - bash_script(stop = "elif" | "else" | "fi") -bash_else := "else" bash_script(stop = "fi") - list_sep := ";" | NEWLINE list_terminator := ";" | NEWLINE+ binding_name := shell_identifier @@ -1507,8 +1501,7 @@ existing lexer values and spans. |---|---| | Existing simple commands, `&&`, `||`, `;`, pipelines, groups, subshells, and static command-string wrappers | Supported and structurally projected | | `for name in words; do ...; done` | Supported | -| `while` / `until` command lists | Supported | -| `if` / `elif` / `else` command lists | Supported | +| `while` / `until` and `if` / `elif` / `else` command lists | Deferred; whole result unparseable until command discovery and state joins are specified | | Completely delimited `$()` substitution in a supported word, redirect value, iterable, or expanding heredoc body | Inner commands visible; produced value `Unknown` | | Legacy backtick command substitution | Whole result unparseable until its distinct escape and nesting rules are modeled | | Static path-shaped glob in a supported iterable | `Pattern` only under the locked covering-directory rule | @@ -1599,8 +1592,6 @@ statements, so only `;` and newline terminate a statement around `foreach`. ```text pwsh_script(stop) := pwsh_statement (statement_terminator pwsh_statement)* pwsh_statement := pwsh_foreach - | pwsh_while - | pwsh_if | pwsh_and_or statement_terminator := ";" | NEWLINE @@ -1611,14 +1602,6 @@ pwsh_pipeline := pipeline_element ("|" pipeline_element)* pwsh_foreach := "foreach" "(" variable "in" foreach_expression ")" script_block_body -pwsh_while := "while" "(" condition_pipeline ")" - script_block_body - -pwsh_if := "if" "(" condition_pipeline ")" script_block_body - pwsh_elseif* pwsh_else? -pwsh_elseif := "elseif" "(" condition_pipeline ")" script_block_body -pwsh_else := "else" script_block_body - foreach_expression := literal_value | literal_array | pipeline_expression @@ -1692,12 +1675,6 @@ receiver/binding creates an unknown incomplete region. A dynamic call operator whose block value is not authored inline remains incomplete because no body is available. -`condition_pipeline` is limited to a pipeline that the existing command parser -can delimit completely. Pure literal and comparison expressions may be -preserved as non-executable condition syntax, but any subexpression, member -invocation, script block, or other form that can execute while escaping -complete command discovery makes the whole result unparseable. - | PowerShell construct | Stable v0.3 status | |---|---| | Existing simple commands, pipelines, statement separators, grouping, and static wrapper / `Invoke-Expression` recursion | Supported and structurally projected | @@ -1708,8 +1685,7 @@ complete command discovery makes the whole result unparseable. | Single-quoted, literal-here-string, or backtick-escaped `$()` text | Literal/opaque data; no invented substitution occurrence | | Execution-bearing `@()` / `@{}` outside a completely modeled foreach literal expression | Whole result unparseable until complete command discovery is modeled | | `foreach ($name in expression) { ... }` for literal scalar, literal array, or fully delimited pipeline iterables | Supported | -| `while (condition_pipeline) { ... }` | Supported | -| `if` / `elseif` / `else` with fully delimited condition pipelines | Supported | +| `while`, `if`, `elseif`, and `else` | Deferred; whole result unparseable until command discovery and state joins are specified | | Pipeline-produced iterator objects | Iterator commands visible; produced values `Unknown` | | Cataloged executing script-block arguments | Typed origin/phase/timing/cardinality region; host and body commands visible | | Cataloged non-executing script-block data | Existing opaque argument; no invented child execution | diff --git a/openspec/changes/v0-3-structured-shell-analysis/proposal.md b/openspec/changes/v0-3-structured-shell-analysis/proposal.md index e224357..cc6d5b6 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/proposal.md +++ b/openspec/changes/v0-3-structured-shell-analysis/proposal.md @@ -17,11 +17,11 @@ fail-closed behavior for incomplete analysis. model, with shell-specific front ends producing one shared structural contract where their semantics actually coincide. - Add a library-owned command-occurrence projection containing every command - that may execute, including condition, iterator, branch, loop-body, wrapped, - substitution, and PowerShell script-block execution-region commands. + that may execute in supported grammar, including iterator, loop-body, + wrapped, substitution, and PowerShell script-block execution-region commands. - Correct the PowerShell script-block boundary: represent direct invocation, current-runspace callbacks, child-runspace/process jobs, module - initialization, and deferred actions as typed execution regions while + initialization, and unknown receivers as typed execution regions while retaining proved non-executing script-block data as opaque values. Origin, phase, timing, and cardinality are public structural facts; variable, location, command-resolution, runspace, and process propagation remain @@ -46,13 +46,22 @@ fail-closed behavior for incomplete analysis. that check `IsUnparseable` continue to fail closed and do not silently miss nested executable commands. Unparseable results expose no command or clause authorization projection. -- Expand grammar in vertical slices: Bash `for ... in`, PowerShell `foreach`, - then the locked condition loops and branches. Preserve the existing Bash +- Expand grammar in vertical slices through Bash `for ... in` and PowerShell + `foreach`. Preserve the existing Bash heredoc grammar while adding body, delimiter, and expansion facts, and add Bash `<<<` here-string semantics. Process substitution, background lists, - C-style loops, arithmetic, Bash `case`, and PowerShell `switch` remain - independently gated by explicit executable-region and value-semantics - requirements. + condition loops and branches, C-style loops, arithmetic, Bash `case`, and + PowerShell `switch` remain fail-closed follow-ups rather than stable-v0.3 + release requirements. +- Bound the stable-v0.3 PowerShell receiver catalog to direct invocation, + synchronous current-runspace callbacks, `Start-Job`, `ForEach-Object + -Parallel`, and local or remote `Invoke-Command` forms already needed by the + validating consumer. Optional `Start-ThreadJob` module/version proof and + exact deferred breakpoint, event, and argument-completion semantics remain + follow-ups. Existing conservative recognition may remain, but no additional + catalog expansion gates v0.3. Unproved receivers still expose completely + delimited bodies as incomplete execution regions, so this scope reduction + does not hide commands. - Keep executable-specific option and operand interpretation, authorization policy, and durable approval scope consumer-owned. - Update `docs/CONSUMER_GUIDE.md` and the README usage path so security gates @@ -107,3 +116,10 @@ short-term containment; a v0.3 integration must switch authorization traversal to the command-occurrence projection and retain strict or prompt behavior for unknown executable shapes. No native dependency or command execution is introduced. + +Stable v0.3 is outcome-gated, not backlog-gated. Shared-analysis refactoring, +additional shell grammar, exact optional/deferred PowerShell receiver semantics, +and the Web/Mermaid showcase remain useful follow-ups, but they do not block the +consumer migration or stable package once the contracted security behavior is +verified. Already-merged conservative behavior may remain; it is not a promise +to expand adjacent catalog or grammar surface during v0.3. diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md index 26db4c6..0faa7b2 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/bounded-shell-analysis/spec.md @@ -401,10 +401,10 @@ complete executable-aware grammar before reusing authorization. - **THEN** the authored variable argument remains distinct from its effective value - **THEN** the consumer applies the native executable grammar to `--force` -### Requirement: Control-flow state joins conservatively +### Requirement: Sequential and loop state joins conservatively Working-directory and supported variable state SHALL be propagated through -sequential regions and joined across branches and loop exits. Disagreement -SHALL never be resolved by arbitrarily choosing one path. +sequential regions and joined across loop exits. Disagreement SHALL never be +resolved by arbitrarily choosing one path. The Bash analyzer SHALL internally partition reachable exit state by command success and failure. `&&` SHALL continue from the success partition, `||` @@ -457,15 +457,6 @@ An unreachable success or failure partition SHALL remain unreachable. The analyzer SHALL NOT substitute a joined state for a missing `&&` or `||` partition merely to publish exact continuation facts. -#### Scenario: Branch-dependent cwd -- **WHEN** one branch changes cwd to `/a` and another changes cwd to `/b` -- **THEN** a following relative path is not resolved solely under `/a` or solely under `/b` -- **THEN** the cwd is unknown because v0.3 does not publish divergent cwd alternatives - -#### Scenario: Identical branch cwd -- **WHEN** every supported branch exits with the same exact cwd -- **THEN** the joined cwd remains exact - #### Scenario: Ungated cd failure keeps the prior cwd possible - **WHEN** Bash parses `cd /maybe; pwd` - **THEN** the `cd` occurrence uses the incoming cwd @@ -625,12 +616,13 @@ blocks; `ForEach-Object` Begin, Process, End, RemainingScripts, and Parallel binding; `Where-Object -FilterScript`; in-process and remote `Invoke-Command -ScriptBlock`; `Measure-Command -Expression`; `Trace-Command -Expression`; `Start-Job` ScriptBlock and -InitializationScript; `New-Module -ScriptBlock`; `Set-PSBreakpoint -Action`; -`Register-ObjectEvent -Action`; `Register-EngineEvent -Action`; and -`Register-ArgumentCompleter -ScriptBlock`. The optional inbox -`Microsoft.PowerShell.ThreadJob` command MAY be complete only when the caller's -pinned module baseline proves its canonical identity; otherwise it follows the -unknown-receiver rule. +InitializationScript; and `New-Module -ScriptBlock`. +Optional-module `Start-ThreadJob` and deferred breakpoint, event, and argument- +completion receivers are not stable-v0.3 catalog-completeness requirements. +Existing conservative recognition MAY remain, but additional module/version or +trigger-time proof does not gate the release. Every unproved form SHALL follow +the unknown-receiver rule: its completely parsed body remains visible while +execution and affected state are incomplete. Known aliases, supported module-qualified spellings, static call-operator spellings, PowerShell parameter prefixes and inline values, positional @@ -698,8 +690,8 @@ SHALL NOT prove concurrency. - **THEN** the consumer can select child-scope or current-scope state flow without reparsing source text `Start-Job` executes initialization before its main block in a child process; -`ForEach-Object -Parallel` and a proved `Start-ThreadJob` execute in child -runspaces. Runspace-local variable and location exit mutation does not flow into +`ForEach-Object -Parallel` executes in child runspaces. Runspace-local variable +and location exit mutation does not flow into the containing continuation. In-process child runspaces share process-wide state such as the environment provider, so a possibly escaping mutation SHALL invalidate later host binding, command-resolution, and location facts. It SHALL @@ -712,10 +704,6 @@ be retained in bounded case-insensitive state. A later matching invocation whose identity is not independently proved SHALL be treated as a possible process-wide mutation. An ambiguous name, wildcard, or candidate-set overflow SHALL fail closed to every unproved command name. -Breakpoint actions, event actions, and argument completers are deferred and -may execute zero or more times; their trigger-time cwd and mutable state are -Unknown unless independently proved. Deferred commands remain in the -may-execute projection even though registration itself does not execute them. #### Scenario: Child scope and shared location are independent - **WHEN** isolated-mode PowerShell parses `foreach ($x in 'outer') { }; & { Write-Output inner -OutVariable x; Set-Location /tmp }; Write-Output $x; Get-Location` @@ -733,12 +721,6 @@ may-execute projection even though registration itself does not execute them. - **THEN** abstract-state execution applies Begin, then Process per input, then End - **THEN** facts are joined back to their authored occurrences -#### Scenario: Deferred trigger-time state is unknown -- **WHEN** an event action contains `Remove-Item relative.txt` -- **THEN** the action occurrence remains visible -- **THEN** its trigger-time cwd is Unknown without an independent proof -- **THEN** no registration-time relative path approval is synthesized - #### Scenario: Ambiguous custom receiver fails closed without hiding the body - **WHEN** PowerShell parses `Invoke-Custom { Remove-Item target.txt }` without a proved receiver contract - **THEN** both host and body commands remain visible diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md index cec6f92..023f6ec 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/consumer-compatibility/spec.md @@ -3,8 +3,8 @@ ### Requirement: Existing Clause projection remains conservative `ParsedCommand.Clauses` SHALL remain available in v0.3 and SHALL contain every authored simple command that may execute for a fully parseable result, including -nested condition, iterator, branch, body, substitution, and execution-region -commands. +nested iterator, loop-body, substitution, and execution-region commands from +the stable-v0.3 grammar. Existing raw spelling, decoded values, source spans, and unaffected v0.2 leaf classifications SHALL remain compatible. A paired real-shell oracle MAY @@ -92,17 +92,12 @@ recursive syntax traversal to discover executable commands. - **THEN** it does not grant scope to the loop keyword itself #### Scenario: Syntax remains useful for explanation -- **WHEN** a UI groups approvals by loop or branch +- **WHEN** a UI groups approvals by loop or execution region - **THEN** it may use syntax ancestry for display - **THEN** authorization still uses the complete occurrence collection -#### Scenario: Deferred execution is authorized at registration -- **WHEN** a registration command contains a deferred event, breakpoint, or completion body -- **THEN** the example authorization algorithm evaluates the registration command and every body occurrence -- **THEN** it does not wait for the external trigger or omit the deferred body - #### Scenario: Execution metadata does not grant approval -- **WHEN** an execution region is synchronous, concurrent, or deferred +- **WHEN** an execution region has known or unknown timing and cardinality - **THEN** timing and cardinality remain explanatory shell facts - **THEN** the consumer still interprets every authored command occurrence and policy-sensitive value diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md index c2f19dd..3a922d1 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/executable-command-projection/spec.md @@ -5,11 +5,6 @@ Every fully parseable result SHALL expose a command-occurrence collection that contains each authored simple command that may execute exactly once, regardless of whether the command is top-level or nested. -#### Scenario: Commands in mutually exclusive branches -- **WHEN** Bash parses `if test -f a; then rm a; else echo missing; fi` -- **THEN** the collection contains `test`, `rm`, and `echo` exactly once each -- **THEN** the collection does not predict which branch will run - #### Scenario: Loop body occurrence is not multiplied - **WHEN** isolated-mode Bash parses `for f in a b c; do echo "$f"; done` - **THEN** the authored `echo` command appears once @@ -90,11 +85,6 @@ partial command and compatibility result. - **THEN** the payload remains an authored `DynamicSkip` value - **THEN** the command occurrence has `IsComplete=false` and cannot authorize hidden code -#### Scenario: While condition and body roles -- **WHEN** Bash parses `while curl URL; do sleep 1; done` -- **THEN** `curl` is identified as a condition occurrence -- **THEN** `sleep` is identified as a loop-body occurrence - #### Scenario: PowerShell iterator pipeline role - **WHEN** PowerShell parses `foreach ($f in Get-ChildItem C:\input) { Remove-Item $f }` - **THEN** `Get-ChildItem` is identified as an iterator occurrence @@ -137,11 +127,6 @@ SHALL NOT omit the body or authorize it as inert data. - **THEN** occurrences contain `Get-ChildItem`, `ForEach-Object`, and `Remove-Item` exactly once - **THEN** the body occurrence has execution-region ancestry nested beneath the pipeline stage -#### Scenario: Deferred callback remains a may-execute command -- **WHEN** PowerShell registers a breakpoint, event action, or argument completer with a supported script block -- **THEN** the registration command and every body command appear exactly once -- **THEN** projection does not predict how many future triggers occur - #### Scenario: Unknown receiver does not hide a script block - **WHEN** a script-block argument's receiver or binding is not statically proved - **THEN** the host and every body command remain visible @@ -188,11 +173,6 @@ the iterator-command collection. - **WHEN** isolated-mode Bash parses `for f in $(find .); do rm "$f"; done` - **THEN** the `find` occurrence precedes the `rm` occurrence -#### Scenario: Branches preserve authored order -- **WHEN** PowerShell parses an `if` statement with then and else commands -- **THEN** condition commands precede then-body commands -- **THEN** then-body commands precede else-body commands in the projection - #### Scenario: Ordinary command substitution precedes its consumer - **WHEN** Bash parses `rm "$(find /tmp)"` - **THEN** `find` precedes `rm` in both command and compatibility projections diff --git a/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md b/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md index 0c17cdf..6bbcf59 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md +++ b/openspec/changes/v0-3-structured-shell-analysis/specs/structured-shell-syntax/spec.md @@ -3,13 +3,16 @@ ### Requirement: Parsed commands expose authored nested structure Every fully parsed command SHALL expose one library-owned syntax root that preserves the authored nesting and source order of supported command lists, -pipelines, groups, simple commands, loops, and branches. +pipelines, groups, simple commands, substitutions, execution regions, and +foreach loops. The public syntax family SHALL be a closed hierarchy of records derived from `ShellSyntaxNode`. Every node SHALL expose a `ShellSyntaxKind` discriminant and zero SHALL mean `Unknown`. The locked family SHALL include block, simple command, pipeline, command list, group, foreach, condition loop, conditional, conditional branch, command substitution, and execution-region nodes. +Condition-loop and conditional node kinds are reserved structural vocabulary; +stable v0.3 does not emit them because their grammar remains fail closed. #### Scenario: Existing flat command receives a structural root - **WHEN** either parser parses `git status && dotnet test` @@ -159,12 +162,6 @@ that `SimpleCommandSyntax` and SHALL identify the exact script-block - **THEN** the whole parse is unparseable until bounded declaration grammar is implemented - **THEN** a realistic argument completer is not partially authorized from only its post-declaration body -#### Scenario: Deferred action is still authorization-visible -- **WHEN** PowerShell parses `Register-EngineEvent -SourceIdentifier ready -Action { Remove-Item marker.txt }` -- **THEN** the host command appears before the action body in the occurrence projection -- **THEN** the action region is `Deferred` with `ZeroOrMore` cardinality -- **THEN** `Remove-Item` remains authorization-visible even though registration does not execute it - #### Scenario: Known non-executing script-block data stays data - **WHEN** a constrained canonical-command context parses `Write-Output { Remove-Item target.txt }` - **THEN** the script block remains one opaque compatibility argument @@ -225,21 +222,6 @@ expressions into a false shared expression grammar. - **THEN** the iterable raw text remains available - **THEN** its outer source start and length are null -### Requirement: Condition loops and branches preserve executable regions -The parser SHALL preserve the condition, every branch body, and the -continuation after each supported Bash or PowerShell condition loop or branch -as distinct structural regions. - -#### Scenario: Bash while condition contains a command -- **WHEN** Bash parses `while curl https://example.invalid/ready; do echo waiting; done` -- **THEN** the condition block contains the `curl` command -- **THEN** the body block contains the `echo` command - -#### Scenario: PowerShell branch preserves both alternatives -- **WHEN** PowerShell parses `if (Test-Path a.txt) { Remove-Item a.txt } else { Write-Output missing }` -- **THEN** the condition, then body, and else body remain distinct regions -- **THEN** no branch is discarded based on predicted runtime behavior - ### Requirement: Source ranges are exact or explicitly unavailable Each direct-source structural node SHALL carry a source range into `ParsedCommand.Source`. A node lifted from decoded, escaped, or encoded wrapper diff --git a/openspec/changes/v0-3-structured-shell-analysis/tasks.md b/openspec/changes/v0-3-structured-shell-analysis/tasks.md index b67cb19..02be1e0 100644 --- a/openspec/changes/v0-3-structured-shell-analysis/tasks.md +++ b/openspec/changes/v0-3-structured-shell-analysis/tasks.md @@ -9,15 +9,18 @@ - [x] 1.7 Synchronize PowerShell grammar and analysis deltas into `SPEC.POWERSHELL.md`. - [x] 1.8 Update `PROJECT_CONTEXT.md` and `IMPLEMENTATION_PLAN.md` with the accepted v0.3 scope and delivery slices. - [x] 1.9 Add a paired Bash and PowerShell design corpus that records current behavior, desired structure, command occurrences, bounded values, redirect facts, compatibility projections, and security invariants. -- [ ] 1.10 Promote each design case into the executable corpus as its production parser slice lands. +- [ ] 1.10 Promote every design case for a stable-v0.3 construct into the + executable corpus as its production parser slice lands. Retain future-scope + design cases as non-gating evidence rather than release work. - [x] 1.11 Correct the PowerShell script-block boundary and lock the additive execution-region node, origin/phase/timing/cardinality facts, authored-versus-semantic ordering, command projection, and independent shell-state analysis contract against local PowerShell 7.6.4 oracles. - [x] 1.12 Expand the PowerShell design corpus with direct call/dot-source, synchronous callback, binder phase, local/remote invocation, child - process/runspace, initialization, deferred action, proved data, and unknown - receiver cases before production implementation. + process/runspace, initialization, proved data, and unknown receiver cases + before production implementation. Deferred-action cases remain design-only + evidence until separately promoted after v0.3. ## 2. Resolver Provenance Correction and Shared Preparation @@ -77,7 +80,9 @@ - [ ] 5.3 Document exact, finite, pattern, unknown, joined-state, redirect, and incomplete-result handling. - [ ] 5.4 Document record equality, hashing, `ToString()`, serialization, and `Clauses` compatibility effects. - [ ] 5.5 Update the README getting-started and migration examples to direct v0.3 consumers to the command-occurrence API and full consumer guide. -- [ ] 5.6 Publish a 0.3.0 prerelease containing the structural API before enabling control flow. +- [ ] 5.6 Publish a 0.3.0 prerelease containing the contracted structural, + substitution, redirect, Bash `for`, and PowerShell `foreach` behavior before + the downstream migration gate. - [ ] 5.7 Migrate Netclaw's existing-command analysis to the occurrence and redirect APIs behind focused regression tests. ## 6. Bash For-In Vertical Slice @@ -189,45 +194,26 @@ - [ ] 7.5b Implement direct `& {}` and `. {}` plus synchronous current-runspace regions for ForEach-Object, Where-Object, Measure-Command, Trace-Command, in-process Invoke-Command, and New-Module with shell-specific state flow. - - [ ] 7.5c Implement Start-Job and initialization, ForEach-Object -Parallel, - remote/session Invoke-Command, `-AsJob`, and pinned Start-ThreadJob child - process/runspace boundaries conservatively. - - [ ] 7.5d Implement deferred breakpoint, event, and argument-completion action - regions with trigger-time unknown state and zero-or-more cardinality. + - [x] 7.5c Implement Start-Job and initialization, ForEach-Object -Parallel, + remote/session Invoke-Command, and `-AsJob` child process/runspace + boundaries conservatively. Additional optional-module `Start-ThreadJob` + proof is not a stable-v0.3 requirement; unproved forms follow the unknown- + receiver rule. - [ ] 7.5e Keep proved non-executing script-block data opaque; over-approximate unknown receivers/bindings as unknown incomplete regions; fail atomically on unsupported interiors or state transfers. - [ ] 7.5f Pin authored projection order separately from semantic phase order, exact host element coordinates, nested regions, wrappers, pipelines, loops, and the 16-container depth boundary. - - [ ] 7.5g Either implement a bounded leading `param(...)` declaration and - direct-block argument-binding grammar or retain explicit atomic-failure - cases for both direct operators and realistic argument completers. + - [x] 7.5g Retain explicit atomic-failure behavior for direct-block arguments + and leading `param(...)` declarations. Declaration and argument-binding + grammar is not required for stable v0.3. - [ ] 7.6 Add adversarial cases for object-valued iterables, mutation, dynamic invocation, splatting, and cap overflow. - [ ] 7.7 Add PowerShell corpus entries, live `pwsh` oracle coverage, and Netclaw integration cases. - Add case-specific `PwshInitialStateMode` support to `PwshCorpusTool` before folding isolated-state entries 362+ into its generated manifest. -## 8. Proven Shared Analysis Extraction - -- [ ] 8.1 Compare the two working loop implementations and inventory only behaviorally identical analysis steps. -- [ ] 8.2 Extract shared command-occurrence traversal without coupling shell token consumption. -- [ ] 8.3 Extract the value-domain lattice, combination cap, and unknown fallback. -- [ ] 8.4 Extract conservative sequential and branch-state join primitives used identically by both shells. -- [ ] 8.5 Keep shell-specific iterable, quoting, scoping, expression, and parser code behind explicit adapters. -- [ ] 8.6 Re-run both complete corpora to prove the extraction is behavior-preserving. - -## 9. Condition Loops and Branches - -- [ ] 9.1 Add Bash `while` and `until` with condition and body command occurrences. -- [ ] 9.2 Add Bash `if` / `elif` / `else` with conservative branch-state joins. -- [ ] 9.3 Defer Bash `case` until after stable v0.3 and add it only after pattern and branch-selection uncertainty is specified. -- [ ] 9.4 Add PowerShell `while` with the locked condition-pipeline boundary; defer `do` forms until after stable v0.3. -- [ ] 9.5 Add PowerShell `if` / `elseif` / `else` with conservative branch-state joins. -- [ ] 9.6 Defer PowerShell `switch` until after stable v0.3 and add it only after string, regex, wildcard, and script-block modes are bounded explicitly. -- [ ] 9.7 Add paired security scenarios proving every condition and branch command remains visible. - -## 10. Heredoc / Here-String Slice and Separately Gated Follow-ups +## 10. Heredoc / Here-String Slice - [x] 10.1 Specify heredoc delimiter adjacency and quoting, expansion mode, body provenance, substitutions, tab stripping, completeness, and Bash here-string semantics. - [x] 10.2 Preserve existing `<<` / `<<-` behavior and fix quoted-delimiter adjacency without regressing the v0.2 compatibility redirect. @@ -235,11 +221,6 @@ - [ ] 10.4 Add Bash `<<<` here-string tokenization, explicit redirect facts, bounded operand analysis, and trailing-newline semantics. - [ ] 10.5 Add direct, malformed, quoted/unquoted, tab-stripped, dynamic, and substitution-bearing corpus cases plus real-Bash parse-only validation. - [x] 10.5a Add direct, executable-corpus, real-Bash output, and real-Bash parse-only coverage for the bounded substitution-discovery slice; explicit redirect facts and the full heredoc matrix remain pending. -- [ ] 10.6 After stable v0.3, specify process-substitution command discovery and the unknown produced descriptor/path value before enabling it. -- [ ] 10.7 After stable v0.3, specify background-list concurrency, ordering, and shell-state boundaries before enabling single `&`. -- [ ] 10.8 Specify C-style loop and arithmetic hidden-execution behavior before enabling either construct. -- [ ] 10.9 Keep URL-versus-glob and environment-assignment approval behavior in executable-aware consumer issues unless a shell lexical fact is missing. -- [ ] 10.10 Reproduce multiline quoted-argument reports against exact parser input before assigning a parser change. ## 11. Verification and Release @@ -250,5 +231,31 @@ - [ ] 11.5 Validate the public API field-for-field against the synchronized shared and PowerShell specifications. - [ ] 11.6 Validate Netclaw's ordinary-command, redirect, bounded-loop, and unknown-value approval matrices against the prerelease package. - [ ] 11.7 Update release notes and remove Netclaw's temporary descriptor workaround only after explicit redirect integration is live. -- [ ] 11.8 Expand the Web sample with curated complex Bash and PowerShell inputs and snapshot-tested deterministic Mermaid diagrams produced only from canonical syntax, occurrence, and compatibility projections; cover ancestry, redirects, and fail-closed results, escape arbitrary shell labels safely, and emit no raw HTML. - [ ] 11.9 Promote stable 0.3.0 only after Linux and Windows CI, package publication, and downstream acceptance succeed. + +## Post-v0.3 Backlog (Non-Gating) + +These are worthwhile follow-ups, not unfinished tasks in this change: + +- Compare the working Bash and PowerShell analyzers and extract only behavior + proven identical; keep shell token consumption, quoting, scoping, and + expression semantics behind separate adapters. +- Specify and implement Bash `while`, `until`, `if`, and `elif` plus PowerShell + `while`, `if`, and `elseif` in shell-specific vertical slices with paired + execution-accounting scenarios. +- Specify Bash `case` and PowerShell `switch` only after their pattern and + branch-selection uncertainty is bounded. +- Specify process-substitution values and command discovery, background-list + concurrency/state, and C-style/arithmetic hidden execution before enabling + those constructs. +- Add exact optional-module `Start-ThreadJob` and deferred breakpoint, event, + and argument-completion receiver semantics only when consumer demand + justifies a pinned runtime/module contract. Existing conservative recognition + may remain; unproved forms keep visible bodies unknown and incomplete. +- Keep URL-versus-glob, environment-assignment, and multiline reproduction + work in executable-aware consumer issues unless an exact missing lexical + fact is demonstrated. +- Expand the Web sample with curated Bash and PowerShell inputs and + snapshot-tested deterministic Mermaid diagrams from canonical projections. + Escape arbitrary labels and emit no raw HTML; this showcase does not gate + package or Netclaw delivery.