Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 22 additions & 17 deletions IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -378,8 +378,8 @@ priorities.
aliases, supported module qualification, exact and abbreviated/inline
parameters, positional slots, parameter sets, `ScriptBlock[]`, authored
ForEach-Object multi-block coordinates, and semantic Begin/Process/End
phases. The optional Microsoft.PowerShell.ThreadJob 2.2.0 entry remains
incomplete unless its separate module-baseline proof is supplied. Local
phases. The optional Microsoft.PowerShell.ThreadJob entry remains an
unknown incomplete receiver and no longer gates stable v0.3. Local
`Invoke-Command -AsJob`, ambiguous prefixes, malformed value binding,
unproved identities, and unknown receivers retain unknown/incomplete
facts. Supported catalog-owned module qualifications now pass structural
Expand Down Expand Up @@ -448,27 +448,21 @@ priorities.
authored `Set-Alias Env:...` invocation.
The generator-owned executable corpus now supports per-entry initial-
state mode and includes the promoted Parallel and remote/session cases.
Pinned child runspace jobs remain before deferred
breakpoint/event/completion actions; then unknown
receiver and nested/adversarial matrices. Preserve script blocks proved
Stable v0.3 stops at the delivered Start-Job, Parallel, and remote/session
boundaries. Optional-module Start-ThreadJob and exact deferred
breakpoint/event/completion actions are post-v0.3 catalog work; unknown
receivers continue to expose incomplete bodies. Preserve script blocks proved
to be data as opaque values, expose ambiguous bodies with incomplete
facts, and fail atomically when any potentially executable interior is
unsupported. Local PowerShell 7.6.4 probes pin variable-versus-location
independence, semantic phase order, child process/runspace boundaries,
registration-versus-trigger timing, and the fact that the in-process
`Invoke-Command` parameter set does not support `-AsJob`.
- [ ] Deliver Bash `for ... in` and PowerShell `foreach` as the first two
language-specific vertical slices, then extract only the shared analysis
proven by both implementations.
boundaries and the fact that the in-process `Invoke-Command` parameter
set does not support `-AsJob`.
- [ ] Complete the stable-v0.3 Bash `for ... in` and PowerShell `foreach`
vertical slices without gating release on a shared-analysis refactor.
- [ ] Build on the delivered bounded Bash heredoc grammar and quoted-delimiter
adjacency by exposing public body/delimiter/expansion/completeness facts,
then add a separately tested Bash `<<<` here-string redirect slice.
- [ ] Near the end of v0.3 delivery, expand the Web sample with curated complex
Bash and PowerShell inputs and deterministic Mermaid views of syntax,
occurrences, compatibility clauses, ancestry, redirects, and fail-closed
outcomes. Keep visualization downstream of the canonical projection so
it cannot become a second command-discovery implementation; snapshot the
rendering, escape arbitrary shell labels, and emit no raw HTML.

---

Expand Down Expand Up @@ -498,4 +492,15 @@ priorities.

## Parked

*(empty; move items here when scope changes rather than deleting them)*
- After stable v0.3, consider shared Bash/PowerShell analysis extraction only
where the two delivered implementations prove identical behavior.
- Add Bash and PowerShell condition loops and branches in shell-specific
vertical slices; keep `case`, `switch`, process substitution, background
lists, and arithmetic independently gated.
- Add or expand exact optional-module Start-ThreadJob and deferred breakpoint,
event, and argument-completion receiver semantics only when consumer demand
justifies a pinned contract; existing conservative recognition may remain.
- Expand the Web sample with curated Bash and PowerShell inputs and
snapshot-tested deterministic Mermaid views produced from canonical
projections. This remains part of the broader product follow-up, not the
stable-v0.3 package or Netclaw migration gate.
23 changes: 14 additions & 9 deletions PROJECT_CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,23 +80,28 @@ zero-native-deps .NET parser sized to what security gates actually need.
- Add a closed, strongly typed syntax-node hierarchy while retaining existing
`Clause` leaves.
- Add a library-owned command-occurrence projection for security consumers so
every potentially executable iterator, condition, branch, substitution, and
body command is evaluated exactly once.
every potentially executable iterator, substitution, execution-region, and
body command in the supported grammar is evaluated exactly once.
- Add typed PowerShell execution regions for direct call/dot-source blocks,
synchronous callbacks, jobs/parallel runspaces, initialization, and deferred
actions. Public origin/phase/timing/cardinality facts remain separate from
synchronous callbacks, jobs/parallel runspaces, and initialization. Public
origin/phase/timing/cardinality facts remain separate from
shell-specific variable, location, command-resolution, runspace, and process
state analysis; proved script-block data stays opaque and unknown receivers
conservatively expose incomplete bodies.
- Add fixed, non-executing value and state analysis: at most 32 candidates, at
most 16 structural container levels, and the existing wrapper depth of 5.
- Deliver Bash `for ... in` and PowerShell `foreach` first, then the locked
`while` and `if` subsets independently for each shell. Shared lowering and
analysis are extracted only after both front ends prove identical behavior.
- Deliver Bash `for ... in` and PowerShell `foreach` in stable v0.3. Condition
loops, branches, and shared-analysis extraction are post-v0.3 work and do
not gate the validating consumer migration.
- Preserve existing Bash heredocs and add explicit body/expansion facts plus
Bash `<<<` here strings. Keep process substitution, background lists, Bash
`case`, PowerShell `switch`, arithmetic/C-style loops, and definitions
independently gated.
condition loops and branches, `case`, PowerShell condition loops and
branches, arithmetic/C-style loops, and definitions independently gated.
- Keep optional-module `Start-ThreadJob` and exact deferred breakpoint, event,
and argument-completion receiver semantics outside the stable-v0.3 catalog.
Existing conservative recognition may remain; unproved receivers still
expose completely delimited bodies as incomplete regions and therefore
remain fail closed.
- Treat `openspec/changes/v0-3-structured-shell-analysis/` and its paired design
corpus as the review authority until the accepted contract is synchronized
into `SPEC.md` and `SPEC.POWERSHELL.md` with the production API change.
Expand Down
69 changes: 27 additions & 42 deletions SPEC.POWERSHELL.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# ShellSyntaxTree — PowerShell Specification (through v0.3)

**Status:** v0.2.0 shipped; the accepted v0.3 contract adds bounded PowerShell
`foreach`, `while`, and `if` structure plus shared command-occurrence and
explicit redirect analysis.
`foreach` structure plus shared command-occurrence and explicit redirect
analysis.
**Audience:** Whoever (human or agent) implements, consumes, or maintains the
ShellSyntaxTree PowerShell parser.
**Read `SPEC.md` (the bash and shared-contract specification) end-to-end
Expand All @@ -17,7 +17,7 @@ It is **not** a PowerShell interpreter. It does not execute, expand, or
evaluate commands. It returns the same structured AST a consumer already
walks for bash. The parsing scope is **Pipeline-aware** (§4): linear
command pipelines parse; stable v0.3 also supports only the explicitly bounded
`foreach`, `while`, and `if` subsets below. Other script-level constructs mark
`foreach` subset below. Other script-level constructs mark
`IsUnparseable`.

`SPEC.md` is the canonical home of the shared public API, AST, sanitization
Expand Down Expand Up @@ -74,13 +74,13 @@ syntax (§5) are all PowerShell 7 semantics. The `pwsh` validation oracle

### v0.3 extension

Stable v0.3 adds structured projection for bounded `foreach`, `while`, and
`if` statements; exposes iterator, condition, branch, and body commands
exactly once; derives exact or finite string values only from proved literal
Stable v0.3 adds structured projection for bounded `foreach` statements;
exposes iterator and body commands exactly once; derives exact or finite string
values only from proved literal
iterables; and joins location and supported binding state conservatively.
Pipeline-produced objects and unsupported expressions remain unknown without
execution. `do`, `switch`, definitions, and arbitrary script evaluation stay
outside the supported grammar.
execution. `while`, `if`, `elseif`, `else`, `do`, `switch`, definitions, and
arbitrary script evaluation stay outside the supported grammar.

---

Expand Down Expand Up @@ -389,8 +389,6 @@ terminators.
```text
pwsh_script(stop) := pwsh_statement (statement_terminator pwsh_statement)*
pwsh_statement := pwsh_foreach
| pwsh_while
| pwsh_if
| pwsh_and_or

statement_terminator := ";" | NEWLINE
Expand All @@ -401,14 +399,6 @@ pwsh_pipeline := pipeline_element ("|" pipeline_element)*
pwsh_foreach := "foreach" "(" variable "in" foreach_expression ")"
script_block_body

pwsh_while := "while" "(" condition_pipeline ")"
script_block_body

pwsh_if := "if" "(" condition_pipeline ")" script_block_body
pwsh_elseif* pwsh_else?
pwsh_elseif := "elseif" "(" condition_pipeline ")" script_block_body
pwsh_else := "else" script_block_body

foreach_expression := literal_value
| literal_array
| pipeline_expression
Expand Down Expand Up @@ -537,8 +527,6 @@ The version-pinned PowerShell 7 catalog covers:
| `Start-Job -InitializationScript` | Initialization | Concurrent | Once | child process before Main |
| `Start-Job -ScriptBlock` | Main | Concurrent | Once | child process; exit isolated |
| `New-Module -ScriptBlock` | Initialization | Synchronous | Once | module state; current-runspace effects analyzed separately |
| `Set-PSBreakpoint -Action`, event `-Action` | Action | Deferred | ZeroOrMore | trigger-time state Unknown without proof |
| `Register-ArgumentCompleter -ScriptBlock` | Completion | Deferred | ZeroOrMore | completion-time state Unknown without proof |

Remote `Invoke-Command` bodies begin with Unknown working directory, bindings,
aliases, functions, modules, profiles, and command resolution. Local parser
Expand Down Expand Up @@ -570,10 +558,12 @@ the same conservative invalidation unless its identity is independently proved.
An ambiguous changed name, wildcard, or candidate-set overflow collapses to all
unproved command names rather than guessing.

The optional inbox `Microsoft.PowerShell.ThreadJob\Start-ThreadJob` follows
the initialization/main child-runspace model only when the caller's pinned
module baseline proves that identity. Otherwise it follows the unknown
receiver rule.
Optional-module `Start-ThreadJob` and deferred breakpoint, event, and argument-
completion receivers are not stable-v0.3 catalog-completeness promises.
Existing conservative recognition may remain, but additional module/version or
trigger-time proof does not gate the release. Every unproved form follows the
unknown-receiver rule: completely parsed bodies remain visible with incomplete
execution and state facts.

Aliases, supported module-qualified spellings, static call-operator spellings,
parameter abbreviations and inline values, positional binding, parameter-set
Expand All @@ -585,26 +575,21 @@ runtime order.

`ExecutionRegionTiming` and `ExecutionRegionCardinality` are not scope facts.
Variable, location, command-resolution, runspace, and process propagation are
analyzed independently. Deferred actions remain authorization-visible at
registration, but relative paths use trigger-time Unknown cwd unless another
proof exists. A constrained canonical `Write-Output { Remove-Item x }`
analyzed independently. Cataloged receivers may retain already-proved
scheduling facts without making further catalog expansion release-gating.
Unproved receiver, binding, or state facts remain Unknown rather than borrowing
registration-time state. A
constrained canonical `Write-Output { Remove-Item x }`
remains opaque data and does not invent a `Remove-Item` occurrence.

A leading `param(...)` declaration inside any execution region remains outside
the stable-v0.3 body grammar and makes the whole parse unparseable. This
deliberately limits realistic argument-completer and directly invoked blocks
until parameter declaration and block-argument binding are modeled together.

`condition_pipeline` is limited to a pipeline the existing parser can delimit
completely. A subexpression, member invocation, script block, or other form
that may execute outside complete command discovery makes the whole result
unparseable. Missing delimiters and every unsupported executable region also
produce empty `Commands` and `Clauses`; partial `Syntax` is diagnostic only.

PowerShell scope and location state remain shell-specific. Grouping `( ... )`
does not isolate location. Branch exits retain an exact cwd only when every
supported alternative agrees; disagreement becomes `Unknown`. Loop exits
include the zero-iteration state. The parser does not publish a finite cwd set.
does not isolate location. Foreach exits include the zero-iteration state. The
parser does not publish a finite cwd set.

`foreach` assignments use a case-insensitive persistent binding map rather than
lexical push/pop restoration. A proved nonempty ordered iterable leaves its
Expand Down Expand Up @@ -635,9 +620,9 @@ clear those resolutions and retain the `<dynamic-cwd>` marker. Decoded child
hosts retain inherited invocation-cwd attribution on their compatibility
leaves while isolating child exit state.

Stable v0.3 continues to defer `do`, `switch`, functions, definitions,
class/type bodies, and arbitrary execution-bearing expressions outside the
bounded forms above.
Stable v0.3 continues to defer `while`, `if`, `elseif`, `else`, `do`, `switch`,
functions, definitions, class/type bodies, and arbitrary execution-bearing
expressions outside the bounded forms above.

---

Expand Down Expand Up @@ -1452,7 +1437,7 @@ in **`SPEC.md` §11**.
here-string, unterminated `<# ... #>` block comment, unbalanced `{ }` /
`$( )` / `@( )` / `@{ }`, unbalanced grouping `( )`.
2. **Unsupported control-flow at statement/verb position** — `switch`, `for`,
`do`, `until`, or an `if`, `elseif`, `else`, `foreach`, or `while` form
`do`, `until`, `if`, `elseif`, `else`, or `while`, plus any `foreach` form
outside the bounded stable-v0.3 grammar in §4.
3. **Definition keywords** — `function`, `filter`, `workflow`,
`configuration`, `class`, `enum`.
Expand Down Expand Up @@ -1815,8 +1800,8 @@ v0.2.0 ships when **all** of these hold:

## 18. Out of Scope

- PowerShell control flow outside stable v0.3's bounded `foreach`, `while`,
and `if` subsets — including `do` and `switch`.
- PowerShell control flow outside stable v0.3's bounded `foreach` subset,
including `while`, `if`, `elseif`, `else`, `do`, and `switch`.
- `function`/`filter`/`class`/`enum` definitions,
`param()`/`begin`/`process`/`end` blocks, `trap`, and `DATA`.
- `.ps1` script-file parsing.
Expand Down
Loading