Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,16 @@ priorities.
> below). What remains is the downstream Netclaw integration, which needs
> actions outside this repository.

- [ ] **v0.3 prerelease consumer-API correction.** Preserve the stable v0.2
API and the `Syntax` / `Commands` / `Clauses` ingestion lanes, but replace
the alpha-only sparse coordinate and public property-bag model before
stable v0.3. Return one parser-owned analyzed argument per authored
non-cwd argument, use closed value-domain and redirect alternatives,
reference actual ancestor nodes, and remove public syntax vocabulary
that stable v0.3 never emits. Update the OpenSpec and source mocks first;
then update implementation, snapshots, corpus DTOs, consumer guide,
README, and Netclaw together. No compatibility shim for 0.3 alphas.

- [x] **v0.3 host-selected grammar and PowerShell dialect — library slice.** The executor
selects one top-level parser; Bash never cross-parses `pwsh` payloads and
PowerShell never cross-parses `bash -c` payloads. Add the extend-only
Expand Down
8 changes: 5 additions & 3 deletions PROJECT_CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,9 +152,11 @@ and IDE-grade concrete syntax remain non-goals.
## Architectural Constraints

- **Public API in `SPEC.md` §2 is the contract.** Everything else is
`internal`. During `0.x`, renaming or removing public fields requires a
deliberate minor version bump and migration notes; after `1.0`, it requires
a major version bump.
`internal`. Stable `0.x` releases use the minor version as their breaking
boundary. Prerelease APIs may change before their corresponding stable
release and receive explicit prerelease migration notes; they do not force a
second minor-version bump. After `1.0`, breaking changes require a major
version bump.
- **`IShellParser` is the multi-shell seam.** Additional parsers such as
Windows `cmd` must be addable without reshaping consumer code.
- **No native dependencies.** AOT-trim friendly; ship a single managed
Expand Down
701 changes: 389 additions & 312 deletions openspec/changes/v0-3-structured-shell-analysis/design.md

Large diffs are not rendered by default.

21 changes: 16 additions & 5 deletions openspec/changes/v0-3-structured-shell-analysis/proposal.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ fail-closed behavior for incomplete analysis.
- Add a library-owned command-occurrence projection containing every command
that may execute in supported grammar, including iterator, loop-body,
wrapped, substitution, and PowerShell script-block execution-region commands.
- Make the unreleased v0.3 result model parser-owned and invalid-state-resistant.
Each authored argument is returned already joined to its v0.2 `Arg` and
`ClauseElement`; value domains, redirect sources, and redirect operations use
closed typed alternatives instead of public property bags. Prune public
condition/branch vocabulary that stable v0.3 never emits.
- Define authorization completeness over authored shell syntax. A complete
occurrence proves that the parser discovered and classified the submitted
executable region; it does not prove the runtime executable selected by
Expand Down Expand Up @@ -70,6 +75,10 @@ fail-closed behavior for incomplete analysis.
that check `IsUnparseable` continue to fail closed and do not silently miss
nested executable commands. Unparseable results expose no command or clause
authorization projection.
- Preserve compatibility with stable v0.2, not with any v0.3 prerelease.
Every `0.3.0-alpha*` package was an integration preview;
their new public members may be renamed, removed, or reshaped before stable
`0.3.0`. Netclaw migrates in lockstep to the corrected prerelease.
- Expand grammar in vertical slices through Bash `for ... in` and PowerShell
`foreach`. Preserve the existing Bash
heredoc grammar while adding body, delimiter, and expansion facts, and add
Expand Down Expand Up @@ -119,12 +128,14 @@ into `SPEC.md` and `SPEC.POWERSHELL.md` before implementation.

## Impact

This is an additive but release-shaped public API change affecting
`ParsedCommand`, new syntax and analysis records, redirect modeling, both
This is an additive change from stable v0.2 and a deliberately breaking
correction from the v0.3 prereleases. It affects `ParsedCommand`, new syntax
and analysis records, redirect modeling, both
shell parsers, corpus schemas, public API snapshots, the shared and
PowerShell specifications, and `docs/CONSUMER_GUIDE.md`. Adding properties to
public records also changes generated equality, hashing, `ToString()`, and
default serialization and therefore requires explicit migration notes.
PowerShell specifications, and `docs/CONSUMER_GUIDE.md`. Stable v0.2 types and
members remain source and binary compatible. The corrected v0.3 result types
are library-owned in-memory models, not consumer-constructible DTOs or a stable
serialized wire format.

This accepted scope supersedes the earlier assumption that every ordinary
PowerShell script-block argument is non-executing. Canonical receivers proved
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -305,8 +305,8 @@ reserved syntax.
- **THEN** the whole result is unparseable until coprocess structure and timing are modeled

### Requirement: PowerShell approvals prove authored commands, not ambient resolution
`PwshParserOptions.InitialStateMode` SHALL default to `Unknown` and SHALL remain
source and binary compatible. Neither that default nor an ambient alias,
`PwshParserOptions.InitialStateMode` SHALL default to `Unknown`. Neither that
default nor an ambient alias,
function, module, profile, executable lookup, or inherited variable SHALL by
itself make a static authored command occurrence incomplete. The parser SHALL
prove that it discovered the executable syntax submitted by the caller; it
Expand Down Expand Up @@ -434,10 +434,11 @@ The analysis SHALL classify a policy-relevant shell value as exact, finite,
bounded symbolic pattern, or unknown, and SHALL NOT present a weaker proof as a
stronger domain.

`Unknown` SHALL contain no values or pattern fields. `Exact` SHALL contain one
value. `FiniteSet` SHALL contain 2–32 distinct values. `Pattern` SHALL contain
no values and SHALL contain a non-empty pattern and covering directory. The
parser SHALL NOT emit any other member combination.
The public domain SHALL be a closed, library-constructed hierarchy.
`Unknown` SHALL carry no payload. `Exact` SHALL contain one value. `FiniteSet`
SHALL contain 2–32 distinct values. `PathPattern` SHALL contain a non-empty
pattern and covering directory. No public property bag or kind enum SHALL let
a consumer construct contradictory member combinations.

#### Scenario: One literal value
- **WHEN** an eligible isolated-mode loop binds a variable from the single literal `a.txt`
Expand Down Expand Up @@ -479,16 +480,18 @@ becomes unknown rather than being truncated.
### Requirement: Structural analysis has fixed depth limits
The parser SHALL support at most 16 nested executable containers and at most 5
decoded command-string wrapper recursions. Structural depth starts at zero for
the root and increments once when entering a foreach loop, condition loop,
conditional, group, command substitution, or execution region. Blocks, conditional-branch
records, command lists, pipelines, and simple-command leaves do not increment
the depth independently. These bounds SHALL NOT be caller-configurable.
the root and increments once when entering a foreach loop, group, command
substitution, or execution region. Blocks, command lists, pipelines, and
simple-command leaves do not increment the depth independently. Future
container types SHALL define their participation when their grammar is added;
they do not reserve stable-v0.3 public vocabulary. These bounds SHALL NOT be
caller-configurable.
Exceeding either bound SHALL make the whole result unparseable rather than
returning an authorization projection for a subset.

The limits SHALL be exposed as static get-only properties rather than public
compile-time constants so downstream assemblies read the installed parser's
contract instead of inlining stale values.
The limits SHALL be documented fixed parser contracts but SHALL NOT be exposed
as public tuning options, constants, or properties. Consumers react to the
resulting unknown or unparseable fact rather than reimplementing the bounds.

#### Scenario: Structural nesting reaches the limit
- **WHEN** a supported input enters exactly 16 nested executable containers
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -157,13 +157,25 @@ finite shell proof SHALL NOT itself grant authorization.
### Requirement: v0.2 consumers have a documented migration path
Release notes and the consumer guide SHALL document the new canonical
projections, retained compatibility fields, record equality and serialization
effects, and the period during which `Clauses` remains supported.
effects, and the period during which `Clauses` remains supported. Stable v0.2
source and binary compatibility SHALL remain the binding baseline. Documented
behavioral differences, including `ParsedCommand` record equality, hashing,
`ToString()`, and explicit oracle-proved classification corrections, SHALL NOT
be mislabeled as behavior-preserving.
No compatibility SHALL be promised between v0.3 prereleases; the migration
guide SHALL map the removed alpha-only members directly to the corrected
stable-v0.3 shapes without shims.

#### Scenario: Consumer remains on Clauses during alpha
- **WHEN** a consumer upgrades to a v0.3 prerelease without adopting `Commands`
- **THEN** existing simple-command behavior remains available
- **THEN** supported nested constructs expose conservative authored clauses rather than omitting commands

#### Scenario: Alpha consumer adopts the corrected result model
- **WHEN** a consumer used sparse effective-argument coordinates or redirect/value property bags from an earlier v0.3 alpha
- **THEN** it migrates to joined arguments and closed typed alternatives
- **THEN** ShellSyntaxTree does not retain obsolete aliases or adapters for the prerelease surface

#### Scenario: Netclaw adopts v0.3 analysis
- **WHEN** Netclaw migrates to the occurrence and explicit redirect APIs
- **THEN** integration tests cover ordinary commands, static fd operations, bounded loops, and unknown-value fallback
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ of whether the command is top-level or nested.
- **WHEN** a selected Bash or PowerShell corpus entry is marked for structural verification
- **THEN** the corpus records the complete syntax tree and command-occurrence collection in authored order
- **THEN** every simple-command node and occurrence references the exact compatibility clause by index and object identity
- **THEN** roles, completeness, ancestry coordinates, and exact-or-null source ranges are compared without weakening legacy corpus entries that omit structural expectations
- **THEN** roles, completeness, joined arguments, ancestor identities, child indices, and exact-or-null node source ranges are compared without weakening legacy corpus entries that omit structural expectations

### Requirement: Occurrences identify structural execution roles
Each command occurrence SHALL identify its immediate structural execution role
Expand All @@ -23,23 +23,21 @@ grouping.

`CommandOccurrenceRole.Unknown` and `CommandAncestryRegion.Unknown` SHALL be
their enum zero values. Ancestry SHALL be ordered outermost to innermost,
exclude the simple-command leaf, and retain child indices and exact-or-null
source ranges for correlation.
exclude the simple-command leaf, and retain the actual ancestor node plus a
child index. A frame SHALL NOT copy a kind or source range that can disagree
with that node.

Each frame SHALL describe the relationship from its ancestor to the next node
on the path. The root block SHALL use `Root`; non-root blocks and command lists
SHALL use `Statement`; pipelines SHALL use `PipelineStage`; groups SHALL use
`GroupBody`; foreach nodes SHALL use `Iterator` or `LoopBody`; condition loops
SHALL use `Condition` or `LoopBody`; conditionals SHALL use `Branch`;
conditional-branch nodes SHALL use `Condition` or `Branch`; and substitutions
SHALL use `Substitution`; execution regions SHALL use `ExecutionRegion`.
Repeated children SHALL use their zero-based authored
index, with an `else` child indexed after all conditional branches. Frame
source ranges SHALL identify the ancestor. Blocks, command lists, and groups
SHALL retain the incoming immediate role; a nearer pipeline, iterator, body,
condition, branch, substitution, or execution-region relation SHALL replace it.

#### Scenario: Root and nested block coordinates are deterministic
`GroupBody`; foreach nodes SHALL use `Iterator` or `LoopBody`; substitutions
SHALL use `Substitution`; and execution regions SHALL use `ExecutionRegion`.
Condition loops and branches are not part of stable v0.3. Repeated children
SHALL use their zero-based authored index. Blocks, command lists, and groups
SHALL retain the incoming immediate role; a nearer pipeline, iterator, body, substitution, or
execution-region relation SHALL replace it.

#### Scenario: Root and nested ancestry is deterministic
- **WHEN** a root statement contains a loop-body pipeline
- **THEN** a stage occurrence has outer-to-inner `Root`, `LoopBody`,
`Statement`, and `PipelineStage` ancestry
Expand All @@ -50,13 +48,12 @@ The projector SHALL accept only a tree with one syntax-node and one `Clause`
reference per authored simple-command position. Node and source-fragment spans
SHALL be both unavailable or a non-negative start/length pair. Structural enum
values consumed by projection SHALL be known. Empty blocks MAY be valid, but
empty pipelines, command lists, and conditionals SHALL be rejected.
empty pipelines and command lists SHALL be rejected.

Value domains, cwd facts, effective-argument coordinates, redirect
coordinates, redirect shapes, and heredoc facts SHALL satisfy their locked
record invariants before projection succeeds. Any repeated identity, malformed
shape, invalid coordinate, cycle, or depth overflow SHALL discard every
partial command and compatibility result.
Value domains, cwd facts, joined argument identities, redirect alternatives,
and heredoc facts SHALL satisfy their locked invariants before projection
succeeds. Any repeated identity, malformed shape, invalid join, cycle, or
depth overflow SHALL discard every partial command and compatibility result.

#### Scenario: Shared leaf identity is not counted twice
- **WHEN** an internal parser bug places one syntax leaf or `Clause` reference
Expand All @@ -66,10 +63,49 @@ partial command and compatibility result.

#### Scenario: Complete occurrence cannot contain invalid facts
- **WHEN** parser-owned analysis supplies an invalid value domain, argument
coordinate, redirect shape, or unknown scope-affecting structural kind
join, redirect alternative, or unknown scope-affecting structural relation
- **THEN** projection fails closed
- **THEN** the occurrence is not published with `IsComplete=true`

### Requirement: Occurrences expose fully joined authored arguments
Each command occurrence SHALL expose exactly one `AnalyzedArgument` for every
non-cwd-attribution compatibility `Arg`, in authored order. Each entry SHALL
reference that exact `Arg`, its exact argument-role `ClauseElement`, and one
closed shell-value domain. Static authored values SHALL normally be exact;
bounded visits MAY be finite or path-pattern values; unproved values SHALL be
unknown. Consumers SHALL NOT correlate a sparse coordinate back into
`Clause.Elements`.

The join SHALL permit multiple compatibility arguments to reference the same
authored element. Inline equals-form native options and PowerShell colon-bound
parameters can split one token into multiple `Arg` records; all split entries
SHALL remain present, ordered, and linked to that one `ClauseElement`.

#### Scenario: Static and loop-derived arguments share one ingestion path
- **WHEN** an occurrence contains a static option and a bounded loop variable
- **THEN** both appear once in `Arguments` in authored order
- **THEN** each entry directly exposes its compatibility argument, authored element, and effective value

#### Scenario: Synthetic cwd attribution is not an authored argument
- **WHEN** the v0.2 clause carries a synthetic cwd-attribution `Arg`
- **THEN** it remains available through `Clause.Args`
- **THEN** it is absent from `Arguments` because `WorkingDirectory` is the canonical v0.3 cwd fact

#### Scenario: Inline option creates a many-to-one authored join
- **WHEN** a supported inline option produces an option `Arg` and an operand `Arg` from one authored token
- **THEN** two analyzed arguments reference their two exact compatibility arguments
- **THEN** both analyzed arguments reference the same exact authored `ClauseElement`
- **THEN** each analyzed value describes its corresponding split `Arg`, not the unsplit element text

### Requirement: Published collections are not consumer-mutable
Every v0.3 `IReadOnlyList<T>` SHALL use an immutable backing collection or a
defensive copy. The library SHALL NOT expose an array or mutable list that a
consumer can cast and mutate after a successful projection.

#### Scenario: Caller retains a parser input collection
- **WHEN** internal lowering receives a mutable collection and publishes a v0.3 fact
- **THEN** later mutation of the original collection cannot change the parsed result

#### Scenario: Dynamic Bash command string remains incomplete
- **WHEN** Bash parses a `bash` or `sh` clause with dynamic wrapper-control input, decoded or combined command-string options, or an expanding quoted body that does not match the complete literal exactly-one wrapper production
- **THEN** the existing outer compatibility leaf remains visible with direct source provenance
Expand Down
Loading