Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,18 @@ priorities.
> below). What remains is the downstream Netclaw integration, which needs
> actions outside this repository.

- [ ] **v0.3 prerelease consumer-API correction.** Preserve the stable v0.2
- [x] **v0.3 prerelease consumer-API correction.** Preserve the stable v0.2
API and the `Syntax` / `Commands` / `Clauses` ingestion lanes, but replace
the alpha-only sparse coordinate and public property-bag model before
stable v0.3. Return one parser-owned analyzed argument per authored
non-cwd argument, use closed value-domain and redirect alternatives,
reference actual ancestor nodes, and remove public syntax vocabulary
that stable v0.3 never emits. Update the OpenSpec and source mocks first;
then update implementation, snapshots, corpus DTOs, consumer guide,
README, and Netclaw together. No compatibility shim for 0.3 alphas.
README, and Netclaw together. No compatibility shim for 0.3 alphas. The
library, specifications, snapshots, corpus DTOs, generated expectations,
README, consumer guide, and prerelease migration notes are synchronized;
Netclaw migration remains the next downstream item.

- [x] **v0.3 host-selected grammar and PowerShell dialect — library slice.** The executor
selects one top-level parser; Bash never cross-parses `pwsh` payloads and
Expand All @@ -37,7 +40,7 @@ priorities.
PowerShell 7.6.4 oracle on Ubuntu and Windows, plus native Windows
PowerShell 5.1 discovery and its dialect-routed oracle on Windows.

- [ ] **v0.3 native-Windows Netclaw integration.** Pass the exact selected
- [x] **v0.3 native-Windows Netclaw integration.** Pass the exact selected
shell through Netclaw's executor, approval policy, and model context;
prefer a compatible `pwsh.exe`, fall back to `powershell.exe`, and
reparse and reauthorize if executable selection changes.
Expand Down Expand Up @@ -132,7 +135,10 @@ priorities.
- [x] **Consumer guide.** Added `docs/CONSUMER_GUIDE.md` with the
shell-neutral security-consumer algorithm, Bash and PowerShell guidance,
worked public use cases, and immutable permalinks to Netclaw's production
integration. Linked it from the README and aligned stale PowerShell
integration. Added compact input-to-result-to-policy examples for command
occurrences, attached arguments, bounded and zero-or-more loops, cwd
propagation, file and descriptor redirects, substitutions, and safe-fail
results. Linked it from the README and aligned stale PowerShell
prerelease/status wording in the public project docs.
- [x] **Issue #52 — hyphenated PowerShell parameters/native options.**
Preserve internal hyphens, apply bash-compatible native
Expand All @@ -157,9 +163,9 @@ priorities.

### 16. Netclaw v0.2.0 integration — SPEC.PWSH §17 #9

- [ ] Netclaw consumes the v0.2.0 package; absorbs the `Clause` rename
- [x] Netclaw consumes the v0.2.0 package; absorbs the `Clause` rename
*(separate repository — cannot be done here)*
- [ ] ≥1 Netclaw integration test exercises a real PowerShell corpus entry
- [x] ≥1 Netclaw integration test exercises a real PowerShell corpus entry
through the live matcher and gets the expected gate decision

### 17. v0.3 structured shell analysis contract — issue #72
Expand Down Expand Up @@ -489,7 +495,7 @@ priorities.
effects, publishes synchronous/once only for one proved target, and
publishes concurrent timing for multiple targets or enabled asynchronous
switches while keeping dynamic cardinality fail closed.
- [ ] Deliver typed PowerShell script-block execution regions before calling
- [x] Deliver typed PowerShell script-block execution regions before calling
tasks 7.5-7.7 complete. The corrected contract adds an execution-region
syntax node with independent origin, phase, timing, and cardinality rather than a
false shared/isolated scope flag. The inert additive public API skeleton,
Expand Down Expand Up @@ -615,7 +621,7 @@ priorities.
independence, semantic phase order, child process/runspace boundaries,
boundaries and the fact that the in-process `Invoke-Command` parameter
set does not support `-AsJob`.
- [ ] Complete the stable-v0.3 Bash `for ... in` and PowerShell `foreach`
- [x] Complete the stable-v0.3 Bash `for ... in` and PowerShell `foreach`
vertical slices without gating release on a shared-analysis refactor.
- [x] Publish `0.3.0-alpha` for the Netclaw migration gate. The bare SemVer tag
published the NuGet package, symbol package, and GitHub prerelease from
Expand Down Expand Up @@ -695,7 +701,7 @@ priorities.
and reserved-form boundaries, including unmodeled `time`, negation,
coprocess, and current-shell brace-group syntax, which fail closed rather
than flattening nested execution into apparent ordinary verb chains.
- [ ] Promote the Bash command-resolution mutation cases into Netclaw's strict
- [x] Promote the Bash command-resolution mutation cases into Netclaw's strict
allow/prompt/deny matrix before the downstream approval-fatigue gate.
- [x] Add the separately tested Bash `<<<` here-string redirect slice with
bounded operand analysis and trailing-newline semantics. Default and
Expand Down
14 changes: 10 additions & 4 deletions PROJECT_CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,18 @@ zero-native-deps .NET parser sized to what security gates actually need.
the dialect-matched live oracle matrix (`pwsh` for PowerShell 7 and
`powershell.exe` for Windows PowerShell 5.1 on Windows CI).

### v0.3 (contract design)
### v0.3 (implementation and downstream acceptance)

- Add a closed, strongly typed syntax-node hierarchy while retaining existing
`Clause` leaves.
- Add a library-owned command-occurrence projection for security consumers so
every potentially executable iterator, substitution, execution-region, and
body command in the supported grammar is evaluated exactly once.
- Return one parser-owned analyzed argument for every authored non-cwd
compatibility argument. Closed value and redirect record families use
runtime type as their discriminant; ancestry and execution-region facts
reference the actual parser-owned source objects. Compatibility is required
against stable v0.2, not experimental v0.3 alpha surfaces.
- Add typed PowerShell execution regions for direct call/dot-source blocks,
synchronous callbacks, jobs/parallel runspaces, and initialization. Public
origin/phase/timing/cardinality facts remain separate from
Expand All @@ -114,9 +119,10 @@ zero-native-deps .NET parser sized to what security gates actually need.
Existing conservative recognition may remain; unproved receivers still
expose completely delimited bodies as incomplete regions and therefore
remain fail closed.
- Treat `openspec/changes/v0-3-structured-shell-analysis/` and its paired design
corpus as the review authority until the accepted contract is synchronized
into `SPEC.md` and `SPEC.POWERSHELL.md` with the production API change.
- Treat `SPEC.md` and `SPEC.POWERSHELL.md` as the canonical synchronized
contract. The accepted OpenSpec change and paired design corpus retain the
design rationale and verification checklist until downstream acceptance and
archival.

### v0.2 explicit non-goals

Expand Down
44 changes: 35 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,27 +83,51 @@ foreach (var occurrence in parsed.Commands)
var clause = occurrence.Clause;
Console.WriteLine(
$"{occurrence.ImmediateRole} {clause.Verb.Joined} " +
$"complete={occurrence.IsComplete} cwd={occurrence.WorkingDirectory.Kind}");
$"complete={occurrence.IsComplete} " +
$"cwd={Describe(occurrence.WorkingDirectory)}");

foreach (var arg in clause.Args.Where(a => a.IsPath))
{
var marker = arg.IsCwdAttribution ? "↳ cwd" : " path";
Console.WriteLine($" {marker}: {arg.Resolved}");
}

foreach (var effective in occurrence.EffectiveArguments)
foreach (var argument in occurrence.Arguments)
{
Console.WriteLine(
$" element[{effective.ClauseElementIndex}]: {effective.Value.Kind}");
$" {argument.Argument.Raw}: {Describe(argument.Value)}");
}

foreach (var redirect in occurrence.Redirects)
{
Console.WriteLine(
$" {redirect.Operation}: {redirect.Target.Kind} " +
$"complete={redirect.IsComplete}");
Console.WriteLine($" {Describe(redirect)}");
}
}

static string Describe(ShellValueDomain value) => value switch
{
ShellValueDomain.Exact exact => exact.Value,
ShellValueDomain.FiniteSet finite => string.Join(" | ", finite.Values),
ShellValueDomain.PathPattern pattern => pattern.Pattern,
ShellValueDomain.Unknown => "unknown",
_ => "unknown",
};

static string Describe(RedirectAnalysis redirect) => redirect switch
{
FileRedirectAnalysis file =>
$"{file.Mode} to {Describe(file.Target)} complete={file.IsComplete}",
DescriptorDuplicateRedirectAnalysis duplicate =>
$"duplicate to fd {duplicate.TargetDescriptor}",
DescriptorMoveRedirectAnalysis move =>
$"move to fd {move.TargetDescriptor}",
DescriptorCloseRedirectAnalysis => "close descriptor",
HereDocumentRedirectAnalysis => "heredoc stdin data",
HereStringRedirectAnalysis hereString =>
$"here-string stdin data: {Describe(hereString.Data)}",
UnresolvedRedirectAnalysis => "unresolved redirect",
_ => "unknown redirect",
};
```

## Consumer guide
Expand Down Expand Up @@ -131,9 +155,11 @@ public enum PwshDialect { Unknown, PowerShell7, WindowsPowerShell51 }

public sealed record ParsedCommand { /* Source, Syntax, Commands, Clauses, IsUnparseable, … */ }
public abstract record ShellSyntaxNode;
public sealed record CommandOccurrence { /* Clause, role, ancestry, effective values, cwd, redirects, completeness */ }
public sealed record ShellValueDomain { /* Exact, FiniteSet, Pattern, or Unknown */ }
public sealed record RedirectAnalysis { /* source, operation, target, heredoc facts, completeness */ }
public sealed record CommandOccurrence { /* Clause, role, ancestry, analyzed arguments, cwd, redirects, completeness */ }
public sealed record AnalyzedArgument { /* direct Arg + ClauseElement + ShellValueDomain join */ }
public abstract record ShellValueDomain; // nested Unknown, Exact, FiniteSet, PathPattern
public abstract record RedirectSource; // nested Unknown, Default, Descriptor, PowerShellAllStreams
public abstract record RedirectAnalysis; // file, descriptor, heredoc, here-string, or unresolved alternative
public sealed record Clause { /* Operator, Verb, Args, Redirects, Elements, IsSubshell, IsCommandStringWrapped */ }
public sealed record ClauseElement { /* Raw, Value, Role, source span, verb-relative position, path facts */ }
public sealed record VerbChain { /* Tokens, Joined, CanonicalVerb, IsDynamic */ }
Expand Down
23 changes: 21 additions & 2 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,35 @@
#### Unreleased ####

## Changed

- Replace the experimental v0.3 sparse `EffectiveArguments` coordinate overlay
with one parser-owned `AnalyzedArgument` per authored non-cwd argument. Each
entry directly references its `Arg`, source `ClauseElement`, and effective
value, including many-to-one inline option bindings.
- Replace the experimental value, redirect-source, and redirect-operation
property bags with closed record families intended for runtime type
matching. Ancestry frames now reference actual syntax nodes, and execution
regions reference their actual host argument.
- Remove alpha-only public syntax kinds, condition/branch vocabulary, analysis
limits, and other shapes the stable parser never emits. Every new v0.3 result
is parser-owned and every read-only list introduced by v0.3 is defensively
backed; stable v0.2 construction and list semantics remain unchanged.

## Consumer migration

- v0.3 security consumers authorize `ParsedCommand.Commands` and use
`ParsedCommand.Syntax` only for display and diagnostics. The conservative
v0.2 `Clauses` projection remains supported throughout v0.3, including every
v0.3.x release; no removal version is scheduled.
- The additive v0.3 records and `ParsedCommand` members change generated record
- The new v0.3 records and `ParsedCommand` members change generated record
equality, hashing, `ToString()`, and reflection-based serialization output.
ShellSyntaxTree does not define a stable serialized wire format. Persisted
results require a consumer-owned, versioned DTO or explicit serializer
mapping that fails closed on unknown node and enum values.
mapping that fails closed on unknown runtime alternatives and enum values.
- No source or binary compatibility is provided for `0.3.0-alpha.*` packages.
Stable v0.2 remains the compatibility boundary. Alpha consumers must migrate
to `CommandOccurrence.Arguments` and pattern-match the closed value and
redirect families; no aliases or obsolete adapters preserve the old model.
- `PwshParserOptions.Dialect` is additive and defaults to `PowerShell7` for
compatibility. Native Windows consumers select it only for a compatible
PowerShell 7.6 host (`>=7.6.4` and `<7.7`) and select
Expand Down
12 changes: 7 additions & 5 deletions SPEC.POWERSHELL.md
Original file line number Diff line number Diff line change
Expand Up @@ -1292,14 +1292,16 @@ about which stream produced it:
| stream merge `N>&1` for `N` in `2`–`6`, or `*>&1` | `ErrOut` when `N` is `2`, else `Out`; `Target` carries `&1` verbatim with `IsDynamicSkip=true` |

The table above remains the v0.2 `Redirect` compatibility mapping. v0.3 also
populates `RedirectAnalysis`: `RedirectSourceKind.PowerShellAllStreams`
preserves `*`, `Descriptor` preserves numeric streams, and `Operation`
distinguishes file output/append from static descriptor duplication. Static
descriptor operations are not path-relevant. PowerShell's grammar does not
populates the closed `RedirectAnalysis` family:
`RedirectSource.PowerShellAllStreams` preserves `*`,
`RedirectSource.Descriptor` preserves numeric streams, and runtime alternatives
distinguish `FileRedirectAnalysis` output/append from static
`DescriptorDuplicateRedirectAnalysis`. Descriptor alternatives are not paths.
PowerShell's grammar does not
admit descriptor close, move, computed merge targets, or file input
redirection; those spellings make the whole parse unparseable. `$null` and
`${null}`
remains an incomplete explicit redirect until the public operation vocabulary
remain an incomplete explicit redirect until the public operation vocabulary
has a discard-sink representation; consumers must continue to fail closed.

---
Expand Down
Loading