Status: v0.2.0 shipped; the accepted v0.3 contract adds structured syntax,
complete command occurrences, explicit redirect analysis, bounded for /
foreach, substitutions, and execution regions while retaining the v0.2
compatibility leaves.
Audience: Whoever (human or agent) works on ShellSyntaxTree.
Read this end-to-end before writing any code.
PowerShell support is specified separately in SPEC.POWERSHELL.md (v0.2.0);
this document is the canonical home of the public API, AST, sanitization
workflow, and consumer contract that PowerShell reuses.
This document specifies the shared public API, AST, Bash grammar, verb tables, resolver semantics, and corpus contract through ShellSyntaxTree v0.3. The library is a focused bash command parser designed for security gate evaluators — tools that inspect agent-emitted shell commands to decide whether to allow, prompt for, or deny execution.
It is not a general-purpose shell interpreter. It does not execute, expand, or evaluate commands. It returns a structured AST that consumers walk to make decisions.
The original consumer is Netclaw's approval policy. The library is designed to be reusable beyond Netclaw — any tool that needs to reason about the shape of an agent-emitted bash command can consume it.
- Parse bash commands into a structured AST with per-clause verbs, args, redirects, and compound operators.
- Extract paths a command operates on with per-verb knowledge of which
positional args are paths vs flags vs literal values (
chmod 755 fileknows755is a mode). - Honor
cd <dir> && cmdpropagation within a compound —<dir>counts as a path each subsequent command operates on. - Recurse into
bash -c "<inner>"so the inner command is parsed and its clauses surface to the consumer. - Mark dynamic-content tokens (unresolved
$VAR, unexpanded globs) so consumers don't misextract literal$VAR/fooas a path. - Multi-shell-ready via
IShellParserinterface — bash is the only v0.1 implementation; PowerShell and cmd are deferred to later versions without breaking the seam.
- PowerShell parsing (deferred; interface seam is present).
- Windows cmd parsing (deferred).
- Command execution. The library never runs anything.
- Variable expansion. We mark dynamic tokens, never resolve them.
- Function definitions, here-docs body extraction, complex parameter
expansion (
${var//pattern/replacement}), arithmetic expansion. - Command-substitution evaluation. The library never executes a substitution
or claims its produced value is known. Stable v0.3 recursively discovers
commands inside supported Bash
$()positions while retaining the authoredKind=DynamicSkip, IsPath=falsecompatibility value. Legacy backticks and incomplete executable interiors fail closed. - Performance tuning beyond "fast enough to invoke per shell call without noticeable latency" (~1ms per typical input).
The execution environment selects exactly one top-level parser. Consumers use
BashParser only when Bash will execute the submitted source and PwshParser
only when PowerShell will execute it. Neither parser guesses a language from
command text or delegates an argument payload to the other parser.
Therefore Bash input such as pwsh -Command 'Get-Content x' remains one
ordinary external pwsh command with a Bash argument; it does not surface a
PowerShell child command. PowerShell input such as bash -c 'rm x' likewise
remains one ordinary external bash command. Same-language wrapper recursion
remains parser-local: Bash owns supported bash / sh -c recursion, while
PowerShell owns its PowerShell-host and static Invoke-Expression recursion.
The library never auto-detects the host shell or probes the machine.
The package exposes a small surface from a single namespace
ShellSyntaxTree. Public types only:
namespace ShellSyntaxTree;
/// <summary>
/// Parses shell command strings into structured ASTs.
/// </summary>
public interface IShellParser
{
/// <summary>
/// Parse the command. Always returns a ParsedCommand; sets
/// <see cref="ParsedCommand.IsUnparseable"/> when the input cannot
/// be tokenized (unbalanced quotes, etc.). Never throws on
/// well-formed strings; throws ArgumentNullException on null input.
/// </summary>
ParsedCommand Parse(string command);
}
/// <summary>Bash implementation of IShellParser.</summary>
public sealed class BashParser : IShellParser
{
public BashParser();
public BashParser(BashParserOptions options);
public ParsedCommand Parse(string command);
}
/// <summary>PowerShell implementation of IShellParser (v0.2.0). The
/// PowerShell grammar, tables, and resolver are specified in
/// SPEC.POWERSHELL.md.</summary>
public sealed class PwshParser : IShellParser
{
public PwshParser();
public PwshParser(PwshParserOptions options);
public ParsedCommand Parse(string command);
}
/// <summary>Shell-neutral resolver configuration shared by every parser
/// (added v0.2.0). HomeDirectory / WorkingDirectory live here.</summary>
public abstract record ShellParserOptions { ... }
/// <summary>Declares which ambient Bash variable facts the caller can prove.</summary>
public enum BashInitialStateMode
{
Unknown,
IsolatedNonInteractive,
}
/// <summary>Configuration knobs for BashParser. As of v0.2.0 a sealed
/// record deriving from ShellParserOptions; the v0.1 object-initializer
/// shape is unchanged.</summary>
public sealed record BashParserOptions : ShellParserOptions
{
public BashInitialStateMode InitialStateMode { get; init; }
}
/// <summary>Compatibility option for PowerShell initial host-state analysis.</summary>
public enum PwshInitialStateMode
{
Unknown,
IsolatedNonInteractiveNoProfile,
}
/// <summary>Selects the PowerShell grammar and versioned metadata.</summary>
public enum PwshDialect
{
Unknown,
// PowerShell 7.6 servicing releases from 7.6.4; versioned tables are pinned.
PowerShell7,
WindowsPowerShell51,
}
/// <summary>Configuration knobs for PwshParser.</summary>
public sealed record PwshParserOptions : ShellParserOptions
{
public PwshInitialStateMode InitialStateMode { get; init; }
public PwshDialect Dialect { get; init; } = PwshDialect.PowerShell7;
}
`PwshDialect` and `PwshParserOptions.Dialect` are source- and binary-additive.
The property initializer preserves the released PowerShell 7 parser semantics
for existing constructors and object initializers. Like every additive public
record property, it deliberately changes generated equality, hashing,
`ToString()`, reflection, and default serializer shape; parser results and
options are not a stable implicit wire format.
// The pre-v0.2.0 BashParserOptions body, now hoisted onto ShellParserOptions:
public abstract record ShellParserOptions
{
/// <summary>
/// User home directory used to expand `~` and `$HOME` tokens during
/// resolution. Defaults to <see cref="Environment.SpecialFolder.UserProfile"/>.
/// </summary>
public string? HomeDirectory { get; init; }
/// <summary>
/// Working directory used to resolve relative path tokens during
/// resolution. Defaults to the daemon-process cwd.
/// </summary>
public string? WorkingDirectory { get; init; }
}
// v0.2 compatibility leaves — see §3.
public sealed record ParsedCommand { ... }
public sealed record Clause { ... }
public sealed record ClauseElement { ... }
public sealed record VerbChain { ... }
public sealed record Arg { ... }
public sealed record Redirect { ... }
public enum ClauseElementRole { Verb, Argument, Redirect }
public enum ArgKind { Literal, EnvVar, Glob, Tilde, DynamicSkip }
public enum RedirectDirection { In, Out, Append, ErrOut, ErrAppend }
public enum CompoundOperator { None, AndIf, OrIf, Sequence, Pipe }
// v0.3 authored structure — see §3.
public abstract record ShellSyntaxNode { ... }
public sealed record ShellBlockSyntax : ShellSyntaxNode { ... }
public sealed record SimpleCommandSyntax : ShellSyntaxNode { ... }
public sealed record PipelineSyntax : ShellSyntaxNode { ... }
public sealed record CommandListSyntax : ShellSyntaxNode { ... }
public sealed record CommandListItemSyntax { ... }
public sealed record GroupSyntax : ShellSyntaxNode { ... }
public sealed record ForEachSyntax : ShellSyntaxNode { ... }
public sealed record ShellSourceFragment { ... }
public sealed record CommandSubstitutionSyntax : ShellSyntaxNode { ... }
public sealed record ExecutionRegionSyntax : ShellSyntaxNode { ... }
public enum ShellGroupKind { ... }
public enum ExecutionRegionOrigin { ... }
public enum ExecutionRegionPhase { ... }
public enum ExecutionRegionTiming { ... }
public enum ExecutionRegionCardinality { ... }
// v0.3 authorization and bounded-analysis projections — see §3.
public sealed record CommandOccurrence { ... }
public sealed record CommandAncestryFrame { ... }
public sealed record AnalyzedArgument { ... }
public abstract record ShellValueDomain { ... }
public abstract record RedirectAnalysis { ... }
public sealed record HereDocumentAnalysis { ... }
public abstract record RedirectSource { ... }
public sealed record FileRedirectAnalysis : RedirectAnalysis { ... }
public sealed record UnresolvedRedirectAnalysis : RedirectAnalysis { ... }
public sealed record DescriptorDuplicateRedirectAnalysis : RedirectAnalysis { ... }
public sealed record DescriptorMoveRedirectAnalysis : RedirectAnalysis { ... }
public sealed record DescriptorCloseRedirectAnalysis : RedirectAnalysis { ... }
public sealed record HereDocumentRedirectAnalysis : RedirectAnalysis { ... }
public sealed record HereStringRedirectAnalysis : RedirectAnalysis { ... }
public enum CommandOccurrenceRole { ... }
public enum CommandAncestryRegion { ... }
public enum FileRedirectMode { ... }
public enum HereDocumentExpansionMode { ... }Stable v0.3 exposes only structural types the parsers can emit. Condition-loop and branch grammar remains fail closed and reserves no public type or enum member. Every new v0.3 result type is parser-owned; its constructor and result setters are not public. The stable v0.2 constructors and setters are unchanged.
That's the entire public API. Everything else is internal. The lexer, parser internals, verb tables, resolver — all implementation detail.
The top-level result of parsing. Always returned (never null).
public sealed record ParsedCommand
{
/// <summary>The original input string, verbatim.</summary>
public string Source { get; init; } = "";
/// <summary>
/// Canonical authored nested structure. Direct-source nodes have exact
/// source ranges; decoded wrapper nodes report unavailable ranges unless
/// an exact outer mapping exists.
/// </summary>
public ShellBlockSyntax Syntax { get; internal init; } = new();
/// <summary>
/// Canonical authorization projection containing every authored simple
/// command that may execute exactly once, in deterministic source order.
/// </summary>
public IReadOnlyList<CommandOccurrence> Commands { get; internal init; } = [];
/// <summary>
/// Conservative v0.2 compatibility projection. Existing simple-command
/// behavior remains available, but v0.3 security consumers use Commands.
/// </summary>
public IReadOnlyList<Clause> Clauses { get; init; } = [];
/// <summary>
/// True when the parser could not account for every executable region.
/// When true, Commands and Clauses are empty; Syntax may contain partial
/// diagnostic evidence only. Consumers must prompt or deny.
/// </summary>
public bool IsUnparseable { get; init; }
/// <summary>
/// Human-readable diagnostic when IsUnparseable=true; null otherwise.
/// </summary>
public string? UnparseableReason { get; init; }
}BashInitialStateMode.Unknown is the default. In this mode the parser does
not publish bounded loop-variable facts: an ambient shell may already have
made the binding readonly, integer-valued, a nameref, exported, or otherwise
semantically significant. A Bash loop whose safety depends on such a binding
is therefore unparseable rather than being analyzed as an ordinary scalar.
BashInitialStateMode.IsolatedNonInteractive is an explicit caller assertion,
not a parser discovery. It means the complete source is executed by a newly
spawned non-interactive Bash process, no profile or BASH_ENV / ENV startup
content is loaded, and no inherited environment entry carries the loop-bound
name. A consumer may select this mode only when its execution path enforces
those conditions. Supplying this option while executing in a reused,
interactive, startup-scripted, or uncontrolled environment invalidates the
analysis.
Recognized variable-state mutation in the analyzed source invalidates isolated
mode for every later region that can observe it. In particular, a decoded
bash -c child after export is analyzed with unknown initial variable state;
the option is not blindly copied into the child. Cwd-only state changes retain
the caller's initial-variable assertion.
The same attribute-state proof governs every simple named-parameter
dereference. With BashInitialStateMode.Unknown, $name and ${name} are
unparseable because an ambient nameref can evaluate an arithmetic array
subscript and execute authored command text. In isolated mode, a fresh-process
variable before reachable source mutation may remain an unknown value while
still being proved free of recursive variable attributes. A modeled ordinary
loop binding retains its explicit proof. Positional and special parameters
that cannot carry variable attributes keep their existing typed cardinality
rules.
Even in isolated mode, the v0.3 bounded loop grammar accepts only ordinary
lowercase scalar binding names matching [a-z][a-z0-9_]*, excluding
auto_resume and histchars. _, uppercase names, and every name outside
that boundary fail the complete loop region closed. This deliberately excludes
Bash magic variables and resolver- or executable-identity-sensitive names such
as RANDOM, LINENO, HOME, PATH, CDPATH, and IFS. The boundary is
extend-only: a later version may add a proved variable-state model or
additional explicitly reviewed ordinary names.
PwshInitialStateMode.Unknown is the default. PowerShell authorization uses
the same boundary as Bash authorization: it proves the static authored command
and every authored executable region, not the runtime implementation selected
through aliases, functions, modules, profiles, executable lookup, inherited
variables, or other ambient host state. Ambient uncertainty alone therefore
does not make a static command occurrence incomplete and does not poison later
authored command occurrences.
PwshInitialStateMode.Unknown does not publish an exact or finite
loop-dependent effective value. An ambient typed, validated, read-only, or
constant binding may coerce or reject an assignment, so the authored iterable
text is not a proved runtime argument. The surrounding static command
occurrence may still be complete; value precision is a separate fact.
PwshInitialStateMode.IsolatedNonInteractiveNoProfile is a caller assertion
that the complete source runs in a newly spawned noninteractive PowerShell
process with profiles disabled and without a reused or caller-initialized
runspace. It permits exact or finite loop-dependent values for ordinary
unscoped names. It does not assert a pinned module, alias, function, PATH, or
executable-resolution baseline, because those runtime externalities are outside
authored-command completeness.
The bounded grammar remains limited to ordinary unscoped variable names that
do not collide, case-insensitively, with automatic, constant, read-only,
preference, or configuration bindings known to the supported PowerShell
runtime. Scoped/provider forms such as $global:x, $script:x, and $env:X
remain outside the bounded loop-binding grammar.
Current-runspace regions such as ( ... ), $(), and a static
Invoke-Expression payload share supported authored binding and location
state. A decoded pwsh -Command or pwsh -EncodedCommand child does not
inherit exact $HOME, environment, provider, or cwd facts unless those facts
are independently proved, but it retains complete static authored command
occurrences. Explicit native, .ps1, cmdlet, alias, and module-qualified
spellings use their authored parser classification even though runtime state
may shadow them.
Recognized source-level mutation of variables, aliases, functions, or modules
invalidates later affected proofs wherever PowerShell scope rules make the
mutation observable. A computed Invoke-Expression payload can hide commands
and mutate current-runspace facts; it therefore invalidates later binding and
cwd proofs and remains incomplete. Cwd-only state changes retain independent
authored-binding facts.
Variable mutation recognition includes argument-vector binding, not only the
invoked verb. The PowerShell common parameters -OutVariable / -ov,
-PipelineVariable / -pv, -ErrorVariable / -ev, -WarningVariable /
-wv, and -InformationVariable / -iv, including accepted unambiguous
prefixes and inline : values, invalidate later observing proofs. The same
rule covers PowerShell 7 variable-writing parameters on Tee-Object,
Import-LocalizedData, Invoke-RestMethod, and Invoke-WebRequest. An opaque
splat can supply any of those parameter keys and therefore also invalidates
later proofs. A recognized writer on Set-Location composes with its
success/failure cwd transfer and invalidates bindings on both reachable
outcomes; location analysis does not bypass argument-vector mutation. This
check is conservative for a command whose runtime command
type is unavailable; treating a possible native argument as a writer can
cause a prompt, but ignoring an advanced-function writer can authorize a stale
value.
PowerShell also accepts U+2013 EN DASH, U+2014 EM DASH, and U+2015 HORIZONTAL
BAR as parameter prefixes. Stable v0.3 fails a token beginning with one of
those alternate dashes atomically rather than exposing it as a literal
positional argument: the retained v0.2 Arg.IsFlag contract recognizes only
an ASCII -, so normalizing the authored Raw spelling would either lose
provenance or require a breaking API change. Unsupported module-qualified
cmdlets are likewise rejected inside structural regions, not only in a flat
command. The existing module-qualified Invoke-Expression exception and the
version-pinned PowerShell execution-region receiver catalog are the only
specified exceptions.
For a successful result, every authored simple command appears once in
Syntax, once in Commands, and once in Clauses, with all three projections
referencing the identical in-memory Clause instance. Serialization is not
required to preserve that reference identity. For an unparseable result,
Commands and Clauses are empty even when Syntax retains partial evidence
for diagnostics.
The syntax family is a closed, library-owned record hierarchy. The
private protected ordinary constructor is paired with an assembly-only
abstract ownership member because C# records also synthesize a protected copy
constructor; together they prevent an external concrete node implementation.
Later library versions may add node kinds, so
authorization code that inspects Syntax must fail closed on an unknown type
or enum value. Security consumers normally enumerate Commands; Syntax is
for structure, explanation, display, and specialized analysis.
public abstract record ShellSyntaxNode
{
private protected ShellSyntaxNode() { }
private protected abstract object LibraryOwnership { get; }
public int? SourceStart { get; internal init; }
public int? SourceLength { get; internal init; }
}
public sealed record ShellBlockSyntax : ShellSyntaxNode
{
internal ShellBlockSyntax() { }
private protected override object LibraryOwnership => this;
public IReadOnlyList<ShellSyntaxNode> Statements { get; internal init; } = [];
}
public sealed record SimpleCommandSyntax : ShellSyntaxNode
{
internal SimpleCommandSyntax() { }
private protected override object LibraryOwnership => this;
public Clause Clause { get; internal init; } = new();
public IReadOnlyList<CommandSubstitutionSyntax> Substitutions { get; internal init; } = [];
public IReadOnlyList<ExecutionRegionSyntax> ExecutionRegions { get; internal init; } = [];
}
public sealed record PipelineSyntax : ShellSyntaxNode
{
internal PipelineSyntax() { }
private protected override object LibraryOwnership => this;
public IReadOnlyList<ShellSyntaxNode> Stages { get; internal init; } = [];
}
public sealed record CommandListSyntax : ShellSyntaxNode
{
internal CommandListSyntax() { }
private protected override object LibraryOwnership => this;
public IReadOnlyList<CommandListItemSyntax> Items { get; internal init; } = [];
}
public sealed record CommandListItemSyntax
{
internal CommandListItemSyntax() { }
public CompoundOperator Operator { get; internal init; }
public ShellSyntaxNode Command { get; internal init; } = null!;
}
public sealed record GroupSyntax : ShellSyntaxNode
{
internal GroupSyntax() { }
private protected override object LibraryOwnership => this;
public ShellGroupKind GroupKind { get; internal init; }
public ShellBlockSyntax Body { get; internal init; } = null!;
}
public enum ShellGroupKind
{
Unknown,
CurrentScope,
IsolatedScope,
}
public sealed record ForEachSyntax : ShellSyntaxNode
{
internal ForEachSyntax() { }
private protected override object LibraryOwnership => this;
public string BindingName { get; internal init; } = "";
public ShellSourceFragment BindingSource { get; internal init; } = null!;
public ShellSourceFragment Iterable { get; internal init; } = null!;
public ShellBlockSyntax IteratorCommands { get; internal init; } = null!;
public ShellBlockSyntax Body { get; internal init; } = null!;
}
public sealed record ShellSourceFragment
{
internal ShellSourceFragment() { }
public string Raw { get; internal init; } = "";
public int? SourceStart { get; internal init; }
public int? SourceLength { get; internal init; }
}
public sealed record CommandSubstitutionSyntax : ShellSyntaxNode
{
internal CommandSubstitutionSyntax() { }
private protected override object LibraryOwnership => this;
public ShellBlockSyntax Body { get; internal init; } = null!;
}
public sealed record ExecutionRegionSyntax : ShellSyntaxNode
{
internal ExecutionRegionSyntax() { }
private protected override object LibraryOwnership => this;
public ExecutionRegionOrigin Origin { get; internal init; }
public ClauseElement? HostArgument { get; internal init; }
public ExecutionRegionPhase Phase { get; internal init; }
public ExecutionRegionTiming Timing { get; internal init; }
public ExecutionRegionCardinality Cardinality { get; internal init; }
public ShellBlockSyntax Body { get; internal init; } = null!;
}
public enum ExecutionRegionOrigin
{
Unknown,
DirectCall,
DotSource,
CommandArgument,
}
public enum ExecutionRegionPhase
{
Unknown,
Main,
Initialization,
Begin,
Process,
End,
Filter,
Action,
Completion,
}
public enum ExecutionRegionTiming
{
Unknown,
Synchronous,
Concurrent,
Deferred,
}
public enum ExecutionRegionCardinality
{
Unknown,
Once,
OncePerInputObject,
ZeroOrMore,
}ForEachSyntax shares proved execution structure only. Iterable.Raw keeps
the shell-specific authored expression; it does not claim that Bash words and
PowerShell expressions share a grammar. Direct-source nodes have exact ranges
into ParsedCommand.Source. Nodes lifted from decoded, escaped, or encoded
wrapper content use null ranges unless an exact outer mapping exists.
Every new v0.3 result-type constructor and new v0.3 member setter is
library-owned. Every IReadOnlyList<T> introduced by v0.3 is a defensive
snapshot, not a mutable array or list that a consumer can cast and change.
Stable v0.2 construction, setters, and list semantics remain unchanged.
Runtime type is the discriminant for each closed
record family; consumers keep a default fail-closed switch arm for future
library-owned alternatives. Compatibility is required against stable v0.2,
not against any experimental 0.3.0-alpha.* surface.
Every v0.3 enum whose model admits an unknown state reserves zero as
Unknown. Consumers fail closed on Unknown or an unrecognized numeric value
when the fact affects policy. FileRedirectMode has no Unknown member:
unresolved operations use UnresolvedRedirectAnalysis, and only the library
can construct a FileRedirectAnalysis with a known mode.
public sealed record CommandOccurrence
{
internal CommandOccurrence() { }
public Clause Clause { get; internal init; } = new();
public CommandOccurrenceRole ImmediateRole { get; internal init; }
public IReadOnlyList<CommandAncestryFrame> Ancestry { get; internal init; } = [];
public IReadOnlyList<AnalyzedArgument> Arguments { get; internal init; } = [];
public ShellValueDomain WorkingDirectory { get; internal init; } = null!;
public IReadOnlyList<RedirectAnalysis> Redirects { get; internal init; } = [];
public bool IsComplete { get; internal init; }
}
public enum CommandOccurrenceRole
{
Unknown,
Ordinary,
PipelineStage,
Iterator,
LoopBody,
Substitution,
ExecutionRegion,
}
public sealed record CommandAncestryFrame
{
internal CommandAncestryFrame() { }
public ShellSyntaxNode Ancestor { get; internal init; } = null!;
public CommandAncestryRegion Region { get; internal init; }
public int? ChildIndex { get; internal init; }
}
public enum CommandAncestryRegion
{
Unknown,
Root,
Statement,
PipelineStage,
GroupBody,
Iterator,
LoopBody,
Substitution,
ExecutionRegion,
}
public sealed record AnalyzedArgument
{
internal AnalyzedArgument() { }
public Arg Argument { get; internal init; } = null!;
public ClauseElement Element { get; internal init; } = null!;
public ShellValueDomain Value { get; internal init; } = null!;
}
public abstract record ShellValueDomain
{
private protected ShellValueDomain() { }
private protected abstract object LibraryOwnership { get; }
public sealed record Unknown : ShellValueDomain { ... }
public sealed record Exact : ShellValueDomain { public string Value { get; } }
public sealed record FiniteSet : ShellValueDomain
{
public IReadOnlyList<string> Values { get; }
}
public sealed record PathPattern : ShellValueDomain
{
public string Pattern { get; }
public string CoveringDirectory { get; }
}
}IsComplete proves that the parser discovered the complete authored
executable region, assigned its structural ancestry, and completed every
parser-owned authored-syntax check. It does not prove which runtime executable
an ambient alias, function, module, profile, PATH, or inherited environment
will select. Static authored command identities remain complete under that
external uncertainty. Computed identities, hidden executable text, and
unsupported regions remain incomplete or make the whole result unparseable.
After an explicit source-level mutation that the parser cannot model, affected
later identities or values remain incomplete.
Commands contains one entry per authored simple command that may execute,
not one per predicted runtime iteration. Ancestry is ordered outermost to
innermost, excludes the simple-command leaf, and retains every enclosing
execution relation. ImmediateRole describes the nearest relation.
SimpleCommandSyntax.Substitutions owns each completely delimited executable
command substitution evaluated for that command's authored words and redirects,
including an expanding heredoc body. The collection is in authored order and
preserves nesting: a substitution inside an inner simple command belongs to
that inner command, not to the outer command or a side table. Clause remains
the unchanged v0.2 compatibility leaf and retains the authored dynamic value.
SimpleCommandSyntax.ExecutionRegions owns each completely delimited body
that a proved or conservatively unknown command binding may execute. The
unchanged host Clause retains its authored script-block DynamicSkip
argument. A direct PowerShell & {} or . {} region appears as a statement,
carries Origin=DirectCall or Origin=DotSource, has
HostArgument=null, and creates no synthetic occurrence for the
invocation operator. A command-owned region carries Origin=CommandArgument
and HostArgument references the exact script-block ClauseElement in the
host command. Consumers use Origin, rather than reparsing source text, to
distinguish the different direct-invocation state semantics.
Execution-region origin, phase, timing, and cardinality are independent
structural facts. Attached regions retain authored script-block order even when a
shell-specific analyzer schedules Begin/Process/End or initialization/main in
another semantic order. Synchronous, Concurrent, and Deferred do not
claim variable, cwd, command-resolution, runspace, or process scope. Those
dimensions remain shell-specific analysis because PowerShell can isolate
ordinary variable assignment while sharing location. Unknown enum values fail
closed.
The canonical Commands and compatibility Clauses projections use these
deterministic ordering rules: disjoint executable regions follow authored
source order; an enclosed substitution precedes its containing command;
nested substitutions are emitted innermost first; a command-owned execution
region follows its host and sibling regions retain authored order; and nodes
without comparable
outer source spans use their containing structural collection order. Thus
rm "$(find /tmp)" projects find, then rm, exactly once each.
Each substitution ancestry frame uses Region=Substitution and its authored
zero-based ChildIndex in the structural collection that owns the
CommandSubstitutionSyntax. For a simple command this is its Substitutions
collection; for an iterator it is the containing iterator-command collection.
Each ancestry frame references the actual Ancestor node and describes its
relationship to the next node on the path. The root block uses Root; non-root blocks and command
lists use Statement; pipelines use PipelineStage; groups use GroupBody;
foreach nodes use Iterator or LoopBody; command substitutions use Substitution; and
execution regions use ExecutionRegion.
Repeated children use their zero-based authored index. Source ranges are read
from the referenced ancestor. Blocks, command lists, and groups retain the
incoming immediate role; pipeline stages, iterator/body regions,
substitutions, and execution regions replace it with their nearer execution role.
Projection accepts only a parser-owned tree: a syntax-node or Clause
reference cannot appear at two authored positions, node and fragment spans are
either both unavailable or a non-negative start/length pair, and structural
enum values consumed by the projector must be known. Empty blocks remain
valid, but empty pipelines and command lists are malformed.
Joined value domains, analyzed-argument references, cwd facts, redirect
coordinates, redirect shapes, and heredoc facts must satisfy their contracts.
Any violation discards the partial Commands and Clauses collections and
makes the outer parse unparseable; it is never published as a complete
occurrence.
Arguments contains exactly one AnalyzedArgument for each non-cwd
Clause.Args entry in authored order. Argument and Element reference the
existing compatibility objects directly. Inline equals/colon forms may create
multiple arguments that share one source element. The analysis never mutates a
compatibility Arg to hold loop-specific values. An occurrence can be
structurally complete while one value remains Unknown; completeness and
value precision are independent.
The parser emits only these value-domain combinations:
Unknown: no payload;Exact: exactly one non-null value;FiniteSet: 2–32 distinct non-null values;PathPattern: a non-empty pattern and non-empty covering directory.
The parser does not execute commands, inspect runtime variables, enumerate the
filesystem, or truncate an over-limit set and call it complete. A result with
33 or more candidates becomes Unknown. The candidate cap of 32, structural
depth cap of 16, and wrapper-recursion cap of 5 are parser contracts, not
public tuning knobs. Structural depth starts at zero for the root and
increments on foreach loops, groups, command substitutions, and execution
regions; blocks, lists, pipelines, and
simple-command leaves do not independently increment it. Exceeding 16
structural containers or 5 decoded-command wrapper recursions makes the entire
result unparseable.
Loop bindings are analyzer-owned shell state; they are not lexical parser
frames. A nonempty Bash loop leaves its final assigned value visible after
done, a loop that executes zero times preserves the incoming value, and a
same-name nested loop does not restore an outer value. v0.3 may continue to
reject nested active-name reuse until that overwrite behavior is implemented;
it must never model the construct as lexical shadowing.
The Bash front end retains every iterable word and every resolver-relevant argument fragment as parser-owned internal provenance. The abstract-state pass then evaluates the iterable once from its incoming variable state and creates one of these internal plans:
Neverfor an explicit empty iterable;- an ordered, duplicate-preserving sequence for at most 32 concrete iterations; or
ZeroOrMore/OneOrMorefixed-point analysis when cardinality or an ordered sequence is not bounded.
The public FiniteSet is only a value summary. It is never used as an
iteration plan: a b a performs three state transitions and leaves an exact
final binding of a; 33 authored values use widening even when every value is
the same. An iterable that depends on an outer binding is evaluated separately
for each concrete outer visit so correlated nested state is not flattened into
an artificial cross-product. The analyzer permits at most 4096 total loop-body
transitions per parse; exceeding that resource budget makes the complete result
unparseable rather than returning a partial cross-product.
Each concrete iteration assigns its candidate into the analyzer variable map,
re-evaluates the complete effective argument vector for every body occurrence,
and carries the joined reachable cwd and variable state into the next
iteration. This re-evaluation includes state-transfer option grammar. For
example, a loop-derived cd argument may become -P, --, -, or an
operand; the analyzer may not substitute only an operand string while retaining
authored flag classification. Effective argument facts at one authored
occurrence join the values from every reachable visit.
Bash flow retains separate reachable success and failure states. && analyzes
only a reachable success continuation, || only a reachable failure
continuation, and sequence operators consume their join. A missing partition
is unreachable and must not be replaced with the joined input merely to
populate exact facts. Structurally present but unreachable commands remain in
the syntax/occurrence projection with conservative facts. An empty loop exits
successfully without a body transition; a known nonempty loop exposes the
final body's exit status; a zero-or-more loop joins its zero path with every
reachable normal exit. Bounded fixed-point analysis widens differing cwd or
variable values to Unknown rather than selecting one path.
cd and chdir use the effective argument vector for the current visit.
pushd and popd may be recognized only with unknown success cwd until the
directory stack is modeled. Unmodeled execution-bearing or
attribute-mutating builtins fail the complete parse closed globally, not only
inside loops. The stable catalog is eval, source / ., trap, let,
declare, typeset, local, readonly, export, unset, read,
readarray, mapfile, getopts, and set, plus printf -v. These forms can
evaluate argument text, install deferred execution, or assign through
unproved integer, nameref, or array attributes. Recognition recursively
unwraps statically proved command and builtin dispatch; dynamic or invalid
wrapper grammar fails closed. Ordinary printf without -v remains
supported.
Authored command-resolution mutation is independent from variable attributes
and cwd.
exec fails the complete parse closed globally because it replaces the shell
or makes commandless redirections persistent. Mutating or ambiguous hash,
alias, unalias, shopt, and enable forms likewise fail globally before a
later command can inherit an unmodeled executable identity. The only retained
forms are exact static queries: bare hash, alias, shopt, and enable;
hash -l without operands and hash -t NAME...; alias [-p] [NAME...]
without a definition; shopt option clusters without s or u; and no-name
enable listing flags composed only from a, n, p, and s. Dynamic or
invalid grammar fails closed, and exact command / builtin wrappers cannot
bypass the boundary. break, continue, return, and exit remain
loop-region failures until their transfers are implemented.
Unquoted time and ! reserved prefixes execute the following pipeline with
current-shell state; coproc starts hidden concurrent execution; and
{ ...; } is a current-shell group. Stable v0.3 fails exact unquoted time,
!, coproc, {, or } in command position closed until their nested
structure and state propagation are modeled. Quoted/escaped spellings,
/usr/bin/time, and command time ... remain ordinary command identities and
do not acquire reserved-word semantics.
Substitutions and subshells inherit the current variable/cwd state but discard
their state changes on exit. Decoded Bash command wrappers inherit invocation
cwd but no loop binding unless export is separately proved. Pipeline stages
enter from the same pipeline input; possible lastpipe leakage joins the full
cwd and variable state, independently of conservative pipefail exit
partitioning.
Compatibility arguments always retain authored loop-variable spelling. A
variable-derived path that is not independently exact keeps or becomes
DynamicSkip, and a relative path whose reachable visit cwds disagree loses
its static resolution. In particular, compatibility projection may not retain
the configured $HOME resolution after a loop binds HOME, even though that
binding is outside the v0.3 supported-name boundary.
PowerShell foreach analysis owns a case-insensitive persistent binding map;
the structural parser does not push and restore lexical loop bindings. A proved
nonempty ordered plan leaves the final assigned value after the loop, including
when a nested loop reuses the same name. A proved empty plan performs no body
transition and preserves the incoming binding and cwd. A zero-or-more plan
joins its zero-iteration entry with every reachable iteration exit. Repeated
visits to one authored occurrence join effective argument and cwd facts rather
than selecting a representative visit.
Concrete and fixed-point PowerShell loop visits share the parse-wide 4096 transition budget. Overflow makes the complete parse unparseable and publishes no partial command or compatibility projection.
Set-Location is modeled from the complete effective argument vector. Its
success exit takes the proved filesystem target cwd and its failure exit retains
the incoming cwd. A successful non-filesystem or unproved target also
invalidates binding and command-resolution proofs because relative provider
operations may mutate that state. && consumes only success, || only failure, and statement
sequence consumes their join. The analyzer publishes no finite cwd set, so any
disagreement becomes Unknown. Unsupported location-stack operations,
state/command-resolution mutation, and dynamic dispatch remain fail closed.
For provider-capable item mutators, a target that cannot be proved outside the
Alias, Function, Variable, and Environment providers invalidates binding
proofs; a dynamic value with a proved filesystem target does not.
PowerShell $() and parenthesized groups propagate supported state in the
current runspace. Decoded child hosts isolate their exit state and do not
inherit the isolated initial-state assertion unless their own invocation proves
it. Compatibility parse-time location attribution is cloned for loop iterator
and body parsing: a proved empty body cannot leak a location change, while a
possibly reached loop mutation poisons any stale exact compatibility
attribution rather than choosing one execution path.
After outcome analysis, cwd-dependent compatibility arguments, clause
elements, redirects, and cwd attribution are rebased to the occurrence's exact
cwd. When the occurrence cwd is unknown, cwd-dependent resolutions are cleared
and attribution uses the existing <dynamic-cwd> marker. A success-path parse
location therefore cannot leak into an exact failure continuation. Decoded
child-host leaves retain their inherited invocation-cwd attribution for this
projection even though child exit state remains isolated.
Correcting a v0.2 compatibility Redirect does not by itself mark the v0.3
occurrence complete. Occurrence-level redirect value completeness remains
governed by the explicit redirect analysis contract below.
Occurrence-specific redirect analysis is additive. The existing Redirect
record remains the v0.2 compatibility leaf and is not reinterpreted.
public abstract record RedirectSource
{
private protected RedirectSource() { }
private protected abstract object LibraryOwnership { get; }
public sealed record Unknown : RedirectSource { ... }
public sealed record Default : RedirectSource { ... }
public sealed record Descriptor : RedirectSource { public int Value { get; } }
public sealed record PowerShellAllStreams : RedirectSource { ... }
}
public abstract record RedirectAnalysis
{
private protected RedirectAnalysis() { }
private protected abstract object LibraryOwnership { get; }
public Redirect Authored { get; internal init; } = null!;
public RedirectSource Source { get; internal init; } = null!;
public bool IsComplete { get; internal init; }
}
public sealed record FileRedirectAnalysis : RedirectAnalysis
{
public FileRedirectMode Mode { get; }
public ShellValueDomain Target { get; internal init; } = null!;
}
public sealed record UnresolvedRedirectAnalysis : RedirectAnalysis { ... }
public sealed record DescriptorDuplicateRedirectAnalysis : RedirectAnalysis
{
public int TargetDescriptor { get; internal init; }
}
public sealed record DescriptorMoveRedirectAnalysis : RedirectAnalysis
{
public int TargetDescriptor { get; internal init; }
}
public sealed record DescriptorCloseRedirectAnalysis : RedirectAnalysis { ... }
public sealed record HereDocumentRedirectAnalysis : RedirectAnalysis
{
public HereDocumentAnalysis Document { get; internal init; } = null!;
}
public sealed record HereStringRedirectAnalysis : RedirectAnalysis
{
public ShellValueDomain Data { get; internal init; } = null!;
}
public enum FileRedirectMode
{
Input,
Output,
Append,
CombinedOutput,
CombinedOutputAppend,
}
public sealed record HereDocumentAnalysis
{
internal HereDocumentAnalysis() { }
public ShellSourceFragment Delimiter { get; internal init; } = null!;
public ShellSourceFragment Body { get; internal init; } = null!;
public HereDocumentExpansionMode ExpansionMode { get; internal init; }
public bool StripLeadingTabs { get; internal init; }
public bool IsComplete { get; internal init; }
}
public enum HereDocumentExpansionMode
{
Unknown,
Literal,
Expand,
}Authored references the exact corresponding Clause.Redirects leaf.
RedirectSource preserves a shell-default stream, an explicit numeric
descriptor, or PowerShell's * selector without erasing shell identity. File
alternatives are path-relevant; descriptor alternatives are not paths.
A quoted heredoc delimiter makes the body Literal; an expanding body is
complete only when every supported execution-bearing substitution has been
discovered as its own command occurrence. Bash HereString data uses Data,
includes the shell's trailing newline in an exact value, and is not
path-relevant. PowerShell here-strings remain ordinary value tokens.
RedirectSource.Unknown is valid only with UnresolvedRedirectAnalysis.
RedirectSource.Default may own ordinary or combined file, descriptor,
heredoc, or here-string alternatives.
RedirectSource.Descriptor may own ordinary file, descriptor, heredoc, or
here-string alternatives but never combined-output. PowerShell all-streams may
own only output/append file alternatives or descriptor duplication to stream
- Any other internal pairing discards the authorization projection.
A Bash file redirect whose expansion cannot prove exactly one target has an
Unknown target and IsComplete=false; its containing command occurrence is
also incomplete. In particular, an unquoted wildcard target is not completed
by enumerating the parser process's filesystem.
The public records define an in-memory typed API, not a stable polymorphic JSON
wire format. Their generated equality, hashing, and ToString() behavior is
part of the normal record shape. Consumers that persist parser results own a
versioned DTO or explicit serializer mapping.
One logical command within a compound. Each clause has its own verb chain, args, redirects, and the operator that joined it to the previous clause.
public sealed record Clause
{
/// <summary>
/// The operator joining this clause to the previous one. The first
/// clause in a ParsedCommand has Operator=None. Subsequent clauses
/// carry the operator that preceded them in the source
/// (e.g. `a && b` produces clauses [{None,a}, {AndIf,b}]).
/// </summary>
public CompoundOperator Operator { get; init; }
/// <summary>The verb chain (see §3.3 and §6).</summary>
public VerbChain Verb { get; init; } = new();
/// <summary>
/// All argument tokens after the verb chain, in source order. Includes
/// flags and positional args. See <see cref="Arg.Kind"/> for token kind.
/// </summary>
public IReadOnlyList<Arg> Args { get; init; } = [];
/// <summary>
/// Compatibility redirects on this clause (>, >>, <, 2>, 2>>, &>,
/// &>>). Each entry retains the v0.2 direction and target projection;
/// v0.3 consumers use CommandOccurrence.Redirects for exact semantics.
/// </summary>
public IReadOnlyList<Redirect> Redirects { get; init; } = [];
/// <summary>
/// Significant source-authored verbs, arguments, and redirects in source
/// order. This is the provenance view; Verb, Args, and Redirects remain
/// compatibility projections. Synthetic cwd attribution is excluded.
/// </summary>
public IReadOnlyList<ClauseElement> Elements { get; init; } = [];
/// <summary>
/// True when this clause is wrapped in a subshell (parens). Subshells
/// isolate cd state — see §9.
/// </summary>
public bool IsSubshell { get; init; }
/// <summary>
/// True when this clause is the result of parser-local recursion into a
/// command-string wrapper — Bash `bash -c "..."` / `sh -c "..."`, or
/// PowerShell `pwsh -Command "..."` / `pwsh -EncodedCommand ...` /
/// static `Invoke-Expression '...'`. One parser never delegates wrapper
/// payloads to the other parser. Useful
/// for consumers that want to surface "this came from a wrapped
/// invocation" in UI.
/// </summary>
/// <remarks>Renamed from `IsCommandStringWrapped` in v0.2.0 — see RELEASE_NOTES.md
/// and SPEC.POWERSHELL.md §3 for the old→new mapping.</remarks>
public bool IsCommandStringWrapped { get; init; }
}One significant source-authored element of a clause. Elements preserves the
cross-projection order that Verb, Args, and Redirects cannot represent on
their own.
public sealed record ClauseElement
{
/// <summary>Exact authored source slice, including quote delimiters.</summary>
public string Raw { get; init; } = "";
/// <summary>
/// Lexer-decoded logical value. For a redirect this is the decoded target;
/// for an inline binding it remains the complete decoded source token.
/// </summary>
public string Value { get; init; } = "";
public ClauseElementRole Role { get; init; }
/// <summary>
/// Span in ParsedCommand.Source. Null for elements surfaced through a
/// decoded command-string wrapper when no exact outer mapping exists.
/// </summary>
public int? SourceStart { get; init; }
public int? SourceLength { get; init; }
/// <summary>
/// Number of parser-classified verb elements authored before this element
/// in the clause. For a verb element, this is its zero-based Verb.Tokens
/// index. This is an AST coordinate, not an executable-specific semantic
/// boundary.
/// </summary>
public int PrecedingVerbElementCount { get; init; }
/// <summary>
/// Argument classification for this token, inline bound value, or redirect
/// target. Verb elements use Literal, except dynamic command names use
/// DynamicSkip.
/// </summary>
public ArgKind Kind { get; init; }
public bool IsFlag { get; init; }
public bool IsPath { get; init; }
public string? Resolved { get; init; }
}
public enum ClauseElementRole
{
Verb,
Argument,
Redirect
}The collection contains significant leaves only: whitespace, comments,
compound operators, grouping delimiters, and shell call operators are excluded.
Each verb token appears exactly once with Role=Verb. Each authored argument
token appears once with Role=Argument; inline forms such as
--work-tree=../repo stay one element even when Args exposes separate flag
and value projections. Shell-adjacent fragments that form one native argument,
such as --data="@request file.json", likewise stay one element spanning the
complete authored argument. The parser consumes the full contiguous fragment
run, including an unquoted value prefix such as --data=@request".json".
Mixed quoting that prevents safe reconstruction of resolver-sensitive literal
syntax ($, glob metacharacters, ~, provider prefixes) safe-fails the bound
value as DynamicSkip, including syntax exposed only after an operand marker
such as curl's leading @ is removed. A parser-defined opaque computed region that is
safe-failed as one DynamicSkip argument also appears as one argument element;
its Raw and Value are the complete source slice rather than a claim that
the parser understood the region's interior. Each redirect appears once with
Role=Redirect; its ordinal among redirect elements matches its ordinal in
Redirects, and Raw spans the operator through its target.
PrecedingVerbElementCount is clause-local and resets to zero at every clause.
For git -C /repo commit, -C and /repo carry 1; for
git commit -C HEAD~1, -C and HEAD~1 carry 2. ShellSyntaxTree reports
that parser-relative coordinate but does not assign Git-specific meaning to
it. Role=Verb mirrors the greedy Clause.Verb heuristic. Therefore an
unrecognized option can stop verb extraction and cause a later semantic
subcommand to appear with Role=Argument; consumers SHALL use the complete
authored element order rather than treating this count as an executable's
semantic command boundary.
Synthetic cwd-attribution args are deliberately absent from Elements: they
remain available through Args with IsCwdAttribution=true. Clauses expanded
from command-string wrappers preserve each element's inner Raw and Value,
but set SourceStart and SourceLength to null rather than guessing how a
decoded or escaped inner character maps into the outer ParsedCommand.Source.
Because Clause is a record, Elements participates in its generated value
equality and hashing. Generated ToString() and default JSON serialization
also include the projection. The API addition is source- and binary-additive,
but these generated behaviors are observably different.
The verb of a clause. Multi-token to handle commands like git push,
docker compose up, dotnet ef migrations add. Length determined by the
greedy verb-chain heuristic in §6.1 — consecutive verb-like Word tokens
from the start of the clause, transparently consuming flag-with-value
pairs, with a 1-token carveout for FILE verbs.
public sealed record VerbChain
{
/// <summary>
/// Verb tokens in source order. Empty when the clause has no verb
/// (e.g. clause is just a redirect or an empty fragment).
/// </summary>
public IReadOnlyList<string> Tokens { get; init; } = [];
/// <summary>
/// The canonical, alias-resolved verb identity (added v0.2.0). Non-null
/// only when the parser rewrote a built-in alias — `ls` → `Get-ChildItem`.
/// Null for every bash clause. See SPEC.POWERSHELL.md §3.
/// </summary>
public string? CanonicalVerb { get; init; }
/// <summary>
/// True when the clause's command name is a dynamic token the parser
/// cannot statically identify — `& $exe`, `& "tool-$name"`,
/// or supported `& $(Get-Thing)` (added v0.2.0). An unsupported
/// executable identity expression makes the whole result unparseable.
/// Always false for bash clauses. See SPEC.POWERSHELL.md §3.
/// </summary>
public bool IsDynamic { get; init; }
/// <summary>Convenience: tokens joined with spaces.</summary>
public string Joined => string.Join(" ", Tokens);
}Note: The single-space form
string.Join(" ", …)is used (not thecharoverloadstring.Join(' ', …)) so the implementation compiles on bothnetstandard2.0andnet8.0. Thecharoverload is net5+ only.
One argument token after the verb chain. Includes resolution state.
public sealed record Arg
{
/// <summary>Verbatim token from the source.</summary>
public string Raw { get; init; } = "";
/// <summary>
/// Resolved value for path tokens — tilde expanded, env vars
/// substituted, normalized to absolute path against
/// BashParserOptions.WorkingDirectory. Null when Kind is not a path
/// (Literal non-path / Glob / DynamicSkip).
/// </summary>
public string? Resolved { get; init; }
/// <summary>Token kind. See <see cref="ArgKind"/>.</summary>
public ArgKind Kind { get; init; }
/// <summary>
/// True when this token starts with '-' or '--' (a flag, not a
/// positional arg).
/// </summary>
public bool IsFlag => Raw.StartsWith('-');
/// <summary>
/// True when this token is a path the clause operates on (per the
/// per-verb pathArgs table; see §7). Set during parsing so consumers
/// don't reapply per-verb rules.
/// </summary>
public bool IsPath { get; init; }
/// <summary>
/// True when this Arg is a synthetic attribution arg representing
/// the working directory inherited from a preceding `cd`/`chdir`
/// clause in the same compound. Default false. See §9 for
/// propagation semantics.
/// </summary>
public bool IsCwdAttribution { get; init; }
}
public enum ArgKind
{
/// <summary>Literal value (string, number, flag).</summary>
Literal,
/// <summary>Token containing an unresolved env var reference.</summary>
EnvVar,
/// <summary>Token containing glob metachars (* ? [).</summary>
Glob,
/// <summary>Token starting with ~ (tilde).</summary>
Tilde,
/// <summary>
/// Token whose value cannot be safely resolved (unresolved env var,
/// unexpandable glob). Consumers SHALL treat as "no value extracted"
/// rather than using Raw as a literal path.
/// </summary>
DynamicSkip
}public sealed record Redirect
{
public RedirectDirection Direction { get; init; }
/// <summary>
/// Redirect target. Normally a path resolved per Arg conventions
/// (§8); for fd-dup / fd-close shorthand (`&N`, `&N-`, `&-`)
/// the raw token is carried verbatim and IsDynamicSkip is true.
/// </summary>
public string Target { get; init; } = "";
/// <summary>
/// True when the target is opaque to path resolution — a dynamic
/// token (env var, command substitution) or an fd-dup / fd-close
/// form. Consumers MUST NOT treat Target as a path when this is true.
/// </summary>
public bool IsDynamicSkip { get; init; }
}
public enum RedirectDirection
{
In, // <
Out, // >
Append, // >>
ErrOut, // 2>
ErrAppend // 2>>
}public enum CompoundOperator
{
None, // first clause; no prior operator
AndIf, // &&
OrIf, // ||
Sequence, // ;
Pipe // |
}Approximate BNF for what the parser accepts. Anything outside this grammar
is unparseable (ParsedCommand.IsUnparseable = true).
command := clause (compound_op clause)*
compound_op := "&&" | "||" | ";" | "|" | NEWLINE
clause := subshell | bash_c_wrapper | simple_clause
subshell := "(" command ")"
bash_c_wrapper := ("bash" | "sh") static_flag* "-c" STATIC_QUOTED_STRING
static_flag := exact-one literal Word beginning with "-"
STATIC_QUOTED_STRING := QuotedString whose outer-shell provenance is entirely
literal and exactly one value
simple_clause := verb_chain arg* redirect*
verb_chain := verb_like_word (FW_pair? verb_like_word)*
// greedy walk per §6.1; FW_pair is a
// flag-with-value pair owned by word_0
// (transparent to the walk); stops at
// the first path-shaped or non-verb-like
// token. For
// word_0 ∈ FileVerbs, exactly 1 token.
verb_like_word := static word satisfying §6.1; the initial command-name
element contains no supported_substitution
arg := word | flag | quoted_string | supported_substitution
flag := "-" letter+ | "--" word
redirect := redirect_op target
redirect_op := descriptor? (">" | ">>" | "<") | "&>" | "&>>"
descriptor := digit+
target := word | quoted_string | supported_substitution
supported_substitution := "$(" command ")"
word := non-whitespace, non-operator fragments; may contain
supported_substitution children in v0.3
quoted_string := single-quoted | double-quoted
// double-quoted values may contain supported_substitution;
// single-quoted and escaped spellings remain literal
Notes:
- Whitespace between tokens is one or more spaces or tabs.
- A bare newline outside quotes, heredoc bodies, line continuations, and
$(...)/ backtick substitutions is a statement separator — semantically equivalent to;, producingCompoundOperator.Sequence. Consecutive newlines, leading and trailing newlines, and a newline immediately following a compound operator all collapse: they never yield an empty clause. The newline after a heredoc terminator likewise separates the heredoc's clause from what follows. \followed by a newline is removed before word-boundary analysis. It joins adjacent fragments (r\+ newline +mis the command namerm); actual surrounding spaces still separate words.- Bash line comments (
#at a word boundary through end-of-line) are whitespace-equivalent at the lexer level — they emit a Comment token for source fidelity but are filtered alongside Whitespace by the parser, so they do not appear in the grammar. See §5 "Comment handling" for boundary rules. \before a metachar inside a double-quoted string escapes the metachar.- Single-quoted strings preserve all bytes literally — no escape processing.
- v0.2 recognizes heredocs (
<<EOF ... EOF) as redirect syntax while the body is skipped. Stable v0.3 preserves delimiter, body, expansion mode, tab-stripping mode, and completeness throughHereDocumentAnalysis. The bounded grammar accepts one terminal<</<<-redirect, including a literal numeric source descriptor such as3<<EOF, on a command header, with optional whitespace or a trailing comment after the delimiter. Additional header tokens, pipelines, and queued heredocs are unparseable until their body-association grammar is modeled. Quote removal determines the delimiter spelling; any quoted or escaped delimiter fragment makes the body literal. In an expanding body, unescaped$()substitutions are executable even when their spelling is surrounded by quote characters, because heredoc body quotes are data rather than shell quoting syntax. Escaped substitutions remain literal. Legacy backticks,$((...))and obsolete$[...]arithmetic expansion, prompt-transformed${name@P}or other parameter operators, line continuations that could hide a substitution boundary, and incomplete substitutions make the whole result unparseable. - Redirect targets matching the POSIX fd-dup / fd-close shorthand —
&N,&N-, or&-(whereNis one or more decimal digits) — are NOT path-resolved. The parser carries the raw token (e.g.&1) onRedirect.Targetand setsRedirect.IsDynamicSkip = true. This prevents2>&1from being incorrectly resolved to<cwd>/&1. - Function definitions, assignment-prefix commands,
while/until,if/elif/else,case/esac, C-style or implicit loops, arithmetic execution, process substitution, and single-&background lists remain unparseable in stable v0.3 because they can hide executable regions outside the bounded grammar below.
Contextual keywords match only in command position. echo for therefore
remains a simple command argument rather than starting a loop.
bash_script(stop) := bash_list_item (list_sep bash_list_item)*
bash_list_item := bash_and_or
bash_and_or := bash_pipeline (("&&" | "||") bash_pipeline)*
bash_pipeline := bash_command ("|" bash_command)*
bash_command := bash_for_in
| bash_subshell
| bash_c_wrapper
| bash_simple_command
bash_for_in := "for" binding_name "in" iterable_word*
list_terminator "do"
bash_script(stop = "done")
"done"
list_sep := ";" | NEWLINE
list_terminator := ";" | NEWLINE+
binding_name := supported_scalar_binding
supported_scalar_binding := [a-z][a-z0-9_]*
except "auto_resume" and "histchars"
iterable_word := word | quoted_string | supported_substitution
The supported stable-v0.3 set is the existing simple-command grammar plus
for name in words. Bash
accepts additional shell identifiers as loop variables, but this bounded
grammar fails them closed for the initial-state reasons specified in §2.
Every fully
delimited $() command substitution in a supported simple-command argument,
redirect value, iterable, or expanding heredoc body is recursively parsed and
exposes its inner commands; its produced value remains Unknown. A nested
substitution is recursively attached to the nearest containing simple command.
Simple $name and ${name} forms additionally require the proved
variable-attribute state from §2; syntactic simplicity alone is not evidence
that dereferencing a nameref cannot execute an array subscript.
Legacy backtick substitution becomes unparseable in v0.3 until its distinct
escape and nesting rules can be mapped without guessing. A Bash path-shaped
glob may produce a Pattern only when its exact static covering directory is
proved without filesystem enumeration. Bash <<< is a non-path HereString
redirect.
A $() fragment in Bash command-name position leaves the outer command
identity runtime-dependent. Stable v0.3 makes the whole result unparseable
rather than changing the v0.2 VerbChain.IsDynamic contract, which remains
PowerShell-specific. Diagnostic Syntax may retain the discovered substitution,
but Commands and Clauses are empty.
Missing do or done; an unsupported substitution whose
commands cannot all be discovered; or any skipped executable region makes the
entire result unparseable. The parser may preserve a diagnostic syntax tree,
but it returns empty Commands and Clauses so consumers cannot authorize a
discovered subset.
The lexer produces tokens consumed by the parser. Token kinds:
- WORD — sequence of non-whitespace, non-operator, non-quote chars.
Example:
git,/etc/foo,--force,~/path,$VAR. A braced parameter is absorbed only when its body is a simple shell identifier, positional parameter, or special parameter. Parameter operators are unparseable because their operands can contain hidden execution; the resolver in §8 decidesKindfor accepted simple forms. - QUOTED_STRING — single- or double-quoted string. The lexer strips
the quote delimiters from the token value. Example:
"hello world"becomes the token valuehello world. - OPERATOR —
&&,||,;,|,>,>>,<, numeric-descriptor forms such as2>,3>>,10<,3<<,4<<-, and5<<<,&>,&>>,(,),<<,<<-,<<<. - WHITESPACE — one or more spaces, tabs, or newlines (newlines inside
a heredoc body are not emitted as ordinary tokens; the delimiter token
retains the body's resolver fragments and authored extent). A whitespace run that
contains a newline — including the newline after a heredoc terminator —
is flagged as a statement separator; the parser retains those
tokens past
FilterSignificantand splits clauses on them per §4. A pure space/tab run carries no flag and is discarded after splitting. - CONTINUATION —
\+\n(or\r\n). Removed before word-boundary analysis; adjacent lexical fragments remain one authored word. - OPAQUE_SUBSTITUTION —
$(cmd)or backtick`cmd`. The full substitution slice (including delimiters) becomes a single token. Boundary tracking handles nested same-kind regions, nested quotes, and\Xescapes via a shared opaque-region scanner. The parser consumes this token asArg{ Kind=DynamicSkip, IsPath=false, Resolved=null }per locked interpretation #2. Expanding-heredoc substitutions use the same opaque fragment semantics but remain attached to the delimiter token rather than entering the ordinary command-token stream. - UNPARSEABLE_SENTINEL —
$((expr))or obsolete$[expr]arithmetic expansion, or any operator-bearing parameter expansion such as${var:-$(cmd)},${var//pat/repl}, or${var@P}. The lexer skips past the matching close ())or}respectively) and emits a sentinel whose reason names the rejected construct. The parser consumes this token by setting outerParsedCommand.IsUnparseable = true(see §11). - COMMENT —
#at a word boundary (start of input, or preceded by whitespace, a newline, an operator, or any other lexer-recognized boundary) starts a line comment running to (but not including) the next newline. The lexer emits a single Comment token covering the#and the comment text, for source fidelity. The parser drops Comment tokens inFilterSignificantalongside Whitespace and Continuation — comments produce no clauses, args, redirects, or flags. See "Comment handling" below for boundary rules.
- Single quotes
'...'preserve bytes literally. No escape processing, no variable expansion. Anything inside is one token. - Double quotes
"..."preserve whitespace but allow:\"escapes the closing quote.\\escapes a backslash.\$escapes a dollar sign.$VARand${VAR}are recognized as env var references but not expanded — the token is markedArgKind.EnvVar(orDynamicSkipif resolution would be required for path classification).
- Unbalanced quotes →
IsUnparseable = truewith reason"unbalanced quote at position N".
\Xoutside quotes: removes the backslash, takes X literally. Example:echo \$HOMEproduces token$HOMEwithArgKind.Literal.\Xinside double quotes: only\",\\,\$,\\, and\\+newline are recognized escape sequences. Other backslashes preserved literally.
Operators terminate the current token. cd /tmp&&ls lexes as
[cd, /tmp, &&, ls] — no whitespace required around operators. The lexer
must handle this. A numeric descriptor is an operator prefix only when its
digits begin at a shell-token boundary and become adjacent to <, >, >>,
<<, <<-, or <<< after Bash removes unquoted line continuations.
Continuations may join digit fragments or the descriptor and operator; LF and
CRLF spellings retain their authored span while producing the same descriptor.
Digits joined to an ordinary, quoted, or escaped word remain part of that word;
command3>file therefore uses command name command3 and a default-source >
redirect.
- An unquoted
#that appears at a word boundary starts a comment that runs to (but does not include) the next newline. A word boundary is: start of input, or the position immediately after a whitespace run, a newline, an operator (&&,||,;,|,>,>>,<, a numeric descriptor adjacent to>,>>,<,<<,<<-, or<<<,&>,&>>,(,),<<,<<-,<<<), a quoted string, or an opaque substitution. Equivalently:#is comment-start everywhere the outer lexer dispatch loop sits, because every other lexer rule has already consumed its territory before#is considered. #inside single or double quotes is a literal character (no comment).#in the interior of an unquoted word (e.g.abc#def) is a literal character.ReadWordconsumes the whole word before the outer loop can see the embedded#; there is no re-scanning.\#(backslash-escaped#outside quotes) is consumed by the normal escape rule — the backslash is dropped and#becomes a regular word character. Equivalent example:cmd \#abcproduces one Word token#abc.- The terminating newline is not consumed by the Comment token. It survives as a Whitespace token, preserving statement-boundary semantics for the parser (see §4).
- A Comment token's
Valueis empty (matchingWhitespace/Continuation);SourceStart/SourceLengthidentify the slice including the leading#so callers that need the literal text can recover it from the original input span. - Effect on parsing: comment-only input parses to
Clauses = [],IsUnparseable = false— mirroring empty-input behavior. A comment leading, trailing, or interleaved with a clause contributes no tokens to the verb chain, args, or redirects of any clause.
These are data, not logic. Implement as static readonly collections.
Per issue #27 (locked in v0.1.4-alpha), the parser does not consult a
static arity table. It walks consecutive verb-like Word tokens from the
clause start. The walk stops before a path-shaped or non-verb-like token.
This rule naturally scales to unknown CLIs
(freshdesk ticket list, kubectl get pods, dotnet ef migrations add)
without curated table entries.
A token is "verb-like" when all of these hold:
Kind == BashTokenKind.Word(quoted strings are values, never verbs at index ≥ 1).- Length is in
[1, 64]characters. - First character is an ASCII lowercase letter
[a-z]. - Remaining characters are drawn from
[a-z0-9._-]only.
The predicate is implemented in BashVerbs.IsVerbLikeToken. The leading
lowercase requirement mirrors real CLI subcommand convention; the
character allow-list naturally excludes flags (-x starts with -),
paths (/, \, ~), env-var refs ($VAR), URLs (://), globs
(* ? [), and user-named identifiers (uppercase first char like
InitialCreate).
The walk also rejects a token that matches the §8 path-shape heuristic. This rule applies even when the lexical predicate accepts the token.
For a clause whose first token is a Word firstVerb:
- Append
firstVerbto the verb chain (it does not need to satisfyIsVerbLikeToken— bare commands likeCurlor_initare still commands). Do not apply the path-shape boundary at command position. A command such asdeploy.shor./deploy.shremains the first verb. - Iterate the remaining tokens in order. For each token
t:- If
t.Kind != Word: stop. - If
tis a flag (IsFlagWord):- If
firstVerbhas aFlagsWithValueentry containingStripEqualsValue(t.Value)AND the next token isWordorQuotedStringANDt.Valuehas no inline=: consume both as a flag-value pair, mark their indices forconsumedFlagValueIndices, and continue walking. - Otherwise: stop.
- If
- If
firstVerb ∈ FileVerbs: stop (1-token carveout — see below). - If
BashResolver.LooksLikePath(t.Value): stop. The argument pass uses the same classifier and preserves the token as a path argument. - If
!IsVerbLikeToken(t): stop. - Otherwise: append
t.Valueto the verb chain and continue.
- If
If the first token is a QuotedString (e.g. "git" push origin main),
emit a 1-token verb chain [firstVerb] and skip the walk entirely. Bash
treats the quoted form as a verb-identity carrier; remaining tokens are
arg-list material.
For verbs in §6.3 FileVerbs (file-mutation, file-read, editors,
compression, shell loaders, etc.), the verb chain stops at exactly one
token. The flag-with-value consumption still runs so the value of
curl -o file, tar -C /path, git -C /repo style flags picks up
IsPath=true via the FlagValueIsPath mechanism.
The carveout exists because FileVerbs use SPEC §7 per-verb positional
rules to classify args as paths. Without it, a bare-name target like
cat README would over-extract — README is shape-wise verb-like —
and lose the IsPath=true classification downstream consumers depend
on for zone-gate evaluation.
| Input | Verb chain | Args |
|---|---|---|
git push origin main |
[git, push, origin, main] |
[] (over-extracts; see §6.1.1) |
git -C /repo worktree list --porcelain |
[git, worktree, list] |
[-C, /repo, --porcelain] |
freshdesk ticket list --status open |
[freshdesk, ticket, list] |
[--status, open] |
kubectl get pods my-pod |
[kubectl, get, pods, my-pod] |
[] |
aws s3 cp src dst |
[aws, s3, cp, src, dst] |
[] (bare-word path args over-extract) |
dotnet ef migrations add InitialCreate |
[dotnet, ef, migrations, add] |
[InitialCreate] (stops at uppercase) |
deploy.sh status |
[deploy.sh, status] |
[] (command position wins) |
git diff install-skills.sh |
[git, diff] |
[install-skills.sh] (path-shaped operand) |
kubectl apply deployment.yaml |
[kubectl, apply] |
[deployment.yaml] (path-shaped operand) |
tool plugin.sh list |
[tool] |
[plugin.sh, list] (path evidence wins) |
cat /etc/passwd |
[cat] |
[/etc/passwd] (FileVerb carveout) |
cat README |
[cat] |
[README] (FileVerb carveout preserves IsPath) |
ls -la /tmp |
[ls] |
[-la, /tmp] (FileVerb carveout) |
chmod 755 file |
[chmod] |
[755, file] (digit-start kills walk; FileVerb anyway) |
echo hello |
[echo, hello] |
[] (echo is not a FileVerb; over-extracts) |
Clause.Verb is a convenience hint, not a security contract.
The parser deliberately over-extracts on bare-word args because no
syntactic rule disambiguates origin (a branch name) from worktree
(a subcommand verb) without per-CLI semantic knowledge — and we will
not bake per-CLI knowledge into the parser.
Consumers needing security-grade command identification choose one of two
strategies over the source-ordered Clause.Elements view:
- Strict authored-stream matching. Match every modeled significant
element in source order. A strict matcher may define explicit operand slots
or wildcards, but it SHALL NOT discard an intervening argument merely
because the parser assigned it
Role=Argument. Therefore a strictgit commitpattern does not matchgit -C /repo commit. - General executable-aware matching. Pass the complete authored stream to a grammar owned by the consumer. The grammar consumes known options and operands, identifies the executable's semantic command, and returns both a normalized approval identity and every policy-relevant operand or scope. Equivalent syntax may reuse an approval only after complete interpretation.
For example, a Git-aware matcher may interpret git -C /repo commit as the
general identity git commit with effective directory /repo. It may then
reuse a git commit approval only when that approval's directory policy covers
/repo. Likewise, executable-aware matchers may intentionally normalize
git push origin main to git push or kubectl get pods my-pod to
kubectl get pods when their grammars establish which suffixes are operands.
There is no shell-generic rule that selects all Role=Verb elements and
compares them as a contiguous semantic prefix. For unknown executables or an
unrecognized option shape, consumers should use strict matching or prompt;
they should not silently fall back to a broader general identity.
False-negative (re-prompt) is recoverable. False-positive (silent destructive grant) is not. Narrow-by-default favors the recoverable failure mode.
The path-shape boundary requires no command dictionary. It uses the same curated evidence as argument classification. A rare extension-shaped subcommand becomes a path argument because the stronger path evidence wins.
Verbs whose first non-flag positional arg becomes the cwd for subsequent clauses in the same compound (see §9).
internal static readonly HashSet<string> CwdVerbs =
new(StringComparer.OrdinalIgnoreCase)
{
"cd", "chdir", "popd", "pushd",
"push-location", "set-location" // PowerShell idioms (forward-compat)
};Verbs whose positional args are paths. The default extraction rule is "all non-flag positional args after the verb chain are paths." Per-verb overrides in §7.
internal static readonly HashSet<string> FileVerbs =
new(StringComparer.OrdinalIgnoreCase)
{
// CWD verbs are also FILE verbs (their target is a path)
"cd", "chdir", "popd", "pushd", "push-location", "set-location",
// File mutation
"rm", "cp", "mv", "mkdir", "rmdir", "touch", "ln",
"chmod", "chown", "chgrp", "stat", "test",
// Read
"cat", "less", "more", "head", "tail", "grep", "rg",
"find", "fd", "locate", "wc", "file",
// Editors / text tools
"sed", "awk", "vi", "vim", "nano", "emacs", "ed",
// Compression
"tar", "zip", "unzip", "gzip", "gunzip", "bzip2", "xz",
// Network with file targets
"curl", "wget", "scp", "rsync", "sftp",
// Shell / interpreter loaders
"bash", "sh", "zsh", "fish",
"python", "python3", "node", "ruby", "perl", "php",
// Diff / patch
"diff", "patch", "cmp",
// Listing
"ls", "dir", "tree",
};The Windows native file utilities. As of v0.2.0 the PowerShell parser's
PwshVerbs.FileVerbs table consumes this reserved set
(type, copy, move, del, xcopy, robocopy, findstr) so a
native Windows file tool in a PowerShell command still gets path
classification. PowerShell cmdlet file verbs (Get-Content,
Remove-Item, Copy-Item, ...) are owned by SPEC.POWERSHELL.md §6.4 —
they are recognized by cmdlet shape and alias resolution, not by this
table. A Windows cmd parser remains deferred (§18).
internal static readonly HashSet<string> CmdFileVerbs =
new(StringComparer.OrdinalIgnoreCase)
{
"type", "copy", "move", "del", "erase", "ren",
"xcopy", "robocopy", "findstr",
};The default rule for FILE verbs: every non-flag positional arg after the verb chain is a path. Per-verb overrides:
| Verb | Rule |
|---|---|
chmod |
First non-flag positional is mode (e.g. 755, +x); rest are paths. |
chown |
First non-flag positional is user[:group]; rest are paths. |
chgrp |
First non-flag positional is group; rest are paths. |
ln |
All positionals are paths (source then target). |
find |
First positional is a path; rest are predicate args (skip). |
grep |
First positional is pattern; rest are paths. |
rg |
First positional is pattern; rest are paths. |
sed |
First positional is script; rest are paths. |
awk |
First positional is program; rest are paths. |
tar |
Action flag determines path roles; default to extracting all non-flag positionals as paths. -F / --info-script / --new-volume-script values are executable command text and safe-fail as DynamicSkip, never paths. |
curl |
First positional is URL, not a path. -o / --output and -D / --dump-header values are paths. -d / --data values are request data unless prefixed with @, which reads a file; @- reads stdin and is not a path. |
wget |
First positional is URL, not a path. -o / --output-file writes a log path; -O / --output-document writes the downloaded document path. |
scp, rsync, sftp |
All positionals are paths (some remote). |
cd, chdir, pushd, popd |
First non-flag positional is the cwd target (a path). |
| Others (in FileVerbs, no override) | All non-flag positionals are paths. |
Some flags take values (-o file, -C /repo, --output=file). The parser
must know which flags consume the next token as a value. Curated table:
internal static readonly IReadOnlyDictionary<string, HashSet<string>>
FlagsWithValue = new Dictionary<string, HashSet<string>>(
StringComparer.OrdinalIgnoreCase)
{
["git"] = new HashSet<string>(StringComparer.Ordinal) { "-c", "-C", "--git-dir", "--work-tree" },
["curl"] = new HashSet<string>(StringComparer.Ordinal) { "-o", "--output", "-d", "--data", "-D", "--dump-header" },
["wget"] = new HashSet<string>(StringComparer.Ordinal) { "-o", "--output-file", "-O", "--output-document" },
["docker"]= new HashSet<string>(StringComparer.Ordinal) { "-v", "--volume", "-f", "--file" },
["tar"] = new HashSet<string>(StringComparer.Ordinal) { "-f", "--file", "-C", "--directory", "-F", "--info-script", "--new-volume-script" },
// Add as corpus surfaces real cases.
};Note: the value type is
HashSet<string>(notIReadOnlySet<string>) becauseIReadOnlySet<string>is .NET 5+ only and the library multi-targetsnetstandard2.0. Internal-only — no public-API impact.
Native option case. The outer verb dictionary retains its existing case-insensitive lookup, but each native option set uses
Ordinal. Native executables receive option spelling unchanged in Bash and PowerShell and may assign different meanings by case. Git lists both-cand-C: both consume a value. The generic table classifies uppercase-Cvalues as paths and lowercase-cvalues as non-paths. Executable-aware consumers still reinterpret command-scoped forms such asgit commit -c/-C, where Git uses the operand as a revision rather than the generic table's global meaning. Every supported case-distinct spelling is listed explicitly: curl-dconsumes request data while-Dconsumes a header-output path; Wget-oand-Oboth consume paths but write different files.
Operand-sensitive values. A fixed
(verb, flag)boolean is insufficient for curl-d/--data: a value beginning with@names a file curl reads. The parser preserves the authored marker in the valueArg.Rawand in the completeClauseElement.Value, strips the leading@only for path resolution, and leaves@-non-path because it denotes stdin. Dynamic and glob filenames continue through the normal §8 safe-fail rules after the prefix is removed.
Command-valued options. GNU tar executes
-F/--info-script/--new-volume-scriptoperands. Those options still consume a value, but the value isKind=DynamicSkip,IsPath=false, andResolved=null; resolving command text as a path would give a security gate false confidence.
Executable context.
FlagsWithValueis a curated parser heuristic, not a complete executable grammar. In particular, Docker's global-vmeans--version, whiledocker run -vconsumes a volume specification. The generic table preserves the establisheddocker runprojection; a Docker-aware consumer usesClause.Elementsto interpret placement and MUST NOT treat the table as universal Docker semantics.
Note: the verb-chain walk consumes flag-with-value pairs transparently. For
git -C /repo log, the walk consumes-C /repobefore evaluating the next token;logis then verb-like and extends the chain, producingVerb.Tokens = ["git", "log"]per §12's example. The same mechanic letsgit -C /repo worktree listextract the full 3-token chain per §6.1.
When a flag-with-value consumes the next token, the consumed token's
IsPath flag is set if the value is path-shaped (per the resolver in §8).
For git -C /repo log: the -C flag consumes /repo, marks it as a
path, then the verb chain continues with log.
--output=file (equals form) is parsed as one token; the path value after
= is extracted into a synthetic Arg with IsPath=true.
For each Arg with potential path content, the resolver attempts to produce a normalized absolute path. Resolution order:
-
Single-quoted bypass. If the source token came from a single-quoted string (per §5: bytes are preserved literally — no escape processing, no variable expansion), the resolver skips steps 1–5 entirely. Kind is
Literal;IsPathistrueandResolvedis set only when the slot is a path ANDTryResolveAbsolutePathon the raw bytes succeeds. Socat '/etc/passwd'still produces a resolved path, butecho '$HOME'stays literal —$HOMEis not expanded inside single quotes. -
Tilde expansion.
~→BashParserOptions.HomeDirectory.~/foo→<home>/foo. The complete tilde prefix must be unquoted; quoted or escaped slash spellings remain literal, while backslash-newline is removed before this test.~usernot supported →DynamicSkip. -
Env-var substitution.
$VARand${VAR}are not expanded even if the value is inEnvironment. We treat any env var reference asDynamicSkipbecause the env var available at parse time may differ from what's available when the agent's command actually runs.$HOMEis the only exception — we treat it as equivalent to~and expand it fromBashParserOptions.HomeDirectory. -
filesystem::/pathprefix stripping. Some tools emitfilesystem::/path/to/file; strip the prefix. Become/path/to/file. -
Glob detection. Tokens containing
*,?, or[are markedArgKind.Glob. The resolver does not expand globs. The token stays as-is inRaw;Resolvedis null.In a path-arg slot:
IsPath = true. Consumers can apply the "covering directory" heuristic (Path.GetDirectoryName(Raw)) to reason about the directory the glob resolves under (e.g./tmp/*.bak→/tmp).In a non-path slot:
IsPath = false.Per locked interpretation #3, glob and DynamicSkip carry distinct signals — globs preserve a useful covering-dir hint that DynamicSkip tokens lack.
-
Relative path resolution. Tokens not starting with
/(or\\on Windows, or a Windows drive letterX:) are joined toBashParserOptions.WorkingDirectory(lazy fallback toEnvironment.CurrentDirectorywhen null). OnIOException/ path-format exceptions during resolution, fall through toKind = DynamicSkip, IsPath = false, Resolved = null. -
DynamicSkip predicates. A token is
Kind = DynamicSkip, IsPath = false, Resolved = nullwhen:- It contains an unresolved env-var reference (other than
$HOME) in a slot the verb's rule classifies as a path. - Resolution throws an
IOExceptionor path-format exception.
Globs do NOT downgrade to DynamicSkip — they carry their own Kind so consumers can still apply the covering-dir heuristic. Consumers must not use
Rawas a literal path forDynamicSkiptokens. - It contains an unresolved env-var reference (other than
When deciding whether a token "looks like a path" (used to decide whether to apply the resolver):
LooksLikePath(token) =
token starts with '/' (Unix absolute)
|| token starts with '\\' or '<letter>:' (Windows absolute)
|| token starts with './' or '../' (Unix relative)
|| token starts with '~' (Tilde)
|| token contains '/' anywhere
|| token contains '\\' at a NON-TRAILING position
|| token ends with a known file extension (.json, .md, .txt, .conf, ...)
|| token is in the args of a FileVerb at a position the per-verb rule
marks as a path
A lone trailing \\ is excluded because it commonly appears as a
double-quote escape-collapse artifact ("foo\\" lexes to Value foo\\)
and is not a meaningful path signal on its own.
The per-verb rule wins when present; the heuristic is the fallback.
The agent's natural idiom is cd /target && cmd1 && cmd2. Bash semantics:
cmd1 and cmd2 execute with cwd /target. The parser honors this for
path attribution within the same compound.
-
First clause is a
cdorchdirverb: the cd target becomes the attributed cwd for subsequent clauses in the same compound. Onlycdandchdirpropagate attribution per locked interpretation #5.pushd,popd,push-location, andset-locationare still listed inCwdVerbsso their first non-flag positional is path-classified (the target shows up asIsPath=true), but they do not add a synthetic attribution arg to subsequent clauses. A future v0.1.x or v0.2 with PowerShell support may modelpushd/popdas a proper directory stack. -
Subsequent clauses inherit the attributed cwd as if it were prepended with
-Csemantics. Specifically: a syntheticArgwithIsPath=true,Resolved=<cd target>, andKind=Literalis added to each subsequent clause'sArgslist at the end, marked with a flagIsCwdAttribution=trueso consumers can distinguish it from user-emitted args.(Add
IsCwdAttribution: boolto theArgrecord. Default false.) -
A subsequent
cdin the same compound replaces the attributed cwd for clauses after it. (cd /a && cmd1 && cd /b && cmd2→ cmd1 inherits/a, cmd2 inherits/b.) The replacingcd /bitself still receives/aas a synthetic attribution arg (rule 2) before becoming the new source — additive semantics per rule 5. -
Subshell boundaries reset attribution.
cd /a && (cd /b && cmd1) && cmd2: cmd1 (inside subshell) inherits/b; cmd2 (outside subshell) inherits/a(the subshell'scd /bdoes not leak out). A subshell inherits outer attribution on entry (socd /a && (cmd)still attributes cmd to /a) but its own cd changes stay isolated. -
Attribution does not change the clause's verb or original args. The attribution is purely additive — the
cdclause itself is still parsed normally, and subsequent clauses retain everything the user typed, plus the synthetic Arg.
When the cd target itself is Kind=DynamicSkip (e.g. cd $REPO), we
statically don't know the resolved cwd. To preserve the cwd-uncertainty
signal for subsequent clauses:
- A synthetic
Arg { Raw="<dynamic-cwd>", Resolved=null, Kind=DynamicSkip, IsPath=false, IsCwdAttribution=true }is appended to each subsequent clause (instead of the literal-cd flavor). - Relative path args in subsequent clauses are not re-resolved against a
fall-back cwd; they surface as
Kind=DynamicSkip, IsPath=false, Resolved=nullso consumers route to safe-fail rather than trust a guessed working directory.
Consumers that iterate IsPath=true args won't see the synthetic
attribution arg; consumers that specifically check IsCwdAttribution
can detect "this clause's cwd context is unknown" and elevate to
user-prompt instead of treating it like a default-cwd command.
Input: cd /target && git -C /other log && cat file.txt
Parsed clauses:
Clause 0: Operator=None, Verb=[cd], Args=[/target]
Clause 1: Operator=AndIf, Verb=[git, log],
Args=[
Arg{Raw="-C",IsFlag=true},
Arg{Raw="/other",IsPath=true,Resolved="/other"},
Arg{Raw="/target",IsPath=true,Resolved="/target",IsCwdAttribution=true}
]
Clause 2: Operator=AndIf, Verb=[cat],
Args=[
Arg{Raw="file.txt",IsPath=true,Resolved="/target/file.txt"},
Arg{Raw="/target",IsPath=true,Resolved="/target",IsCwdAttribution=true}
]
Note: file.txt in clause 2 resolves against the attributed cwd
/target to produce /target/file.txt. The attributed-cwd Arg is also
appended for completeness, even though the resolver already used it.
Consumers can choose to ignore IsCwdAttribution=true args if they
already see the resolved path in another arg.
Subshells are clauses wrapped in parens: (cd /a && cmd). The parser
recognizes the parens and flattens the subshell's inner clauses into
the parent's Clauses list, marking each with IsSubshell=true so
consumers can distinguish them from outer-compound clauses. A subshell
inherits the outer compound's cd attribution on entry but its own cd
changes stay isolated to the subshell (rule 4 above).
Specifically: (cd /b && cmd) && cmd2 produces three clauses:
Clause 0: Op=None, Verb=cd, Args=[/b], IsSubshell=true
Clause 1: Op=AndIf, Verb=cmd, Args=[/b attribution], IsSubshell=true
Clause 2: Op=AndIf, Verb=cmd2, Args=[] // no /b attribution — subshell isolated
bash -c "inner command" and sh -c "inner command" are common wrappers
the agent emits. The parser:
- Recognizes the
bash -corsh -cprefix. - Parses the quoted argument as a fresh
ParsedCommand. - Surfaces the inner command's clauses inline in the outer's
Clauseslist, each withIsCommandStringWrapped=true.
Example: bash -c "cd /a && cmd" produces:
Clause 0: Op=None, Verb=cd, Args=[/a], IsCommandStringWrapped=true
Clause 1: Op=AndIf, Verb=cmd, Args=[/a attribution], IsCommandStringWrapped=true
The outer bash -c itself does not appear as a clause — it's "consumed"
by the recursion. Consumers that care that this came from a wrapper can
inspect IsCommandStringWrapped on the surfaced clauses.
A bash or sh clause whose authored arguments are dynamic, contain a decoded
-c, or contain a combined short option that may select command-string mode,
but that does not match the complete static wrapper production, remains visible
through its v0.2 compatibility leaf, including its direct outer source spans.
Its v0.3 command occurrence has IsComplete=false. Wrapper-control tokens and
the quoted body must each have literal, exactly-one outer-shell provenance;
token kind or decoded spelling alone is insufficient. A proved -- ends this
conservative option scan. The parser does not claim to have discovered a
dynamic or otherwise unsupported command-string body.
Recursion limit: parse bash -c "bash -c ..." chains up to depth 5.
Deeper nesting → set the outer ParsedCommand.IsUnparseable = true with
reason "bash -c recursion depth exceeded (>5)" per locked interpretation
#4. (Clause has no IsUnparseable field; we surface the overflow on the
top-level ParsedCommand so consumers safe-fail per §11.)
When the parser cannot produce a clean AST:
- Set
ParsedCommand.IsUnparseable = true. - Set
UnparseableReasonto a human-readable diagnostic. - Return empty
CommandsandClauses.Syntaxmay retain partial diagnostic structure, but it is never authorization evidence. Historical v0.1/v0.2 parsers could retain partial clauses; v0.3 deliberately closes that subset-authorization hazard. - Never throw on well-formed input strings (only throw on null).
Conditions that produce IsUnparseable = true:
- Unbalanced quotes (
"foowith no closing"). - Unbalanced parens (
(cmd && cmd2). - Unrecognized control-flow keywords (
for,while,do,done,then,fi,case,esac). - Function definitions (
name() { ... }). - Process substitution (
<(cmd),>(cmd)). - Arithmetic expansion
$((expr))(per §1 non-goal; lexer emits an UNPARSEABLE_SENTINEL token; parser sets the outer flag). - Operator-bearing parameter expansion such as
${var:-$(cmd)}or${var//pat/repl}(per §1 non-goal; same mechanism). Only simple braced identifiers, positional parameters, and special parameters are accepted. - Recursion depth exceeded on
bash -cchains (>5 levels).
Diagnostic precedence. When multiple conditions could fire on a
single input (e.g. case x in a) ;; esac is both a control-flow
keyword AND has unbalanced parens), the parser checks them in this
order so the most informative reason wins:
- Lexer-emitted
UnparseableSentineltokens (unbalanced quote / unterminated heredoc / arithmetic / complex parameter expansion). - Control-flow keyword at verb position (start of input or
immediately after a clause separator
&&,||,;,|, or(). Catchescase x in a) ;; esacbefore the)triggers a paren-balance error. - Function definition pattern (
Wordimmediately followed by(,)). - Process substitution (
<(or>(adjacent). - Segment-split errors (unbalanced parens, unexpected operator).
bash -crecursion depth cap.
Consumers (e.g. Netclaw's gate evaluator) route unparseable commands to a safe-fail path (prompt the user; offer only Once and Deny — no persistent grants on shapes the parser can't model).
A handful of input/expected-AST pairs to anchor understanding. These belong in the corpus (§13) verbatim.
Input: ls -la /tmp
ParsedCommand {
Source = "ls -la /tmp",
IsUnparseable = false,
Clauses = [
Clause {
Operator = None,
Verb = VerbChain { Tokens = ["ls"] },
Args = [
Arg { Raw = "-la", IsFlag = true, Kind = Literal },
Arg { Raw = "/tmp", IsPath = true, Resolved = "/tmp", Kind = Literal }
],
Redirects = [],
IsSubshell = false,
IsCommandStringWrapped = false
}
]
}
Input: git push origin main
Clauses = [
Clause {
Verb = VerbChain { Tokens = ["git", "push", "origin", "main"] },
Args = []
}
]
The greedy heuristic absorbs origin and main because they're
syntactically indistinguishable from subcommand verbs (lowercase
identifiers, no path-shape). Consumers gating on git push * use
pattern-prefix length 2 — see §6.1.1.
Input: freshdesk ticket list --status open
Clauses = [
Clause {
Verb = VerbChain { Tokens = ["freshdesk", "ticket", "list"] },
Args = [
Arg { Raw = "--status", Kind = Literal, IsFlag = true },
Arg { Raw = "open", Kind = Literal, IsPath = false }
]
}
]
The walk stops at --status (a flag with no FlagsWithValue entry for
freshdesk). The full subcommand stack is captured without requiring a
curated table entry — the canonical benefit motivating the change.
Input: cd /target && cmd1 && cmd2 file.txt
Clauses = [
Clause { Verb = [cd], Args = [/target attributed-as-path], Op = None },
Clause {
Verb = [cmd1], Op = AndIf,
Args = [Arg { Raw = "/target", Resolved = "/target",
IsPath = true, IsCwdAttribution = true }]
},
Clause {
Verb = [cmd2], Op = AndIf,
Args = [
Arg { Raw = "file.txt", Resolved = "/target/file.txt", IsPath = true },
Arg { Raw = "/target", Resolved = "/target",
IsPath = true, IsCwdAttribution = true }
]
}
]
Input: git -C /repo log
Clauses = [
Clause {
Verb = VerbChain { Tokens = ["git", "log"] },
Args = [
Arg { Raw = "-C", IsFlag = true },
Arg { Raw = "/repo", IsPath = true, Resolved = "/repo" }
],
Elements = [
ClauseElement { Value = "git", Role = Verb,
PrecedingVerbElementCount = 0 },
ClauseElement { Value = "-C", Role = Argument,
PrecedingVerbElementCount = 1 },
ClauseElement { Value = "/repo", Role = Argument,
PrecedingVerbElementCount = 1 },
ClauseElement { Value = "log", Role = Verb,
PrecedingVerbElementCount = 1 }
]
}
]
Input: cmd > /tmp/out.txt
Clauses = [
Clause {
Verb = [cmd],
Args = [],
Redirects = [Redirect { Direction = Out, Target = "/tmp/out.txt" }]
}
]
Input: cd /a && (cd /b && cmd1) && cmd2
Clauses = [
Clause { Verb = [cd], Args = [/a], Op = None },
Clause { Verb = [cd], Args = [/b], Op = AndIf, IsSubshell = true,
Args = [/a attribution from outer compound] },
Clause { Verb = [cmd1], Op = AndIf, IsSubshell = true,
Args = [/b attribution — local to subshell] },
Clause { Verb = [cmd2], Op = AndIf,
Args = [/a attribution — inherited from outer cd, NOT /b] }
]
Input: rm $UNRESOLVED/foo
Clauses = [
Clause {
Verb = [rm],
Args = [
Arg { Raw = "$UNRESOLVED/foo", Kind = DynamicSkip, IsPath = false,
Resolved = null }
]
}
]
Consumer impact: zone-gate sees zero paths to evaluate; routes to the fallback "treat as one untrusted path = the raw token" prompt.
Input: for ((i = $(next); i < 10; i++)); do run "$i"; done
ParsedCommand {
Source = "for ((i = $(next); i < 10; i++)); do run \"$i\"; done",
IsUnparseable = true,
UnparseableReason = "C-style loops and arithmetic execution are unsupported",
Syntax = ShellBlockSyntax { ... } // optional diagnostic evidence only
Commands = [],
Clauses = []
}
The corpus is the acceptance contract for the parser. Implementation is "done" when every corpus entry parses to its expected AST.
tests/ShellSyntaxTree.Tests/Corpus/bash/*.json — one file per corpus
entry. File name pattern: NN_descriptive_slug.json where NN is a
zero-padded sequence number.
Each file:
{
"name": "Multi-token verb: git push",
"input": "git push origin main",
"expected": {
"isUnparseable": false,
"clauses": [
{
"operator": "None",
"verb": ["git", "push"],
"args": [
{ "raw": "origin", "kind": "Literal", "isPath": false },
{ "raw": "main", "kind": "Literal", "isPath": false }
],
"redirects": [],
"isSubshell": false,
"isCommandStringWrapped": false
}
]
},
"notes": "Optional explanation of edge case being captured."
}An entry may set bashInitialStateMode to Unknown or
IsolatedNonInteractive when the expected result depends on the caller-proved
Bash variable-state contract. When omitted, the Bash corpus runner uses
IsolatedNonInteractive. The field is rejected outside the Bash corpus.
An entry may add an elements list to a clause to pin the complete
Clause.Elements projection (raw, value, role, sourceStart,
sourceLength, precedingVerbElementCount, kind, isFlag, isPath, and
resolved). The field is opt-in so older corpus entries remain readable;
issue-specific provenance entries SHALL include it.
An entry may also add both of the following v0.3 structural expectations:
syntaxis the completeParsedCommand.Syntaxtree flattened in preorder. Each item recordskind,parentIndex, the incoming ancestryregionandchildIndex, exact-or-nullsourceStart/sourceLength, and the kind-specificclauseIndex,groupKind, orlistOperator.clauseIndexidentifies the exact compatibilityClauseinstance owned by aSimpleCommandnode; it is not a copied value comparison.commandsis the completeParsedCommand.Commandsprojection in authored order. Each item records itsclauseIndex,immediateRole,isComplete, and outermost-to-innermostancestryframes. Each frame recordsancestorKind,region,childIndex, and exact-or-null source range.
These fields are independently opt-in so legacy corpus entries retain their
v0.2 shape; structural acceptance cases normally provide both. When present,
the runner compares every node, relationship, range, occurrence, role,
completeness bit, ancestry frame, and Clause reference. Unknown JSON members
are rejected. An unparseable result always asserts empty Clauses and
Commands, even when those arrays are omitted from the JSON.
The corpus runner also lexes every direct, parseable input and verifies that each authored verb, argument, opaque region, and redirect token is covered by exactly-positioned clause-element provenance. This invariant applies even when an older entry omits the optional field, preventing silent argument loss across the legacy corpus.
Author at least:
- 10 simple-verb cases (ls, pwd, echo, cat, grep, etc.)
- 10 multi-token-verb cases (git push, dotnet test, docker compose up, etc.)
- 15 compound cases (
&&,||,;,|combinations) - 10
cd-in-compound propagation cases (single, sequential, with subshell) - 10 quote-handling cases (single, double, escaped, mixed)
- 10 redirect cases (
>,>>,<,2>,2>>, multiple redirects) - 10 subshell cases (with and without isolation effects)
- 10
bash -crecursion cases (depth 1, 2, with inner compounds) - 10 dynamic-skip cases (
$VAR,${VAR},~user, glob args) - 10 per-verb path-rule cases (chmod, chown, find, grep, curl, git -C, etc.)
- 10 unparseable cases (unbalanced quotes, control-flow keywords, function definitions)
Total minimum: 105 entries. Strive for 150+ once seeded from sanitized real-world commands (see §14).
A single xunit test method enumerates tests/ShellSyntaxTree.Tests/Corpus/bash/*.json, parses
each input, and asserts the result matches expected field-by-field.
The runner emits a per-corpus-entry test name so failures point at the
specific case.
[Theory]
[MemberData(nameof(CorpusEntries))]
public void Corpus_entry_parses_to_expected_ast(CorpusEntry entry)
{
var parser = new BashParser();
var actual = parser.Parse(entry.Input);
AstAssert.Equal(entry.Expected, actual); // structural equality
}
public static IEnumerable<object[]> CorpusEntries()
{
var dir = Path.Combine(AppContext.BaseDirectory, "Corpus", "bash");
foreach (var file in Directory.GetFiles(dir, "*.json"))
{
var entry = JsonSerializer.Deserialize<CorpusEntry>(File.ReadAllText(file));
yield return [entry];
}
}AstAssert.Equal is a helper that does structural equality with helpful
diff messages on mismatch. Implement to taste.
A portion of the corpus seeds from real shell commands captured from
agent dogfood logs. The seed source is a daemon log file at
~/.netclaw/logs/daemon-2026-05-09.log (and similar). These logs contain
PII (usernames, repo paths, channel/thread IDs) that must not appear
in the public corpus.
Apply these transformations to every seeded entry before committing:
| Pattern | Replacement |
|---|---|
/home/<username>/ (any specific username) |
/home/user/ |
/Users/<username>/ (macOS) |
/Users/user/ |
~/<username>/ |
~/ |
Specific repo paths like /home/user/repositories/stannardlabs/<repo> |
/home/user/repos/sample-repo |
| Specific repo names (not in the org's public list) | sample-repo or project |
Slack channel IDs (D[A-Z0-9]{10}) |
<channel> (only if appears in command) |
Slack thread IDs (\d{10}\.\d{6}) |
<thread> |
| Internal hostnames | internal-host.example |
| Email addresses | user@example.com |
API keys, tokens, secrets (any [A-Za-z0-9]{20,} that looks key-shaped) |
<redacted> (but prefer to drop the entry entirely) |
- Pull candidate commands from logs:
grep -oP "command \K\{[^}]+\}" ~/.netclaw/logs/daemon-*.log \ | jq -r .Command | sort -u > /tmp/raw-corpus.txt
- Apply sanitization (script TBD) — for each line, walk the table above.
- Manual review of each sanitized entry before committing. The script can miss patterns; a human (or careful agent) reviews for residual PII.
- Drop any entry that can't be cleanly sanitized (too many specific identifiers; rewrite as a fully-synthetic entry instead).
- Commit with a clear message:
chore(corpus): seed from sanitized agent logs (NN entries).
Before any corpus PR merges, CI runs a regex check against the corpus
files for residual PII patterns. The check fails the build if any
sanitization-rule pattern appears in any committed corpus file. Implement
as a small dotnet test that scans tests/ShellSyntaxTree.Tests/Corpus/bash/*.json for the
forbidden patterns.
The repo template already has:
.github/workflows/pr_validation.yml— runsdotnet teston PR..github/workflows/publish_nuget.yml— publishes to NuGet on release tag.
Adapt for ShellSyntaxTree:
- Trigger NuGet publish on a bare SemVer tag (for example,
0.3.0-alpha). A leadingvis invalid. - Test job runs the corpus runner plus all unit tests.
- PII audit job runs the sanitization-pattern scan over
tests/ShellSyntaxTree.Tests/Corpus/.
- v0.1.x-alpha — pre-release alpha cycle. Public API surface per §2 is
locked; internal data and behavior are subject to course-correction
while real-world feedback lands (e.g. v0.1.4-alpha replaces the
BashAritystatic table with the greedy verb-chain heuristic per issue #27). - v0.1.0 — first publishable non-alpha cut. Bash-only.
- v0.1.x (post-0.1.0) — additive changes and SPEC-conformance fixes (more verb table entries, more corpus, bug fixes). A fix may shift the parsed-AST shape when the prior shape violated this SPEC — e.g. v0.1.5 makes a bare newline a statement separator per §4. The §2 public API surface stays locked.
- v0.2.0 — first PowerShell parser implementation (
PwshParser). Adds the sharedShellParserOptionsbase, the additiveVerbChain.CanonicalVerb/VerbChain.IsDynamicfields, and the breakingClause.IsBashCWrapped→IsCommandStringWrappedrename. A breaking AST change on a0.xminor is permitted by Appendix A whenRELEASE_NOTES.mdcarries the old→new mapping and Netclaw is updated in lockstep. SeeSPEC.POWERSHELL.md. - v1.0.0 — ready when at least one external consumer beyond Netclaw ships against it without finding API gaps.
Update RELEASE_NOTES.md for each tagged release. Format:
0.1.0-alpha YYYY-MM-DD
* First publishable cut.
* Bash parser per SPEC.md v0.1.
* Corpus: N entries.
* Public API: IShellParser, BashParser, ParsedCommand, Clause, VerbChain,
Arg, Redirect, ArgKind, RedirectDirection, CompoundOperator.
A natural order for the implementer:
- Bootstrap projects. Create
src/ShellSyntaxTree/ShellSyntaxTree.csproj(library) andtests/ShellSyntaxTree.Tests/ShellSyntaxTree.Tests.csproj(xunit). UpdateSampleSln.slnx(rename toShellSyntaxTree.slnx) and delete theAkka.Consolesample. - Update template defaults.
Directory.Build.props: replace Akka metadata with ShellSyntaxTree.README.md: real intro.LICENSE: keep Apache-2.0 (already correct).Directory.Packages.props: add xunit, drop Akka.Hosting.Tags: bash, shell, parser, ast. - Write public API skeleton (§2): interface + record stubs that compile
but throw
NotImplementedExceptiononParse(). Lock the surface first. - Implement BashLexer (§5). Heavy unit tests on tokenization.
- Implement FILE / CWD verb tables and IsVerbLikeToken predicate (§6) as static data + helper.
- Implement BashParser (§4). One production at a time; unit-test each.
- Implement Resolver (§8). Unit-test each resolution rule.
- Implement per-verb path-arg rules (§7). Unit-test per verb.
- Implement cd-in-compound propagation (§9). Unit-test.
- Implement subshell + bash -c recursion (§10). Unit-test.
- Implement parser anomaly safe-fail (§11). Unit-test.
- Author corpus (§13) — start with 105 hand-authored entries covering each section. Iterate parser to make all pass.
- Sanitize and seed from real logs (§14) — script + manual review. Add 50-100 more corpus entries.
- Wire CI (§15). Tag
0.1.0-alphawhen the corpus is green and the PII audit passes.
Estimated implementation effort: 600-800 LOC of source + 400-600 LOC of test infrastructure + 100-150 corpus entries (~50 KB JSON).
Post-v0.1.0 increments (e.g. v0.1.5 newline-as-statement-separator) are
sequenced through IMPLEMENTATION_PLAN.md — §16 records the one-time
v0.1.0 build order, not the ongoing changelog.
v0.1.0-alpha ships when all of the following hold:
- ✅ Public API matches §2 exactly.
dotnet packproduces a ShellSyntaxTree.0.1.0-alpha.nupkg. - ✅ Every corpus entry in
tests/ShellSyntaxTree.Tests/Corpus/bash/*.jsonparses to its expected AST.dotnet testruns them all and passes. - ✅ Corpus has at least 105 entries spanning the categories in §13.
- ✅ PII audit scan over
tests/ShellSyntaxTree.Tests/Corpus/bash/*.jsonfinds zero hits. - ✅
dotnet testruns on PR via GitHub Actions and passes. - ✅ Tagging
0.1.0-alphatriggerspublish_nuget.ymland the package appears on nuget.org. - ✅ Netclaw can consume the package via
<PackageReference>and theIShellParserresolves at runtime in Netclaw's DI container. - ✅ At least one Netclaw integration test exercises a real corpus entry through the live Netclaw matcher and gets the expected gate decision.
Stable v0.3 deliberately continues to exclude:
- Windows
cmdparsing. - Command execution, filesystem glob enumeration, runtime variable lookup, or live-shell evaluation.
- Executable-specific option, operand, object, revision, or subcommand grammars; consumers own those semantics.
- Bash
while,until,if,elif,else, process substitution, single-&background lists,case, C-style or implicit positional-parameter loops, arithmetic execution, functions, and definitions until each hidden-execution and state boundary is specified. - PowerShell
while,if,elseif,else,do,switch, functions, definitions, class/type bodies, arbitrary execution-bearing expressions, and.ps1file-content parsing. - A stable serialized wire format for the polymorphic v0.3 records.
- Caller-configurable analysis limits, filesystem-dependent pattern expansion, or unbounded value/state alternatives.
- Full IDE-style concrete syntax mapping. Exact-or-null source ranges exist for security correlation, not lossless editing.
- Performance optimization beyond "fast enough" (~1ms typical).
- Extensible verb-table loading from config.
What a v0.3 Netclaw-style security consumer expects from this library:
IShellParser.Parse(string)returns oneParsedCommand. IfIsUnparseableis true orCommandsis empty, authorization prompts or denies; neither partialSyntaxnor raw-prefix inference can authorize.- The consumer evaluates every
CommandOccurrence, including condition, iterator, branch, body, substitution, and pipeline-stage occurrences.Syntaxmay group the UI but is not the command-discovery API. - An incomplete occurrence, dynamic authored verb, unknown or unrecognized role, ancestry kind, value kind, redirect kind, or policy-sensitive fact prompts or denies. Unknown executable operands are never dropped to reuse a broader approval. Ambient runtime resolution does not by itself make a static authored occurrence incomplete; the approval covers the command text the user was shown.
- For every exact or finite effective value, the consumer reapplies the shell's binding rules and the complete executable-specific grammar at the candidate's authored position. A finite shell proof is not authorization; option-like candidates remain option-like.
- Every
RedirectAnalysisis evaluated. Static descriptor operations are not paths. File targets are path-relevant. Complete heredoc/here-string bodies remain data unless executable-specific stdin policy makes that data sensitive; expanding-body substitutions appear as separate commands. - Hard-deny and protected-path rules still precede reusable grants. Stored approval never bypasses a hard deny, and every command occurrence is evaluated.
- A v0.2 consumer may temporarily continue reading
Clauses. The projection includes every authored simple command from supported nested syntax and retains dynamic authored operands, but it does not expose proved effective loop values. Migration toCommandsis required for bounded reuse.
The contract is extend-only — additive records and members with safe defaults
are source and binary compatible; renaming, removing, or changing signatures
is breaking. Adding members changes generated record equality, hashing,
ToString(), and default reflection serialization. Consumers that persist
results own a versioned DTO or serializer mapping rather than treating the
in-memory hierarchy as a stable wire union.
Before v1.0.0, while the library is in its 0.x line, a breaking AST change
MAY ship in a minor bump (e.g. the Clause.IsCommandStringWrapped →
IsCommandStringWrapped rename in v0.2.0) provided RELEASE_NOTES.md
documents the old→new mapping and the consumer (Netclaw) is updated in
lockstep. From v1.0.0 onward, renaming or removing a field requires a major
version bump.
OpenCode (Node) uses tree-sitter-bash. We considered porting that approach to .NET. The packaging cost is real:
- No first-class .NET tree-sitter binding. Community bindings exist but vary in maintenance.
- Native dependency: ship
libtree-sitter+libtree-sitter-bashper platform (Linux x64, Linux arm64, macOS x64, macOS arm64, Windows x64). Five binaries to ship and maintain, plus PowerShell would need a separate native lib. - AOT-trimming compatibility is uncertain.
- We don't need IDE-grade fidelity. Fork bombs and function definitions
legitimately confuse our parser; we want them to mark
IsUnparseableso the consumer routes to safe-fail. tree-sitter would parse them and we'd have to teach the consumer to ignore the result anyway.
The hand-rolled approach trades a higher ceiling for control over scope,
zero native deps, and a clean upgrade path to PowerShell via the same
IShellParser seam. For our use case, that trade is correct.