Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# PRF CI gates — macOS-first lane (owner decision).
# Follow-up lanes planned: ubuntu-latest (pwsh 7 cross-check) and
# windows-latest (Windows PowerShell 5.1 matrix). This repo is not yet pushed;
# the workflow activates automatically on first push.
# windows-latest (Windows PowerShell 5.1 matrix). Note: PSScriptAnalyzer 1.25
# does not load under 5.1, so the windows lane needs PSSA <=1.21.
name: ci

on:
Expand Down
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,19 +26,21 @@ PowerShell-Reliability-Framework/
│ ├── compatibility.md # 5.1 ↔ 7.x delta matrix, module availability reality
│ ├── security-rules.md # attack surface & defensive generation posture
│ ├── enterprise-patterns.md # ShouldProcess, logging, remoting, bulk-op etiquette
│ └── anti-patterns.md # AP-01..AP-24 catalog (bad → good)
│ ├── anti-patterns.md # AP-01..AP-24 catalog (bad → good)
│ └── here-strings-and-heredocs.md # delimiter grammar, bash heredoc collision, writer divergence
├── rules/
│ └── reliability-rules.yaml # machine-readable rule corpus (v0.1: 100 rules)
│ └── reliability-rules.yaml # machine-readable rule corpus (v0.1: 109 rules)
├── examples/
│ ├── bad/ # 10 negative samples (NEVER execute)
│ ├── good/ # 10 matching positive samples
│ ├── bad/ # 12 negative samples (NEVER execute)
│ ├── good/ # 12 matching positive samples
│ └── EXAMPLES.md # pair index
├── benchmark/
│ └── cases.json # reliability benchmark (v0.1: 100 cases)
│ └── cases.json # reliability benchmark (v0.1: 110 cases)
├── tools/
│ ├── probe-environment.ps1 # canonical target-machine probe
│ ├── validate.py # package integrity gate (L0)
│ └── rule-selftest.py # detector-vs-snippet consistency gate (L1)
├── .github/workflows/ci.yml # CI gates (macOS-first lane)
└── docs/
├── README.zh-CN.md # 简体中文说明
└── testing-plan.md # testing & evaluation plan (A/B protocol)
Expand Down Expand Up @@ -69,7 +71,7 @@ PowerShell-Reliability-Framework/
## Rule schema

Each rule in `rules/reliability-rules.yaml` carries: `id · category · severity · title · description · bad · good · rationale · applies_to · tags · detection{pattern, psscriptanalyzer}`.
36 of 100 rules currently include detection hints (`detection_rule_count` in the YAML header); coverage grows with v0.2 static-analysis integration.
45 of 109 rules currently include detection hints (`detection_rule_count` in the YAML header); coverage grows with v0.2 static-analysis integration.

## Roadmap

Expand All @@ -86,10 +88,10 @@ Each rule in `rules/reliability-rules.yaml` carries: `id · category · severity

| Metric | Target | Current |
|---|---:|---:|
| Benchmark cases | 100 | 100 ✅ |
| Rules | 100+ | 100 |
| Knowledge docs | 5 | 5 |
| Example pairs | — | 10 |
| Benchmark cases | 100 | 110 ✅ |
| Rules | 100+ | 109 |
| Knowledge docs | 5 | 6 |
| Example pairs | — | 12 |
| Agent support | 3+ | any Skills-capable agent (portable format) |

Measured effectiveness targets (A/B protocol in `docs/testing-plan.md`) are being
Expand Down
22 changes: 19 additions & 3 deletions SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: powershell-reliability-framework
description: PRF v0.1.0 reliability layer for AI-generated PowerShell. Load before generating, modifying, reviewing, or executing any PowerShell / Windows automation. Enforces an Environment-Probe -> Operation-Classification -> Risk-Gate protocol plus hard safety, compatibility, idempotency and error-handling rules, and a propose(-WhatIf)-then-execute loop.
description: PRF v0.1.0 reliability layer for AI-generated PowerShell. Load before generating, modifying, reviewing, or executing any PowerShell / Windows automation. Enforces an Environment-Probe -> Operation-Classification -> Risk-Gate protocol plus hard safety, compatibility, idempotency, error-handling and here-string/heredoc rules, and a propose(-WhatIf)-then-execute loop.
version: 0.1.0
---

Expand All @@ -19,6 +19,8 @@ Apply whenever you are asked to produce, change, review, explain-for-execution,
- remoting (`Invoke-Command`/`Enter-PSSession`), scheduled tasks, services, processes
- registry, filesystem ACLs, firewall, network configuration
- Active Directory, Azure (`Az.*`), Microsoft Graph, Exchange Online operations
- here-strings (`@'...'@`, `@"..."@`), generated `.ps1` bodies, or any PowerShell
produced from bash/sh on a CI runner for execution on a Windows host

## 1. Generation Protocol (mandatory, in order)

Expand Down Expand Up @@ -74,12 +76,19 @@ A single request can span several classes; classify the *highest* class involved
echo secrets into output, logs or transcripts. (SEC-014..)
8. **Observable.** Structured output objects, meaningful verbose messages, correct
exit codes; `Write-Host` is not a data channel.
9. **Strings are carried correctly.** Both here-string delimiters at column one
(HD-001, HD-002); no bash `<<EOF` in PowerShell, which tokenizes as file
redirection and runs nothing (HD-003); `-Encoding UTF8` is not a cross-runtime
fix — 5.1 still writes a BOM (HD-004); no unquoted `-Command` string into a
POSIX shell (HD-005); no script body through `Invoke-Expression` (HD-006);
one stdin cannot carry a script and a payload (HD-007); generated artifacts
written with `[IO.File]::WriteAllText`, not piped into a cmdlet sink (HD-009).

### Step 4 — Self-review against the rule corpus

Before answering, scan your draft against `rules/reliability-rules.yaml`
(categories: `compatibility`, `safety`, `idempotency`, `security`,
`error-handling`). If you intentionally deviate, quote the violated rule id and
`error-handling`, `heredoc`). If you intentionally deviate, quote the violated rule id and
justify it in one sentence.

### Step 5 — Propose → Confirm → Execute → Verify → Report
Expand All @@ -102,8 +111,14 @@ justify it in one sentence.
| `Copy-Item a b` bare | `try { Copy-Item a b -ErrorAction Stop } catch { Write-Error $_; exit 1 }` |
| `ForEach-Object -Parallel` | plain pipeline unless probe proved PowerShell 7+ |
| `Get-WmiObject Win32_Service` | `Get-CimInstance Win32_Service` |
| indented `'@` terminator | `'@` at column one — indentation is a parse error, not a style choice |
| `cmd <<EOF` in pwsh | rewrite in PowerShell — `<<` is file redirection here, nothing runs |
| `Set-Content -Encoding UTF8` on a 5.1 host | `[IO.File]::WriteAllText(... , (New-Object System.Text.UTF8Encoding $false))` |
| `@'...'@ \| Out-File f` | build the string, then `WriteAllText` — cmdlet sinks add CRLF and a runtime-dependent default encoding |
| `pwsh -Command "...\$var..."` from bash | `pwsh -File script.ps1` — write the script from a quoted `<<'PSEOF'` heredoc first |

Full catalog with explanations: `knowledge/anti-patterns.md`.
Here-string and heredoc specifics: `knowledge/here-strings-and-heredocs.md`.

## 3. Deeper references

Expand All @@ -114,8 +129,9 @@ Full catalog with explanations: `knowledge/anti-patterns.md`.
| `knowledge/security-rules.md` | attack surface & defensive generation posture |
| `knowledge/enterprise-patterns.md` | ShouldProcess, logging, remoting, bulk ops |
| `knowledge/anti-patterns.md` | AP-01..AP-24 catalog with repairs |
| `knowledge/here-strings-and-heredocs.md` | delimiter grammar, bash heredoc collision, 5.1 ↔ 7.x writer divergence, POSIX transport layer |
| `examples/bad/`, `examples/good/` | paired negative/positive scripts |
| `benchmark/cases.json` | scored evaluation rubric (v0.1: 100 cases) |
| `benchmark/cases.json` | scored evaluation rubric (v0.1: 110 cases) |

## 4. Escalation triggers — stop and ask the human

Expand Down
Loading