Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "purlin",
"version": "0.9.4",
"version": "0.9.5",
"description": "Rule-proof spec-driven development — specs define rules, tests prove them, sync_status shows coverage",
"author": {
"name": "Rich LaBarca",
Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/windows-proofs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: windows-proofs

# Runs the Windows-native proofs for static_checks (RULE-29/30) on a real windows-latest
# runner and commits the emitted `windows`-tier proof file back to the branch, so
# purlin:verify/purlin:status can read genuine Windows verification — not host simulation.

on:
push:
# Skip the runner's own proof commit (loop guard, paired with [skip ci]).
paths-ignore:
- '**/*.proofs-windows.json'
workflow_dispatch:

permissions:
contents: write

jobs:
windows-native:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: true

- uses: actions/setup-python@v5
with:
python-version: '3.11'

- name: Install pytest
run: pip install pytest

- name: Run Windows-native proofs
run: python -m pytest dev/test_windows_native.py -v

- name: Commit Windows proof file back to the branch
shell: bash
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add specs/audit/static_checks.proofs-windows.json
if git diff --cached --quiet; then
echo "No proof-file changes to commit."
else
git commit -m "test(static_checks): windows-native proofs from windows-latest [skip ci]"
git push origin HEAD:${GITHUB_REF_NAME}
fi
2 changes: 1 addition & 1 deletion .purlin/report-data.js

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@

### Added

- **Deterministic Pass-1 coverage for C#/.NET tests.** Pass 1 previously recognized only pytest/Jest/Shell markers, so xUnit/NUnit/MSTest tests carrying `[Trait("PurlinProof", "feature:PROOF-N:RULE-N:tier")]` produced zero proofs and the structural checker silently no-op'd on `.cs` files. The new `check_csharp` analyzer parses those trait markers, locates each `[Fact]`/`[Theory]` method body via a C#-aware brace/string scanner, and runs assert-true / no-assertion detection — recognizing xUnit `Assert.*`, NUnit `Assert.That`, MSTest `Assert.*`, and FluentAssertions `.Should()` as assertions (`static_checks` RULE-31). The dispatch was extracted into `analyze_test_file()` and now routes `.cs`. `references/supported_frameworks.md` documents the new coverage.
- **Deterministic Pass-1 coverage for C#/.NET tests.** Pass 1 previously recognized only pytest/Jest/Shell markers, so xUnit/NUnit/MSTest tests carrying `[Trait("PurlinProof", "feature:PROOF-N:RULE-N:tier")]` produced zero proofs and the structural checker silently no-op'd on `.cs` files. The new `check_csharp` analyzer parses those trait markers, locates each `[Fact]`/`[Theory]` method body via a C#-aware brace/string scanner, and runs assert-true / no-assertion detection — recognizing xUnit `Assert.*`, NUnit `Assert.That`, MSTest `Assert.*`, FluentAssertions `.Should()`, and Playwright fluent assertions (`Expect(...)`/`Assertions.Expect(...)` chained to a `To*Async()` matcher) as assertions (`static_checks` RULE-31). A bare `Expect(x)` with no matcher chain is still flagged `no_assertions`. The dispatch was extracted into `analyze_test_file()` and now routes `.cs`. `references/supported_frameworks.md` documents the new coverage. To keep this broader structural acceptance from masking weak tests, the Pass-2 audit criteria gained a "presence/visibility-only assertion" WEAK check (`references/audit_criteria.md` → Criteria-Version 17).
- **Portable interpreter guidance in `purlin:audit`.** `skills/audit/SKILL.md` documents a `python3 → python → py -3` fallback for invoking `static_checks.py`, since stock Windows does not put `python3` on PATH (`skill_audit` RULE-15).
- **`purlin:audit` reaches C# tests even when `test_file` is empty.** Under `dotnet test`, `TestCase.CodeFilePath` is often null (no surfaced source info), so the xUnit logger emits proof entries with a blank `test_file` — which previously blocked the audit skill from locating the `.cs` code to scan (Pass 1) or read (Pass 2). `static_checks.py` now exposes `--resolve-source <test_name>`, which derives the declaring type from the fully-qualified `test_name` and locates its `.cs` file (skipping `bin`/`obj`), and `skills/audit/SKILL.md` uses it as a fallback (`static_checks` RULE-32, `skill_audit` RULE-16). Populating `test_file` natively still requires `RunConfiguration.CollectSourceInformation=true` with full PDBs — now documented.

### Testing

- New proofs: `static_checks` PROOF-44/45 (AST structural guards — no unconditional `fcntl` import; `encoding='utf-8'` on every text-mode `open()`), PROOF-46–49 (C# assert-true / no-assertion / multi-framework assertion recognition / `.cs` dispatch), PROOF-50 (Windows lock path driven through an injected fake `msvcrt`, `@integration`), PROOF-51 (full CLI pipeline via real subprocess, `@e2e`), PROOF-52 (the cp1252 failure reproduced by running `--load-criteria` under `PYTHONUTF8=0`/`LC_ALL=C`, `@e2e`); `skill_audit` PROOF-15. Windows behavior is verified by simulation (fake `msvcrt`) plus a faithful ASCII-locale subprocess repro — not yet a Windows CI runner. Independent audit of the new proofs: 7/7 behavioral proofs STRONG (100% integrity), 0 WEAK/HOLLOW. `static_checks` VERIFIED at 29/29, `skill_audit` at 15/15.
- New proofs: `static_checks` PROOF-44/45 (AST structural guards — no unconditional `fcntl` import; `encoding='utf-8'` on every text-mode `open()`), PROOF-46–49 (C# assert-true / no-assertion / multi-framework assertion recognition / `.cs` dispatch), PROOF-50 (Windows lock path driven through an injected fake `msvcrt`, `@integration`), PROOF-51 (full CLI pipeline via real subprocess, `@e2e`), PROOF-52 (the cp1252 failure reproduced by running `--load-criteria` under `PYTHONUTF8=0`/`LC_ALL=C`, `@e2e`); `skill_audit` PROOF-15. PROOF-55 covers the Playwright assertion recognition (and the bare-`Expect` negative). Windows behavior is now verified for real on a `windows-latest` GitHub Actions runner — `.github/workflows/windows-proofs.yml` runs PROOF-53 (native `msvcrt` lock path, `_HAS_FCNTL` genuinely False) and PROOF-54 (`--load-criteria` under the native console codec) in the dedicated `@windows` proof tier and commits the results back — in addition to the host simulation (fake `msvcrt`) and ASCII-locale subprocess repro. Independent audit of the new proofs: STRONG at 100% integrity, 0 WEAK/HOLLOW.

## v0.9.4 — Plugin-bundled MCP server & e2e proof quality

Expand Down
5 changes: 3 additions & 2 deletions dev/test_schema_proof_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ class TestProofFormatConventions:

@pytest.mark.proof("schema_proof_format", "PROOF-4", "RULE-4")
def test_standard_tiers_documented(self):
valid_tiers = {'unit', 'integration', 'e2e'}
valid_tiers = {'unit', 'integration', 'e2e', 'windows'}
# Verify all existing proof files only use valid tiers
proof_files = glob.glob(os.path.join(PROJECT_ROOT, 'specs', '**',
'*.proofs-*.json'), recursive=True)
Expand All @@ -177,12 +177,13 @@ def test_standard_tiers_documented(self):
assert entry_tier in valid_tiers, \
f"Invalid entry tier '{entry_tier}' in {entry.get('id')} of {path}"

# Verify format reference documents all 3 standard tiers
# Verify format reference documents the standard tiers + the platform-gated windows tier
with open(os.path.join(PROJECT_ROOT, 'references', 'formats',
'spec_format.md')) as f:
fmt = f.read()
assert '@integration' in fmt
assert '@e2e' in fmt
assert '@windows' in fmt

@pytest.mark.proof("schema_proof_format", "PROOF-6", "RULE-6")
def test_proof_files_not_gitignored(self):
Expand Down
12 changes: 12 additions & 0 deletions dev/test_skill_specs.py
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,18 @@ def test_documents_portable_interpreter_fallback(self):
assert '`python`' in content or 'then `python`' in content or 'python`,' in content, \
"audit SKILL.md must document the 'python' interpreter fallback"

@pytest.mark.proof("skill_audit", "PROOF-16", "RULE-16")
def test_documents_empty_test_file_fallback(self):
"""SKILL.md documents resolving an empty test_file from test_name via
--resolve-source, and names CollectSourceInformation as the native fix."""
content = _read('audit')
assert '--resolve-source' in content, \
"audit SKILL.md must document the --resolve-source fallback"
assert 'test_file' in content and 'test_name' in content, \
"audit SKILL.md must explain resolving an empty test_file from test_name"
assert 'CollectSourceInformation' in content, \
"audit SKILL.md must name CollectSourceInformation as the native way to populate test_file"


# ── skill_build ───────────────────────────────────────────────────────

Expand Down
66 changes: 66 additions & 0 deletions dev/test_static_checks.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,16 @@
compute_proof_hash,
prune_audit_cache,
read_audit_cache,
resolve_test_file_from_name,
write_audit_cache,
_read_rule_descriptions,
load_criteria,
)

_STATIC_CHECKS_PY = os.path.join(
os.path.dirname(__file__), '..', 'scripts', 'audit', 'static_checks.py'
)

STATIC_CHECKS_PY = os.path.join(
os.path.dirname(__file__), '..', 'scripts', 'audit', 'static_checks.py'
)
Expand Down Expand Up @@ -1467,3 +1472,64 @@ def test_dispatch_routes_cs_to_check_csharp(self):
os.unlink(other)
finally:
os.unlink(path)

@pytest.mark.proof("static_checks", "PROOF-55", "RULE-31")
def test_recognizes_playwright_fluent_assertions(self):
"""A C# test asserting only via Playwright's Expect(...).To*Async() is recognized
as an assertion (status=pass), but a bare Expect(x) with no matcher is still flagged."""
# Mirrors the PR #4 repro: Nav_ShowsUser_AndSignOut asserts only through Playwright.
playwright = (
'await Assertions.Expect(page.Locator("text=" + user)).ToBeVisibleAsync();\n'
'await Assertions.Expect(page.Locator("a[href=\'logout\']")).ToBeVisibleAsync();'
)
path = self._cs(playwright)
try:
results = check_csharp(path, "csfeat")
assert len(results) == 1
assert results[0]['status'] == 'pass', \
f"Playwright assertion not recognized — {results[0]}"
finally:
os.unlink(path)

# Bare Expect(x) with no To*Async() matcher chain is NOT an assertion.
bare = self._cs("Expect(result);")
try:
results = check_csharp(bare, "csfeat")
assert len(results) == 1
assert results[0]['status'] == 'fail'
assert results[0]['check'] == 'no_assertions', \
f"bare Expect(x) should be no_assertions — {results[0]}"
finally:
os.unlink(bare)

@pytest.mark.proof("static_checks", "PROOF-56", "RULE-32")
def test_resolve_test_file_from_name(self):
"""When a proof's test_file is empty (C#/xUnit under dotnet test), the
source file is resolved from the fully-qualified test_name by locating the
declaring class — preferring the authored file over a bin/ build copy."""
with tempfile.TemporaryDirectory() as root:
os.makedirs(os.path.join(root, 'tests'))
os.makedirs(os.path.join(root, 'bin'))
authored = os.path.join(root, 'tests', 'AuthLogicTests.cs')
with open(authored, 'w', encoding='utf-8') as f:
f.write('namespace Demo.Tests {\n public class AuthLogicTests {\n'
' [Fact] public void Evaluate_NullRow() { }\n }\n}\n')
# A build-output copy that must be skipped.
with open(os.path.join(root, 'bin', 'AuthLogicTests.cs'), 'w', encoding='utf-8') as f:
f.write('public class AuthLogicTests { }\n')

got = resolve_test_file_from_name(
'Demo.Tests.AuthLogicTests.Evaluate_NullRow', root)
assert got == 'tests/AuthLogicTests.cs', f"resolved to {got!r}"

# A test_name whose class is not declared anywhere resolves to ''.
assert resolve_test_file_from_name('Demo.Tests.MissingClass.X', root) == ''

# The --resolve-source CLI prints JSON with the resolved test_file.
r = subprocess.run(
[sys.executable, _STATIC_CHECKS_PY, '--resolve-source',
'Demo.Tests.AuthLogicTests.Evaluate_NullRow', '--project-root', root],
capture_output=True, text=True)
assert r.returncode == 0, r.stderr
payload = json.loads(r.stdout)
assert payload['test_file'] == 'tests/AuthLogicTests.cs', payload
86 changes: 86 additions & 0 deletions dev/test_windows_native.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
"""Windows-native proofs for static_checks (RULE-29/30).

These verify the real Windows code paths — the native `msvcrt.locking` file lock and
UTF-8 text I/O under the native console codec — on an actual `windows-latest` runner,
not the host simulations (PROOF-50 fake msvcrt, PROOF-52 simulated ASCII locale).

They are gated to Windows and emit the dedicated `windows` proof tier
(`specs/audit/static_checks.proofs-windows.json`). The tier keeps host runs — where
these tests are skipped — from clobbering the CI-emitted proof file, since the pytest
proof plugin overwrites per (feature, tier) only for tiers it actually collected.

Run on Windows: python -m pytest dev/test_windows_native.py
"""

import os
import subprocess
import sys
import tempfile

import pytest

sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'scripts', 'audit'))

import static_checks
from static_checks import write_audit_cache, read_audit_cache

_STATIC_CHECKS_PY = os.path.join(
os.path.dirname(__file__), '..', 'scripts', 'audit', 'static_checks.py'
)
_REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))

pytestmark = pytest.mark.skipif(
sys.platform != "win32",
reason="Windows-native proofs run only on a windows-latest CI runner",
)


def _audit_entry(assessment, feature, proof_id, rule_id):
return {
"assessment": assessment,
"criterion": "matches rule intent",
"why": "test exercises the rule correctly",
"fix": "none",
"feature": feature,
"proof_id": proof_id,
"rule_id": rule_id,
"priority": "LOW",
"cached_at": "2026-04-01T00:00:00+00:00",
}


@pytest.mark.proof("static_checks", "PROOF-53", "RULE-29", tier="windows")
def test_real_msvcrt_lock_path():
"""On real Windows, fcntl is genuinely absent so write_audit_cache uses the native
msvcrt.locking path with no fake. The cache must round-trip and the adjacent
audit_cache.json.lock must be created."""
assert static_checks._HAS_FCNTL is False, (
"expected fcntl unavailable on Windows — the native msvcrt path would not be exercised"
)
with tempfile.TemporaryDirectory() as tmpdir:
write_audit_cache(tmpdir, {
"h1": _audit_entry("STRONG", "feat_a", "PROOF-1", "RULE-1"),
"h2": _audit_entry("WEAK", "feat_a", "PROOF-2", "RULE-2"),
})
after = read_audit_cache(tmpdir)
assert len(after) == 2, f"entries lost on native Windows lock path: {list(after)}"
assert "h1" in after and "h2" in after
lock_path = os.path.join(tmpdir, '.purlin', 'cache', 'audit_cache.json.lock')
assert os.path.exists(lock_path), "msvcrt lock file was not created adjacent to the cache"


@pytest.mark.proof("static_checks", "PROOF-54", "RULE-30", tier="windows")
def test_load_criteria_native_console():
"""Under the native Windows console codec (no PYTHONUTF8/LC_ALL/LANG overrides),
--load-criteria must read the tool's own non-ASCII audit_criteria.md and print it
without UnicodeDecodeError (read) or UnicodeEncodeError (stdout)."""
env = dict(os.environ)
for var in ('PYTHONUTF8', 'PYTHONIOENCODING', 'LC_ALL', 'LANG'):
env.pop(var, None)
with tempfile.TemporaryDirectory() as tmpdir:
r = subprocess.run(
[sys.executable, _STATIC_CHECKS_PY, '--load-criteria', '--project-root', tmpdir],
cwd=_REPO_ROOT, env=env, capture_output=True, text=True, encoding='utf-8',
)
assert r.returncode == 0, f"--load-criteria failed under native console codec: {r.stderr}"
assert any(ord(ch) > 127 for ch in r.stdout), "no non-ASCII glyph in --load-criteria output"
2 changes: 2 additions & 0 deletions docs/testing-workflow-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ public void ValidLoginReturns200()

NUnit `[Category]`/`[Property]` and MSTest `[TestProperty]` surface the same way. Run with `dotnet test --logger purlin -- RunConfiguration.CollectSourceInformation=true`. Setup is manual — see [proofs_format.md](../references/formats/proofs_format.md) for wiring the `Purlin.TestLogger` assembly.

`CollectSourceInformation=true` (plus full PDBs) is what populates each proof's `test_file`; without surfaced source info `dotnet test` leaves it empty. `purlin:audit` handles that case anyway — it resolves the source from the fully-qualified `test_name` via `static_checks.py --resolve-source`, so C# Pass-1/Pass-2 works even when `test_file` is blank.

### Shell (Bash)

```bash
Expand Down
Loading
Loading