Conversation
Comprehensive supply-chain dependency auditing skill that performs 12-dimension analysis across 8 ecosystems (Python, Node, Go, Rust, .NET, containers, GitHub Actions, credentials). Key features: - Dependency manifest parsing (requirements.txt, package.json, go.mod, Cargo.toml, etc.) - Pinning verification and mutable-reference detection - Vulnerability scanning via external tools (grype, npm audit, pip-audit, cargo-audit) - SLSA compliance assessment (Levels 1-4) - GitHub Actions SHA-pinning enforcement - Container image tag-pinning and base-image freshness checks - Credential/secret leak detection in config files - License compliance scanning - Structured findings with severity ratings and remediation guidance Closes #3440 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repo Guardian - PassedAll 59 changed files in this PR were reviewed. No ephemeral content was found. Files Analyzed
Analysis Notes
|
github-actions
bot
added
enhancement
PR Triage ReportRisk: Medium | Priority: Medium | Draft — needs deep review Assessment
Findings
Recommendation🔍 Needs deep review before converting to ready. Good test coverage and isolated scope, but the manual recovery from a hollow-success should be verified: confirm all checkers are implemented (not stubs), integration tests actually exercise the skill, and the skill entry point works end-to-end. Consider a
|
- Replace assert with TypeError for runtime validation (SEC-001) - Fix hardcoded INFO-001 duplicate IDs (REL-001) - Improve symlink error handling with debug logging (SEC-003) - Add public skip_reasons property to EcosystemScope (REL-002) - Narrow exception catch in ToolClient to SubprocessError (REL-003) - Extract shared checker utils into _utils.py (DEAD-001) - Refactor run_audit() into _run_all_checkers/_build_advisory_messages (STRUCT-001) - Refactor render() into _render_summary_table/_render_findings_section/_render_slsa_section (STRUCT-002) - Add Dim 6 credential hygiene unit tests (TEST-002) - Add Dim 12 Docker build chain unit tests (TEST-001) - Add debug logging for unreadable workflow files in detector (SILENT-001) - Remove dead _SEVERITY_ORDER from actions.py (DEAD-002) - Remove unused _FROM_PATTERN and _USER_INSTRUCTION_PATTERN from containers.py (DEAD-003) - Remove break after first Azure cred to report all instances (SILENT-002) - Mark dependabot.yml and CODEOWNERS as planned in howto doc (DOC-001) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repo Guardian - Action RequiredAll 61 changed files in PR #3481 were reviewed. One violation was found. Violation: Hardcoded environment-specific pathFile: Problem: The agents:
- name: "verifier"
type: "system"
config:
shell: "bash"
cwd: "/home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th"This path is:
This is a classic example of a test script with hardcoded environment-specific values that cannot be reused outside the specific developer workstation where it was authored. Where it should go instead: The All Other Files: Passed ✅
To override, add a PR comment containing
|
- Use defensive dict.get() in _assign_ids (C2-01) - Remove redundant per-checker _assign_ids calls (C2-02/03) - Fix circuit breaker over-triggering from retries (C2-04) - Sort XPIA workflow glob for determinism (C2-06) - Add assertion guards in accepted-risk tests (C2-08) - Tighten XPIA SYSTEM pattern to avoid false positives (C2-09) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repo Guardian - Action RequiredAll 61 changed files in PR #3481 were reviewed. 1 violation was found — the same one flagged in the previous run (workflow run 23467893328) that has not yet been resolved. Violation: Hardcoded machine-specific and branch-specific pathFile: Problem: The agents:
- name: "verifier"
type: "system"
config:
shell: "bash"
cwd: "/home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th"This path is:
A test fixture with a hardcoded, non-portable path will fail for every contributor and in every CI environment except the exact machine and worktree where it was authored. Fix: Replace the hardcoded path with a relative path or an environment variable. For example: cwd: "."or cwd: "$GITHUB_WORKSPACE"All other paths in this file already use relative paths (e.g. All Other Files: Passed ✅
To override, add a PR comment containing
|
The XPIA detector previously threw XpiaEscalationError on pattern match, aborting the entire audit. This made the tool unusable on real repos with legitimate LLM workflow files containing <system> tags. - _check_xpia returns advisory strings instead of raising - XPIA matches appear in the report advisory section - Added live audit gadugi tests running against the actual repo - Verified: 195 findings, 17 XPIA advisories on amplihack repo Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repo Guardian - Action RequiredAll 61 changed files in PR #3481 were reviewed. 1 violation found — previously flagged and not yet resolved. Violation: Hardcoded machine-specific and branch-specific paths (pervasive)File: This file contains hardcoded absolute paths specific to a single developer's machine and to the PR's worktree branch. The violation is more extensive than previous runs indicated — it appears in the agent 1. Agent config agents:
- name: "verifier"
type: "system"
config:
cwd: "/home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th"2. Step command: "cd /home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th && ..."3. Hardcoded paths in # chr(47)+'home'+chr(47)+'azureuser'+chr(47)+'src'+chr(47)+'amplihack'
# evaluates to: /home/azureuser/src/amplihack
run_audit(chr(47)+'home'+chr(47)+'azureuser'+chr(47)+'src'+chr(47)+'amplihack')All of these paths:
Why this matters: This test cannot be run by any contributor or in CI. It is a snapshot of the development environment at the time it was authored, not a portable, reusable test fixture. Recommended fix: Replace all hardcoded absolute paths with the agent agents:
- name: "verifier"
config:
cwd: "." # repo root — portable
steps:
- name: "Live audit run"
params:
command: 'PYTHONPATH=.claude/skills/supply-chain-audit python3 -c "from supply_chain_audit.audit import run_audit; r = run_audit(\".\"); ..."'All other test steps already use relative paths ( All Other Files: Passed ✅
To override, add a PR comment containing
|
XPIA-matching files get full audit coverage AND are flagged for manual review. Checkers extract structured data (action refs, permissions), not raw content, so prompt injection patterns don't leak into the report. Verified on amplihack repo: bot-detection.lock.yml gets 3 real security findings (secret exposure) plus XPIA advisory for human review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repo Guardian - Action RequiredAll 61 changed files in PR #3481 were reviewed. 1 violation found — flagged in runs 23467893328, 23468344541, and 23469983330 and still not resolved as of workflow run 23470842882. Violation: Hardcoded machine-specific and branch-specific paths (unresolved)File: The file contains hardcoded absolute paths specific to a single developer's machine and worktree. The paths appear in three places: 1. Agent config agents:
- name: "verifier"
config:
cwd: "/home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th"2. Step command: "cd /home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th && ..."3. Obfuscated path in # chr(47)+'home'+chr(47)+'azureuser'+chr(47)+'src'+chr(47)+'amplihack'
# evaluates to: /home/azureuser/src/amplihack
run_audit(chr(47)+'home'+chr(47)+'azureuser'+chr(47)+'src'+chr(47)+'amplihack')All three patterns are machine-specific ( Recommended fix: Set # Agent config
cwd: "." # repo root — portable
# Live audit steps
command: 'PYTHONPATH=.claude/skills/supply-chain-audit python3 -c "from supply_chain_audit.audit import run_audit; r = run_audit(\".\"); ..."'All Other Files: Passed ✅
To override, add a PR comment containing
|
…, test coverage
Security fixes:
- Add _sanitize_for_display() defense-in-depth layer that strips XPIA patterns
from all rendered finding fields (schema.py, report.py)
- Redact expected_value when contains_secret=True (was only current_value)
- Fix SYSTEM: regex false positives on BUILD_SYSTEM:, CI_SYSTEM: by adding
underscore and digit to negative lookbehind
Reliability fixes:
- Fix Dim 6 detection: only trigger when ${{ secrets.* }} found (was dead loop)
- Add circuit breaker checks to run_or_raise() (was bypassing breaker)
- Surface SLSA file-read errors as advisories (was silent except pass)
- Add deprecation docstring to XpiaEscalationError (never raised, kept for compat)
- Remove dead _assign_ids() from checkers/_utils.py
Test coverage:
- Add TestDim4CachePoisoning (3 tests for cache key validation)
- Add XPIA tests: findings-still-produced, multi-pattern, scope-is-workflow-only,
sanitization, SYSTEM: false-positive prevention
- Add Dim 6 scope test: workflow without secrets does not trigger Dim 6
299 tests pass. Live audit: 195 findings, 17 XPIA advisories, 0 false positives,
0 XPIA patterns in rendered output.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Repo Guardian - Action RequiredAll 61 changed files in PR #3481 were reviewed. 1 violation found — flagged in prior runs (23467893328, 23468344541, 23469983330, 23470842882) and still not resolved. Violation: Hardcoded machine-specific and branch-specific pathsFile: The file contains hardcoded absolute paths tied to a single developer's machine and to this PR's worktree. The violation appears in three places: 1. Agent config agents:
- name: "verifier"
config:
cwd: "/home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th"2. Step command: "cd /home/azureuser/src/amplihack/worktrees/feat/issue--review-github-issue-3440-gh-issue-view-3440-and-th && ..."3. Obfuscated path in # chr(47)+'home'+chr(47)+'azureuser'+chr(47)+'src'+chr(47)+'amplihack'
# evaluates to: /home/azureuser/src/amplihack
run_audit(chr(47)+'home'+chr(47)+'azureuser'+chr(47)+'src'+chr(47)+'amplihack')These paths are machine-specific ( Recommended fix: Set # Agent config — portable
cwd: "."
# Live audit steps — use relative path
command: 'PYTHONPATH=.claude/skills/supply-chain-audit python3 -c "from supply_chain_audit.audit import run_audit; r = run_audit(\".\"); ..."'All Other Files: Passed ✅
To override, add a PR comment containing
|
- Add 2 missing patterns to _XPIA_SANITIZE_PATTERNS (</sys> truncated tag, bare SYSTEM: directive) to match audit.py's detection list - Apply _sanitize_for_display() to rationale and fix_url fields in both Finding.render() and report.py _render_findings_section() All 8 XPIA patterns now covered in both detection and sanitization. All rendered finding fields (current_value, expected_value, rationale, fix_url) are now sanitized. 299 tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 participant
Summary
Implements #3440 — adds a supply-chain-audit skill that scans repositories for supply chain security risks across 12 dimensions.
Features
Documentation
SKILL.mdandREADME.md— skill reference documentationreference/directory — 11 detailed dimension reference docsMerge Criteria Evidence
✅ CI Green
All GitHub Actions checks pass (Validate Code ✅, GitGuardian ✅, test-plugin ✅, Documentation Policy ✅, etc.).
✅ Gadugi Outside-In Testing
Scenario:
tests/gadugi/supply-chain-audit-skill.yaml— 28 steps, all passStructural checks (24 steps):
Live audit checks (4 steps, run against amplihack repo):
LIVE_AUDIT_PASS: 195 findings detectedMULTI_DIM_PASS: Findings span dimensions [1, 2, 3, 5, 7, 8, 12]XPIA_ADVISORY_PASS: 17 XPIA advisories for legitimate LLM workflow filesSCOPED_AUDIT_PASS:.github/scoped audit returns dimensions [1, 2, 3]Validated:
gadugi-test validate -s supply-chain-audit-skill -d tests/gadugiExecuted:
gadugi-test run -s supply-chain-audit-skill -d tests/gadugi→ ✓ Passed: 1, ✗ Failed: 0✅ Live Repository Testing
Ran
run_audit()against the amplihack repo itself:</system>template tags)contains_secret=True→ render as<REDACTED>✅ Quality Audit (3 cycles converged)
_utils.py, refactoredrun_audit()/render(), added Dim 6/12 tests, removed dead code, fixed silent fallbacksKey findings addressed:
assertreplaced withTypeErrorfor subprocess argument validationINFO-001finding IDs_utils.pyrun_audit()and 184-linerender()into focused helpers✅ No Unrelated Changes
All changes scoped to
.claude/skills/supply-chain-audit/,tests/gadugi/, anddocs/howto/.