Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .claude/commands/CyClaw-Sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: >
at three realism tiers, triple-gated Grok/Claude fallback, API key
redaction, Phase 2 guardrails, due-diligence invariants, the full terminal
console REST surface, and a Python-3.12 runtime gate. Produces a dated
report and opens a draft PR.
report.
---

Invoke the `CyClaw-Sandbox` skill and run the full sandbox audit: clone,
Expand All @@ -15,10 +15,10 @@ See `.claude/skills/CyClaw-Sandbox/SKILL.md` for the full audit (its
Operator map + "Steps -- the full audit" section): realism ladder, config
validation, gate/graph standalone checks, the 5-query swarm test,
triple-gate + key redaction, due-diligence invariants, guardrails, the
full terminal- and harness-console REST + slash-command surface, the 3.12
runtime gate, the Guardrails invariant table, and the report/PR machinery.
full terminal-console REST + slash-command surface, the 3.12 runtime gate,
the Guardrails invariant table, and the sign-off report.

For a fast check against your current checkout with no clone/report/PR —
For a fast check against your current checkout with no clone or report —
use the skill's **Quick Mode** section instead (it absorbed the former separate
`/run-cyclaw` command; `/run` still ships as the thin Quick-Mode entry point):

Expand All @@ -33,5 +33,5 @@ bash .claude/skills/CyClaw-Sandbox/smoke.sh
- Run the highest Ollama realism tier the environment supports (Tier 0
stubs / Tier 1 mock-over-HTTP / Tier 2 real daemon) and record which tier
was used — never claim Tier 2 realism from a Tier 0/1 run.
- Draft PR only; a human decides when to merge.
- This skill opens no PR (CLAUDE.md §9); it produces a dated sign-off report.
- `status: degraded` without live Ollama is expected and normal.
213 changes: 96 additions & 117 deletions .claude/skills/CyClaw-Sandbox/SKILL.md

Large diffs are not rendered by default.

38 changes: 19 additions & 19 deletions .claude/skills/CyClaw-Sandbox/run_full_verification.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
_REPORT_NAMES = ("verification_report.json", "query_results.json")

# Which of the 3-tier Ollama realism ladder this run actually exercised (see
# SKILL.md's "3-tier realism" section). Set once in main()
# SKILL.md's "Operator map" realism tiers). Set once in main()
# by _probe_ollama_tier() before any phase that talks to the local-LLM base_url
# runs, then stamped into both report files this script writes.
OLLAMA_TIER: int | None = None
Expand Down Expand Up @@ -955,34 +955,34 @@ def phase_key_redaction() -> PhaseResult:
banner("Phase 6: API Key Redaction (Grok + Claude Parity)")
phase = PhaseResult("Key Redaction")

gate_src = Path("gate.py").read_text()
# Redaction lives in utils/logger.py (redact_sensitive: every os.environ
# value plus _MINIMUM_SECRET_PATTERNS) and reaches HTTP errors through
# utils/public_errors.public_error; gate.py no longer carries its own
# tuple (the old gate.py greps failed on every run once it moved).
logger_src = Path("utils/logger.py").read_text()

# Check ANTHROPIC_API_KEY in env-var redaction tuple
has_anthropic_env = "ANTHROPIC_API_KEY" in gate_src
log(f" [{'PASS' if has_anthropic_env else 'FAIL'}] ANTHROPIC_API_KEY in env-var redaction")
phase.checks.append(Check("anthropic_key_env_redaction", has_anthropic_env))
has_env_scan = "os.environ.items()" in logger_src and "def redact_sensitive" in logger_src
log(f" [{'PASS' if has_env_scan else 'FAIL'}] redact_sensitive scans every os.environ value (Grok + Anthropic parity)")
phase.checks.append(Check("env_value_redaction", has_env_scan))
Comment on lines +964 to +966

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Test the secret-name filter instead of any env iteration

This check and its PASS message claim that every environment value is redacted, but redact_sensitive only replaces values whose names match SECRET_POLICY and whose values are at least eight characters. Merely finding os.environ.items() therefore passes even if provider names stop being classified, and arbitrary environment values demonstrably remain unchanged; exercise the classification/redaction behavior directly or narrow the advertised guarantee.

AGENTS.md reference: AGENTS.md:L33-L33

Useful? React with 👍 / 👎.


# Check sk-ant-* pattern in _SECRET_PATTERNS
has_sk_ant_pattern = "sk-ant-" in gate_src
log(f" [{'PASS' if has_sk_ant_pattern else 'FAIL'}] sk-ant-* pattern in _SECRET_PATTERNS")
phase.checks.append(Check("sk_ant_pattern_in_gate", has_sk_ant_pattern))
has_sk_ant_pattern = "sk-ant-" in logger_src
log(f" [{'PASS' if has_sk_ant_pattern else 'FAIL'}] sk-ant-* pattern in utils/logger.py _MINIMUM_SECRET_PATTERNS")
phase.checks.append(Check("sk_ant_pattern_in_logger", has_sk_ant_pattern))

# Check GROK_API_KEY is still there (regression check)
has_grok_env = "GROK_API_KEY" in gate_src
log(f" [{'PASS' if has_grok_env else 'FAIL'}] GROK_API_KEY in env-var redaction (regression)")
phase.checks.append(Check("grok_key_env_redaction", has_grok_env))
has_public_error = "redact_sensitive" in Path("utils/public_errors.py").read_text()
log(f" [{'PASS' if has_public_error else 'FAIL'}] public_error routes HTTP error text through redact_sensitive")
phase.checks.append(Check("public_error_uses_redaction", has_public_error))

# Test redaction of an Anthropic key in error messages
try:
# Import and test _sanitize_error if available
from gate import _sanitize_error
from utils.public_errors import public_error
test_msg = "Error: API call failed with key sk-ant-api03-testkey123456789"
sanitized = _sanitize_error(test_msg)
redacted = "sk-ant" not in sanitized or "[REDACTED]" in sanitized
sanitized = public_error(test_msg, {})
redacted = "sk-ant-api03" not in sanitized
log(f" [{'PASS' if redacted else 'FAIL'}] Anthropic key sk-ant-api03-... redacted in errors")
phase.checks.append(Check("anthropic_key_sanitized", redacted))
except (ImportError, AttributeError) as e:
log(f" {Y}SKIP{N}] _sanitize_error not importable: {e}")
log(f" {Y}SKIP{N}] public_error not importable: {e}")
phase.checks.append(Check("anthropic_key_sanitized", False, str(e)))

# Verify in config.yaml -- policy.privacy, not logging.audit (Phase 1's
Expand Down
33 changes: 18 additions & 15 deletions .claude/skills/CyClaw-Sandbox/test-specifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,32 +197,35 @@ When `online_provider` is not set, `user_gate_router` defaults to "grok"

### Background
Commit `78515b0` added Anthropic key redaction with the same rigor as Grok:
- `ANTHROPIC_API_KEY` added to env-var redaction tuple in `gate.py`
- `sk-ant-[A-Za-z0-9_\-]{20,}` pattern added to `_SECRET_PATTERNS`
- `sk-ant-*` pattern added to `config.yaml` `audit.redact_secrets_like`
- every `os.environ` value is redacted by `utils/logger.py::redact_sensitive`
(so `ANTHROPIC_API_KEY` and `GROK_API_KEY` get the same treatment)
- `sk-ant-[A-Za-z0-9_-]{20,}` pattern in `utils/logger.py` `_MINIMUM_SECRET_PATTERNS`
- `sk-ant-*` pattern in `config.yaml` `policy.privacy.redact_secrets_like`
- HTTP error text reaches the client only through `utils/public_errors.public_error`
(the redaction tuple that once lived in `gate.py` moved; R1/R2 grep the new home)

### Test Cases

#### R1: Env-Var Redaction
```python
def test_anthropic_key_env_redaction():
gate_src = Path("gate.py").read_text()
assert "ANTHROPIC_API_KEY" in gate_src
assert "GROK_API_KEY" in gate_src
def test_env_value_redaction():
logger_src = Path("utils/logger.py").read_text()
assert "def redact_sensitive" in logger_src
assert "os.environ.items()" in logger_src
```

#### R2: Pattern Redaction in gate.py
#### R2: Pattern Redaction in utils/logger.py
```python
def test_sk_ant_pattern_in_secret_patterns():
gate_src = Path("gate.py").read_text()
assert "sk-ant-" in gate_src
logger_src = Path("utils/logger.py").read_text()
assert "sk-ant-" in logger_src
```

#### R3: Pattern Redaction in config.yaml
```python
def test_sk_ant_in_audit_config():
def test_sk_ant_in_privacy_config():
cfg = yaml.safe_load(Path("config.yaml").read_text())
patterns = cfg["logging"]["audit"]["redact_secrets_like"]
patterns = cfg["policy"]["privacy"]["redact_secrets_like"]
assert any("sk-ant" in str(p) for p in patterns)
```

Expand Down Expand Up @@ -304,7 +307,7 @@ From `tests/test_due_diligence_invariants.py` (14 test classes):
- `TestShippedAgenticConfigContract` (`tests/test_agentic_config.py`) already
pins the agentic block against the real file; the core request path had no
equivalent, and the gap is not hypothetical -- the operator commit of
2026-08-07 flipped `app.mode`, `app.debug`, and BOTH `models.<provider>.enabled`
2026-08-07 flipped `app.mode`, the since-removed `app.debug`, and BOTH `models.<provider>.enabled`
in the shipped file and no test broke, because no test read those keys from it
- Matters specifically for I3: per `INVARIANTS.md`, two of the triple gate's
three conditions (`mode == "hybrid"`, `<provider>.enabled`) are NOT enforced
Expand Down Expand Up @@ -407,8 +410,8 @@ policy:
send_local_context_to_claude: false
grok_max_prompt_chars: 8000
claude_max_prompt_chars: 8000
logging:
audit:
policy:
privacy:
redact_secrets_like:
Comment on lines +413 to 415

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep privacy under the existing policy mapping

This example now declares the top-level policy key twice. PyYAML silently keeps only the second mapping, so parsing or copying the documented hybrid configuration drops the entire fallback block immediately above it; strict YAML parsers may reject it outright. Nest privacy alongside fallback under the first policy mapping so the test specification represents the actual config.yaml structure.

AGENTS.md reference: AGENTS.md:L5-L7

Useful? React with 👍 / 👎.

- "sk-[a-zA-Z0-9]{20,}" # OpenAI-style
- "sk-ant-[a-zA-Z0-9_\\-]{20,}" # Anthropic (Claude)
Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/CyClaw-Sandbox/verify.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
# CyClaw sandbox runtime verification — proves the entire main branch runs
# under a clean Python 3.12 runtime. Nine labeled stages, fail-fast on required
# under a clean Python 3.12 runtime. Seven labeled stages, fail-fast on required
# ones. Labels are historical and NOT sequential in source order (1, 2, 3, 5,
# 8, 4, 7, 9, 6) — do not renumber; CI logs cite them. See SKILL.md
# 4, 7, 6) — do not renumber; CI logs cite them. See SKILL.md
# "Operator map" for what each stage proves vs does not prove.
# Run from the repo root: bash .claude/skills/CyClaw-Sandbox/verify.sh
set -uo pipefail
Expand Down
Loading
Loading