diff --git a/CLAUDE.md b/CLAUDE.md index 7bb5b8b..e66660c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -16,8 +16,11 @@ agents/ scenario-generator.md # Step 3: Scenarios env-factory-generator.md # Step 4: Environment Factory implementation scenario-validator.md # Step 5: Scenario lifecycle validation + auth-login-validator.md # Step 5: agentic fallback for login probe test-case-generator.md # Step 6: E2E tests focused-test-case-generator.md +skills/agent-browser/SKILL.md # agent-browser CLI reference +skills/validate-auth-login/SKILL.md # headless browser login probe hooks/ hooks.json pipeline-kickoff.sh @@ -54,7 +57,8 @@ Validators are in `hooks/validators/`. | `validate_endpoint_implemented.py` | `*/autonoma/.endpoint-implemented` | handler path and factory integrity | | `validate_creation_file_immutable.py` | `*/autonoma/.endpoint-implemented` | accepted audit creation files were not rewritten unsafely | | `validate_factory_fidelity.py` | `*/autonoma/.endpoint-implemented` | semantic per-model factory fidelity | -| `validate_scenario_validation.py` | `*/autonoma/.scenario-validation.json` | Step 5 terminal-state contract | +| `validate_scenario_validation.py` | `*/autonoma/.scenario-validation.json` | Step 5 terminal-state contract (incl. `loginProbe`) | +| `login_probe.py` | invoked by `scenario-validator.md` between `up` and `down` | headless-browser login verification via `agent-browser` | | `validate_scenario_recipes.py` | `*/autonoma/scenario-recipes.json` | recipe schema | | `validate_test_index.py` | `*/autonoma/qa-tests/INDEX.md` | test totals and folder sums | | `validate_directory_structure.py` | `*/autonoma/qa-tests/INDEX.md` | test directory structure | @@ -76,5 +80,5 @@ pytest - Step 4 implements the Environment Factory and may edit target backend code. - Step 4 writes `autonoma/.endpoint-implemented` only after discover smoke and factory-integrity checks pass. -- Step 5 validates signed `discover` / `up` / `down` for every scenario and may fix handler bugs or reconcile `scenarios.md`. +- Step 5 validates signed `discover` / `up` / `down` for every scenario and may fix handler bugs or reconcile `scenarios.md`. Between `up` and `down` on the first auth-carrying scenario it also runs the login probe (`hooks/validators/login_probe.py`), which drives headless Chrome via [`agent-browser`](https://github.com/vercel-labs/agent-browser) to prove the returned credentials actually reach a logged-in page. Install via `brew install agent-browser` or `npm install -g agent-browser`; the probe skips cleanly if the binary is missing. - Step 6 is gated on `autonoma/.endpoint-validated`. diff --git a/agents/auth-login-validator.md b/agents/auth-login-validator.md new file mode 100644 index 0000000..bf4ba9c --- /dev/null +++ b/agents/auth-login-validator.md @@ -0,0 +1,47 @@ +--- +description: > + Login probe subagent invoked via `claude -p` from the scenario-validator + (Step 5). Uses the agent-browser CLI to verify that auth credentials + returned by the Environment Factory's `up` action actually reach an + authenticated page. Headless only. +tools: + - Bash + - Read +maxTurns: 20 +--- + +# Auth Login Validator + +You are a login probe. You have `agent-browser` available as a Bash CLI. +Read the `skills/agent-browser/SKILL.md` reference if you need the full +command surface. Your job: verify that the auth payload you receive actually +produces a logged-in browser session. + +## Rules + +- Always use `--session login-probe-