🔥 PhoenixQA

Self-healing test automation framework for fragile frontends. When a selector breaks, PhoenixQA doesn't crash — it heals.

---

🧠 What is this?

Frontend tests break constantly — not because the feature is broken, but because the page underneath it changed. A class was renamed. A data-testid rotated. A wrapper <div> appeared around a button after a refactor. A component moved into a Shadow DOM boundary.

PhoenixQA intercepts those failures, feeds the context to an LLM, and either:

• proposes a fix for human review (Safe Mode — live)
• applies the fix automatically within a confidence/budget policy and continues (Autonomous Mode — live)

Every decision — human-reviewed or autonomous — is logged today, including which provider made the call, how many tokens it cost, and how long it took. Once Healing History (Sprint 7) lands, that log becomes the basis for a self-training loop that improves future healing (Sprint 8) — not yet built, but the logging that feeds it already is.

Scope: where this starts, and where it's going

"Test fails even though the app is fine" has more than one root cause. PhoenixQA classifies failures into four types, but builds them in phases rather than all at once:

The original four-way FailureType split (selector_not_found / detached_from_dom / not_visible / timeout_waiting) has been fully replaced internally by FailureCategory (LOCATOR_RESOLUTION / ACTIONABILITY / dormant REFERENCE) + ActionabilityReason (5 values) — see below for why. Mapped onto the original four labels for orientation:

Failure type	Status
selector_not_found → FailureCategory.LOCATOR_RESOLUTION	✅ Live (Sprint 2-5), SelectorReplacement action, Safe + Autonomous Mode both verified end-to-end
detached_from_dom → dormant FailureCategory.REFERENCE	🔬 Investigated (Sprint 6A) — no reproduction found for Locator-based automation across four escalating attempts; deprioritized based on current evidence, not proven impossible — see docs/gaps.md Gap #4
not_visible / timeout_waiting → FailureCategory.ACTIONABILITY (5 ActionabilityReason values)	🚧 In progress — RECEIVES_EVENTS and VISIBLE fully implemented, unit-tested, AND live-verified (both directions of the policy guardrail — see below); ENABLED fully implemented, unit-tested, AND live-verified 2/2 (Sprint 8); EDITABLE not started; STABLE blocked on a deterministic Chaos App mechanism. Healer EXECUTES WAIT_AND_RETRY/NO_SAFE_RECOVERY for all three implemented reasons — VISIBLE, RECEIVES_EVENTS, and ENABLED — all live-verified. RECEIVES_EVENTS' DISMISS_BLOCKER remains proposal-only, still rejected (no DOM-validated guardrail exists for it yet).

Why phase it: better to prove the full pipeline (collect → analyze → heal → validate) end-to-end on one well-understood failure type first, then extend to the others with real lessons learned — rather than build four shallow strategies at once. See LEARNINGS.md for the full reasoning, or docs/gaps.md for a quick-scan status table of every open architectural question.

Note on detached_from_dom: the original working assumption (based on direct Selenium/Salesforce Lightning experience) was that framework re-render-mid-action failures would be the dominant real-world case for PhoenixQA to handle. Sprint 6A tested that assumption directly — a real reproduction mechanism (componentRemount.jsx) was built and run across four escalating configurations, ending with a deterministic, zero-timing-luck trigger. The consistent finding: no observable DETACHED_FROM_DOM failure, at any setting, for this project's specific interaction pattern. Research into Playwright's own documentation and issue tracker offers a likely explanation, not just a coincidence — Locator-based actions (all BasePage ever uses; never ElementHandle) are documented to retry automatically on mid-action detachment, which would plausibly absorb much of the failure class Selenium-style automation is vulnerable to. This is deliberately framed as evidence supporting a deprioritization, not proof that the failure type doesn't exist — see LEARNINGS.md "Sprint 6A conclusion" for the precise scope of the claim, the full hypothesis → experiment → finding → decision trail, sources, and what it means for the project's remaining scope.

Note on not_visible/timeout_waiting: before picking one of these as the redirected target, a set of small diagnostic experiments checked whether Playwright's own error messages can actually tell them apart — timeout_waiting's definition ("never reached an actionable state") is close to a superset of not_visible's. They can't, in the way the original four-way FailureType split assumed. Real production logs plus targeted live captures show Playwright's model is genuinely two-stage: first "did the locator resolve at all" (a bare waiting for locator(...) message, identical for click() and fill(), means no), and only if it resolved, one of five distinct actionability reasons (visible, enabled, editable, stable, receives events). A flat choice between not_visible and timeout_waiting as separate top-level types would have inherited the same ambiguity the classifier already had for selector_not_found before Sprint 4's fix.

The replacement, now implemented and live: FailureCategory (LOCATOR_RESOLUTION / ACTIONABILITY / a dormant REFERENCE for the detached_from_dom family) plus ActionabilityReason for the five concrete reasons above. LOCATOR_RESOLUTION, deliberately not SELECTOR — an unresolved locator has more than one plausible real cause (genuine selector drift, an element that's conditionally not yet mounted, the app being in an unexpected state), which Playwright's own message can't distinguish; naming the category after "selector" would have presupposed the diagnosis the evidence argued against. parse_playwright_call_log() (the classifier), the ContextCollector/LocatorResolutionCollector/ActionabilityCollector router split, the HealingAction hierarchy, and two of the five ActionabilityReason provider paths (RECEIVES_EVENTS, VISIBLE) are all implemented, unit-verified, and live-verified against a real Chaos App + Ollama. ENABLED is implemented AND live-verified too (2/2); EDITABLE is not started; STABLE is blocked on a deterministic Chaos App mechanism. See LEARNINGS.md "Sprint 6B (decision)" and docs/gaps.md Gap #5/#12 for the full model, and Gap #13/#14 for the two limits this decision explicitly does not resolve (the model's dependence on Playwright's unversioned diagnostic text, and LocatorResolutionCollector's continued inability to tell apart why a locator never resolved).

LLM proposes, PhoenixQA validates (Sprint 6B): live testing on RECEIVES_EVENTS found llama3.2 deterministically proposing wait_and_retry for a blocker it had itself correctly described, in its own reasoning, as persistent with no dismiss affordance — confidence unchanged. Prompt engineering alone did not fix this (see LEARNINGS.md Sprint 6B). The response is phoenix/healing/actionability_policy.py: a deterministic guardrail that validates a proposed wait_and_retry against collector_metadata (structured DOM facts gathered by ActionabilityCollector), never against the model's freeform reasoning text, and corrects the strategy when the evidence doesn't support it. Live-verified in both directions — correctly blocking RECEIVES_EVENTS's unsupported wait_and_retry every time it was tried, and correctly passing through VISIBLE's two ground-truth cases (permanent → model self-selects no_safe_recovery, transient → model self-selects wait_and_retry) without needing to intervene at all. See Gap #11 in docs/gaps.md.

Important framing shift for the remaining failure types (Gap #12): for selector_not_found, the selector itself is what's broken, and the fix is a replacement selector. For the remaining failure types the selector may be perfectly correct — what actually failed is the in-flight ACTION (e.g. a click against an element that isn't yet actionable, or never becomes actionable in time). PhoenixQA treats selector_not_found as the special case where "recover the action" happens to mean "propose a new selector" — the other failure types recover the action a different way (retry with a wait, dismiss an overlay, scroll into view, etc.). See docs/gaps.md Gap #12 and LEARNINGS.md Sprint 6.

How this project approaches quality as a whole — not just unit tests on the framework's own code, but a layered strategy covering integration, end-to-end behavior against a real LLM, regression benchmarking of healing effectiveness, and non-functional resilience to malformed model output — is laid out in docs/testing-strategy.md.

The open empirical questions driving the project's direction — does an LLM meaningfully outperform a cheap heuristic, which failure types actually justify LLM-based healing, is model confidence a reliable proxy for correctness, and others — are collected in one place in docs/research_hypotheses.md, separate from docs/gaps.md's architectural TODOs: a gap is something to build, a hypothesis is something to find out.

---

🏗️ Architecture

Test Failure
    │
    ▼
Failure Classifier        ← parse_playwright_call_log() → ClassifiedFailure
    │                          FailureCategory (locator_resolution, actionability, reference)
    │                          + ActionabilityReason (5 values) — live, replaces the old flat
    │                          FailureType enum entirely (no compatibility adapter kept)
    │
    ▼
Context Collector          ← router (context_collector.py) over one collector per FailureCategory:
    │                          locator_resolution_collector.py (DOM snapshot, weighted scoring) — live
    │                          actionability_collector.py   (RECEIVES_EVENTS, VISIBLE — live;
    │                                                         ENABLED/EDITABLE not started; STABLE blocked)
    │                          reference_collector.py         (dormant, no active plan)
    ▼
LLM Analyzer               ← Ollama (local) or Anthropic API [stub, NotImplementedError] →
    │                          structured HealingAction:
    │                          SelectorReplacement    — live, Safe + Autonomous Mode verified
    │                          ActionabilityStrategy   — live for RECEIVES_EVENTS/VISIBLE, guarded by
    │                                                     actionability_policy.py (deterministic
    │                                                     evidence check, not the model's own text) —
    │                                                     PROPOSED and validated, but Healer still
    │                                                     rejects it outright; no execution yet
    │                          RetryStrategy            (reference, dormant) — declared only
    │
    ├──► Safe Mode        ← Human reviews full context, accepts/rejects → Ground Truth
    │
    └──► Autonomous Mode  ← Confidence gate + budget check, auto-applies fix, retries
              │
              ▼
        Healing History   ← SQLite log of all decisions (Sprint 7)
              │
              ▼
        Self-Training     ← Few-shot context for better future repairs (Sprint 8)

Note: PhoenixQA recovers the ABILITY to perform an action after a
failure — it does not judge whether the resulting behavior was
business-correct (e.g. "did login actually succeed"). That judgment
stays with the test's own assertions, same as it always has. See
docs/gaps.md Gap #11 for why this boundary is deliberate.

Autonomous Mode raises one of three distinct exception types depending on why it didn't heal — HealingRejectedError (bad/low-confidence proposal), HealingLimitExceededError (budget exhausted), HealingFailedError (provider/API crashed) — so a CI failure report says exactly what happened, not just "healing didn't work."

Current implementation status: the Context Collector → LLM Analyzer → HealingAction split above is implemented, not just planned — ContextCollector is a router over per-category collectors, prompts live under phoenix/ai/prompts/ (plus the not-yet-migrated phoenix/ai/prompt_templates.py for the selector path — see docs/known-limitations.md), and providers return HealingAction subtypes. Healer executes SelectorReplacement (live-verified) and WAIT_AND_RETRY/NO_SAFE_RECOVERY for VISIBLE, RECEIVES_EVENTS, and ENABLED — all three live-verified. What's NOT yet done: ActionabilityReason.EDITABLE/STABLE, RECEIVES_EVENTS' DISMISS_BLOCKER execution, and AnthropicProvider (both methods still raise NotImplementedError — ollama is the only working provider today, despite the badge/table above listing both). See docs/known-limitations.md for the precise current-vs-planned boundary.

---

🧪 Chaos Levels — a benchmark, not just randomness

Chaos App isn't randomized weirdness — each level isolates one variable and answers a specific research question. This is closer to a controlled experiment than a typical "Playwright + sample app" portfolio repo.

Level	Mechanisms (cumulative)	Research question
LOW	selector rotation	Does the test survive a selector rename?
MEDIUM	+ DOM structure mutation	Does the test survive a UI refactor?
HIGH	+ async delay	Does the test survive a refactor + timing issues?

Shadow DOM is a separate, independent flag (SHADOW_DOM_ENABLED), not a 4th level — it's a different kind of difficulty (structural DOM access), combinable with any level above (e.g. HIGH + Shadow DOM tests refactor + timing + structural access at once).

Mechanisms ranked by real-world realism (most enterprise frontends break this way most often):

Mechanism	Realism	Why
DOM Mutation	10/10	UI library upgrades, wrapper changes, component migrations
Selector Rotation	9/10	Classic — renamed class/id/data-testid
Async Delay	8/10	Lazy loading, animations, network-dependent rendering
Shadow DOM	5/10	Real, but narrower — mostly Web Components / LWC-style platforms

A 5th, independent mechanism, component remount / detach-mid-action (componentRemount.jsx, COMPONENT_REMOUNT_ENABLED), was built in Sprint 6A specifically to give detached_from_dom something real to classify against. It's implemented and verified live, but — see the Scope section above — did not reproduce the target failure against Locator-based interactions even under a deterministic, zero-timing-randomness trigger. It remains in the codebase as a verified research tool, not as an active target for the next healing strategy.

Two more independent mechanisms were built in Sprint 6B, specifically to give ActionabilityReason.RECEIVES_EVENTS and .VISIBLE real, deterministic ground truth to classify against — both are live and verified:

• pointerEventsOverlay.jsx (VITE_POINTER_EVENTS_OVERLAY_MODE=off|permanent|transient, VITE_POINTER_EVENTS_OVERLAY_MS) — a transparent, full-viewport overlay that intercepts every click, simulating a real cookie banner / modal / sticky header blocking a click. permanent never removes itself (ground truth: no_safe_recovery); transient unmounts after the configured delay (ground truth: wait_and_retry) — added (Sprint 8) mirroring visibilityDelay.jsx's two-mode split, once Healer's execution widened to cover RECEIVES_EVENTS too. RECEIVES_EVENTS' evidence is a declared CSS capability checked once, not VISIBLE's observed-change-across-a-narrow-window — but live testing still found a real lower bound: the delay is measured from component mount, not from when click() is first attempted (login() fills two fields first), so too short a value lets the overlay vanish before Healer is ever invoked at all. Verified working value: 15000 — see docs/known-limitations.md.
• disabledState.js (VITE_DISABLED_STATE_MODE=off|permanent|transient, VITE_DISABLED_STATE_MS) — the login button carries a plain disabled attribute, simulating ActionabilityReason.ENABLED (a "Submit" button disabled until validation passes, or while an async save is in flight). permanent never enables it (ground truth: no_safe_recovery); transient enables it after the configured delay (ground truth: wait_and_retry) — the third ActionabilityReason, built directly with both modes from the start (Sprint 8) now that the PERMANENT/TRANSIENT pattern is established. Evidence model mirrors VISIBLE (an observed state change across two snapshots), not RECEIVES_EVENTS (a declared CSS capability) — see disabledState.js's own docstring.
• visibilityDelay.jsx (VITE_VISIBILITY_DELAY_MODE=off|permanent|transient, VITE_VISIBILITY_DELAY_MS) — hides the password input via visibility: hidden. permanent never reveals it (ground truth: no_safe_recovery); transient reveals it after the configured delay (ground truth: wait_and_retry) — the first mechanism built to test both directions of the policy guardrail live. Important: VITE_VISIBILITY_DELAY_MS must land INSIDE ActionabilityCollector's fixed observation window ([T, T+1200] where T ≈ Playwright's own fill() timeout, 30000ms by default) — not merely exceed 30000ms, and not too far past it either. 30500 is intermittent (system-jitter-dependent), 32000 misses deterministically (past the window entirely). Working value: 30800. See docs/known-limitations.md and LEARNINGS.md's "[Verification] Margin correction" entry for the full reasoning.

Controlling the chaos level:

# chaos_app/.env
VITE_CHAOS_LEVEL=HIGH                      # LOW | MEDIUM | HIGH
VITE_SHADOW_DOM_ENABLED=true               # true | false — independent of level
VITE_COMPONENT_REMOUNT_ENABLED=false       # true | false — independent of level, see note above
VITE_POINTER_EVENTS_OVERLAY_MODE=off       # off | permanent | transient — independent of level, ActionabilityReason.RECEIVES_EVENTS
VITE_VISIBILITY_DELAY_MODE=off             # off | permanent | transient — independent of level, ActionabilityReason.VISIBLE

Edit chaos_app/.env, then restart npm run dev. The "Active chaos config" panel at the top of the running app confirms which mechanisms are live — no guessing required.

Mechanism overrides (development/verification only, never for the official benchmark): each core mechanism (selector_rotation, dom_mutation, async_delay) can be forced on/off independently of the active level via VITE_OVERRIDE_SELECTOR_ROTATION / VITE_OVERRIDE_DOM_MUTATION / VITE_OVERRIDE_ASYNC_DELAY, useful for isolating one mechanism's effect during development without inventing a new named chaos level for every combination. CHAOS_LEVELS's three named scenarios remain the only thing the eventual benchmark runner configures via plain CHAOS_LEVEL.

End goal (Sprint 8 — Healing Benchmark Runner): run the full suite at every level, comparing No Healer vs. Heuristic Healer vs. LLM Healer — not just "it works," but "here's exactly how much the LLM adds over a cheap fuzzy-match baseline, and where."

Chaos Level	No Healer	Heuristic Healer	LLM Healer
LOW	~72%	?%	~98%
MEDIUM	~51%	?%	~95%
HIGH	~29%	?%	~90%

The middle column is the actual experiment — a simple fuzzy/Levenshtein selector matcher (zero LLM calls, same provider interface as Anthropic/Ollama) might already solve a surprising fraction of cases. Without this baseline, "90% healed" doesn't prove the LLM was necessary.

Autonomous Mode has hard stop conditions from day one (max_attempts_total, token budget, max_time_per_heal) — no infinite LLM retry loops in CI, by design, not as a later hardening pass. Budget is tracked in tokens and elapsed time, never in currency — model pricing changes over time, token counts don't. See docs/architecture-decisions.md for the full reasoning.

PhoenixQA/
├── LEARNINGS.md             # chronological journal — problem → analysis → decision → test → conclusion
├── docs/                    # thematic indexes (fast lookup by topic, not by sprint)
│   ├── gaps.md              # all numbered architectural gaps, status at a glance
│   ├── architecture-decisions.md
│   ├── known-limitations.md
│   ├── future-ideas.md
│   ├── testing-strategy.md  # unit/integration/e2e/regression-benchmark/non-functional plan + actual state
│   └── research_hypotheses.md  # open empirical questions the project is trying to answer, not just architectural gaps to fix
├── chaos_app/                # React/Vite — intentionally unstable test target
│   └── src/chaos/            # selectorRotation, domMutation, asyncDelay, shadowDom, componentRemount (Sprint 6A)
├── phoenix/
│   ├── collector/            # failure_classifier, context_collector (router, planned) + collectors/ (per-FailureType, planned)
│   ├── healing/               # healer, safe_mode, decision_logger, autonomous_mode
│   ├── ai/                   # base_provider, ollama_provider, anthropic_provider,
│   │                         # prompts/ (per-FailureType, planned), response_parser, provider_factory
│   ├── training/             # Healing history (Sprint 7)
│   └── reporting/            # Allure Phoenix Healing Report (Sprint 9)
├── pages/                    # Page Objects for Chaos App (POM pattern)
├── tests/
│   ├── chaos/                # tests running against Chaos App
│   ├── unit/                 # tokenizer, classifier, parser, logger, healer tests
│   └── integration/
└── config/

---

🔒 Privacy-first AI design

Provider	When to use	Status
ollama	Air-gapped / NDA environments, local LLM	✅ Live — the only working provider today
anthropic	Cloud projects, best quality healing suggestions	🔴 Stub — AnthropicProvider.analyze_failure()/health_check() both raise NotImplementedError

Switch via single env variable — architecturally true (provider_factory.py already dispatches on AI_PROVIDER), but setting AI_PROVIDER=anthropic today raises immediately, not a working alternative yet.

---

🗺️ Roadmap

Sprint	Focus	Status
Sprint 0	Repo scaffold, config, AI provider stubs	✅ Done
Sprint 1	Chaos App — React/Vite, selector rotation, DOM mutation, async delay, Shadow DOM	✅ Done
Sprint 2	Context Collector — selector_not_found only (DOM snapshot, weighted scoring)	✅ Done
Sprint 3	LLM Analyzer — prompt engineering, structured JSON response, confidence score	✅ Done
Sprint 4	Safe Mode — Human-in-the-loop terminal review, JSON-lines decision log	✅ Done
Sprint 5	Autonomous Mode — stop conditions (attempts/tokens/time budget), confidence policy gate, distinct exception types	✅ Done
Sprint 6	Failure type expansion — action-recovery reframing, polymorphic collector, split prompts, HealingAction hierarchy all implemented; target failure type redirected after Sprint 6A findings	🚧 In progress
Sprint 6A	componentRemount.jsx (TIMEOUT + MOUSEDOWN triggers) + classifier extended to recognize DETACHED_FROM_DOM	✅ Done — controlled experiment (4 escalating configurations) found no reproduction against Locator-based automation; a finding about Playwright's architecture, not a mechanism gap. See LEARNINGS.md "Sprint 6A conclusion"
Sprint 6B	FailureCategory/ActionabilityReason model: classifier rewrite, ContextCollector router split, HealingAction hierarchy, ActionabilityCollector + provider path + actionability_policy.py guardrail for RECEIVES_EVENTS and VISIBLE	✅ Both reasons implemented, unit-tested (134/134), AND live-verified end-to-end, including both directions of the policy guardrail. Healer still rejects any ActionabilityStrategy — proposal only, no execution. ENABLED/EDITABLE not started, STABLE blocked. See LEARNINGS.md Sprint 6B
Sprint 6C-D	Remaining ActionabilityReason slice (EDITABLE)	⏸️ ENABLED done instead (Sprint 8) — see below; EDITABLE still not started
Sprint 8, third reason	ENABLED — full vertical slice (Chaos App disabledState.js, collector, prompt, policy, provider wiring, Healer execution)	✅ Implemented, unit-verified (214/214), AND live-verified 2/2 (VITE_DISABLED_STATE_MS=11200)
Sprint 7	Healing History — SQLite store, decision log, healing correctness definition	⏳ Planned
Sprint 8	Narrow baseline slice (Gap #9, decided) — HeuristicProvider (LOCATOR_RESOLUTION, fuzzy matching) + PolicyOnlyProvider (ACTIONABILITY/VISIBLE, decides off collector_metadata directly) vs. llama3.2, small-scale, deliberately BEFORE the full multi-level Benchmark Runner. Sequenced ahead of Sprint 6C-D — see LEARNINGS.md "Sprint 8 (pre-coding)"	✅ Complete — 8/8 across LOW/MEDIUM, baseline matched llama3.2 every time. ticket_row_ambiguity near-tie idea investigated and deprioritized (Gap #16)
Sprint 8, Option A (narrowed)	Healer executes an approved ActionabilityStrategy — scoped to VISIBLE's WAIT_AND_RETRY/NO_SAFE_RECOVERY only, RECEIVES_EVENTS untouched	✅ Implemented, unit-tested (185/185) — live verification against a real Chaos App still pending
Sprint 9	Allure Phoenix Report, CI/CD, demo GIF	⏳ Planned

Sprint 6 architectural decisions made before any code was written (full reasoning in LEARNINGS.md, indexed in docs/gaps.md Gap #12 and docs/architecture-decisions.md):

1. The non-selector_not_found failure types are reframed as action recovery, not selector replacement — there may be nothing wrong with the selector itself.
2. ContextCollector becomes a router over one BaseContextCollector subclass per FailureType, replacing the current if/elif ladder.
3. The prompt layer splits the same way — one prompt module per FailureType, since "find a selector" and "should this action be retried, and after what wait" are different cognitive tasks for the model.
4. HealingProposal is retired as the universal provider return type, replaced by a HealingAction hierarchy — SelectorReplacement (locator_resolution), one merged ActionabilityStrategy parameterized by reason + recovery kind (actionability, superseding the originally-declared separate WaitStrategy/VisibilityStrategy), and a dormant RetryStrategy (reference) — a required, blocking refactor touching Healer, safe_mode.py, decision_logger.py, and response_parser.py. See LEARNINGS.md "Sprint 6B (decision)" for the finalized shape.

These four decisions have all been implemented as described — they were designed to generalize, not built around detached_from_dom specifically, and turned out to generalize cleanly to ActionabilityReason. What changed after Sprint 6A is only the answer to "which failure type comes next": a controlled experiment against Playwright's Locator API (see the Scope section above and LEARNINGS.md "Sprint 6A conclusion" for the full hypothesis → experiment → finding → decision trail) found that detached_from_dom isn't a high-value target for this architecture right now. A follow-up round of diagnostics (Sprint 6B) then found that picking not_visible or timeout_waiting outright wasn't the right move either — Playwright's own model splits failures into "did the locator resolve" and, if so, one of five actionability reasons. Two of those five (RECEIVES_EVENTS, VISIBLE) are now fully built and live-verified; the remaining open decision is whether Sprint 6C-D adds a third reason or builds actual execution of an already-approved ActionabilityStrategy — see docs/gaps.md Gap #12.

---

🚀 Quickstart

# 1. Clone
git clone https://github.com/MarcinMikula/PhoenixQA.git
cd PhoenixQA

# 2. Install Python deps
pip install -r requirements.txt
playwright install chromium

# 3. Configure
cp .env.example .env
# Edit .env — choose AI provider, chaos level

# 4. Run the Chaos App (test target)
cd chaos_app
npm install
cp .env.example .env
npm run dev
# → http://localhost:5173

# 5. In a SEPARATE terminal (npm run dev keeps step 4's terminal busy):
cd ..
# Run tests against it — both Safe Mode (Sprint 4) and Autonomous Mode
# (Sprint 5) are live. Switch via .env: HEALING_MODE=safe | autonomous
#
# -s is REQUIRED for Safe Mode: it prompts for accept/reject via input(),
# and pytest swallows stdin/stdout without -s — the prompt never
# reaches the terminal and the run just hangs with no explanation.
# Autonomous Mode doesn't need -s (no prompts), but it doesn't hurt either.
pytest tests/chaos/ -m chaos -s

---

🎬 Demo

Healing is confirmed working live as of Sprint 5 — Safe Mode and Autonomous Mode have both been run end-to-end against the real Chaos App and a local LLM, with selectors successfully healed and retried in place.

The actual demo artifact, though, is parked until Sprint 9: rather than a pile of terminal screenshots, the plan is a single Allure Healing Dashboard (success rate, healing timeline, confidence distribution, top repaired selectors, failure reasons, budget usage, provider comparison) — built once Sprint 6-8 (failure type expansion, healing history, benchmark runner) produce real data for it to render. See docs/future-ideas.md for the reasoning.

---

🤝 Part of the QA Ecosystem

PhoenixQA is one piece of a larger AI-powered QA toolkit:

Repo	Role
qa-automation-framework	POM/SOM skeleton — PhoenixQA heals its selectors
defect-pilot	AI bug reproduction & retest agent
llm-qa-toolkit	LLM-as-judge test framework for AI chatbots

---

📄 License

MIT