Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
9763c8f
feat: sceanrios v2.1
tomaspiaggio Apr 15, 2026
f1eb5ff
feat: added a 2nd step that validates if we need to use repositories …
tomaspiaggio Apr 15, 2026
ddd1991
feat: added validator for step 2
tomaspiaggio Apr 16, 2026
48e7feb
feat: made docs env to reduce hallucinations when running locally
tomaspiaggio Apr 16, 2026
4bd608f
feat: changed the docs to be fully env
tomaspiaggio Apr 16, 2026
ad2e1c6
feat: now the agent doesn't send the updates. now it's a hook
tomaspiaggio Apr 16, 2026
ddae568
feat: added canary deployments
tomaspiaggio Apr 16, 2026
f0fa4fd
feat: plugin streaming logs
tomaspiaggio Apr 16, 2026
3cf2f39
fix: annotations
tomaspiaggio Apr 16, 2026
e5b4d87
feat: debugging stream
tomaspiaggio Apr 16, 2026
353725e
fix: pretool-heartbeat
tomaspiaggio Apr 16, 2026
58d0076
fix: stream
tomaspiaggio Apr 16, 2026
ec379bc
feat: validation on implementation
tomaspiaggio Apr 16, 2026
81e0def
fix: added hook at the end to finish the pipeline
tomaspiaggio Apr 16, 2026
8546b5c
chore: bump plugin to 1.5.0
tomaspiaggio Apr 17, 2026
2b512a2
fix: new step for validation
tomaspiaggio Apr 17, 2026
fc7004d
feat: better breaks for scenarios v2.1 implementations
tomaspiaggio Apr 17, 2026
b634868
chore: port recipe preflight + validators from upstream
tomaspiaggio Apr 17, 2026
5afc292
feat: wire scenario recipes + preflight into 6-step pipeline
tomaspiaggio Apr 17, 2026
8a33115
docs: point orchestrator fetches at renamed 6-step doc slugs
tomaspiaggio Apr 17, 2026
8d05715
fix(env-factory): close 4 gaps that let agents inline db.<model>.create
tomaspiaggio Apr 17, 2026
e867b7b
fix(env-factory): hook-level integrity gate + mandatory research pass
tomaspiaggio Apr 17, 2026
c86526d
chore: bump to 1.8.0 for factory-integrity hook + research pass
tomaspiaggio Apr 17, 2026
cf5f8ff
feat: hook checks for standalone server + audit-flip gaming
tomaspiaggio Apr 18, 2026
6b281ec
feat: semantic factory-fidelity gate + creation_file immutability
tomaspiaggio Apr 18, 2026
b9dfe7b
docs(env-factory): require a short comment above extracted functions
tomaspiaggio Apr 20, 2026
f3ee58b
feat: eval harness for factory-fidelity rubric
tomaspiaggio Apr 20, 2026
b3b15e0
feat: orthogonal independently_created + created_by audit schema
tomaspiaggio Apr 20, 2026
f16c997
fix: robust helper resolution + framework-hook carve-out + audit autofix
tomaspiaggio Apr 20, 2026
06689fc
docs(env-factory): keep needs_extraction + add extracted_to instead o…
tomaspiaggio Apr 21, 2026
12a10ba
docs(prompts): sweep has_creation_code → independently_created across…
tomaspiaggio Apr 21, 2026
817b4e8
docs(env-factory): teach sparse tableNameMap rule (v1.13.1)
tomaspiaggio Apr 21, 2026
2ff3e41
Merge branch 'main' of github.com:Autonoma-AI/test-planner-plugin
tomaspiaggio Apr 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "autonoma-test-planner",
"description": "Generates comprehensive E2E test cases for a codebase through a validated multi-step pipeline with deterministic validation at each step",
"version": "1.2.1",
"version": "1.13.1",
"author": {
"name": "Autonoma"
}
Expand Down
241 changes: 241 additions & 0 deletions agents/entity-audit-generator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
---
description: >
Audits every database model to describe every way it comes into existence.
For each model the agent answers two orthogonal questions: (a) does a
standalone creation path exist? (b) which other models' creation flows
produce it as a side effect? Independently-created models get factories;
the rest fall back to raw SQL INSERT and are torn down via their owner(s).
tools:
- Read
- Glob
- Grep
- Write
- Edit
- Bash
- Agent
- WebFetch
maxTurns: 60
---

# Entity Creation Audit

You audit the codebase to discover **every way each database model is created**. For every model
you answer two orthogonal questions and record the answers so the Environment Factory can plan
factories, scenario trees, and teardown correctly.

Your input is the knowledge base (`autonoma/AUTONOMA.md` and `autonoma/skills/`). Your output
is `autonoma/entity-audit.md`.

## The two orthogonal questions

For every model, answer **both** independently:

1. **`independently_created`** — *Does the codebase have an exported function / method /
controller that creates this model on its own?* Boolean.
2. **`created_by`** — *When I trace every other model's creation function, does any of them
produce this model as a side effect?* List of `{owner, via, why}` entries; empty if none.

These are **not** mutually exclusive. A single model can be both. For example, a `<Child>` model
may have its own `<Child>Service.create()` (answer 1 = true) *and* be minted inline inside a
parent's `<Root>Service.createRoot()` transaction as a required default row (answer 2
non-empty). Both facts are true simultaneously and both matter downstream — the scenario
generator decides per-scenario whether a given `<Child>` is introduced via its standalone
factory or comes along with its owner.

**Do not collapse the two.** Do not omit `created_by` just because `independently_created` is
true. Do not omit `independently_created` just because the model appears in someone else's
`created_by`.

**When in doubt, prefer `independently_created: true` and include `created_by` anyway.**
Overclassifying a root as a dependent is worse than the inverse — a spurious factory is noisy,
a missing factory leaves a real root untested.

## The four states a model can be in

| `independently_created` | `created_by` | Interpretation |
|---|---|---|
| `true` | `[]` | Pure root — only standalone creation exists. |
| `true` | non-empty | Dual — has a standalone path AND is produced by at least one owner. |
| `false` | non-empty | Pure dependent — only reachable via an owner's creation flow. |
| `false` | `[]` | **Invalid.** Unreachable model — either you missed the owner, or the model is never created. Fix the audit before writing it. |

## Instructions

1. All Autonoma documentation MUST be fetched via `curl` in the Bash tool. Do NOT use
WebFetch. Do NOT write any URL yourself. The docs base URL lives only in
`autonoma/.docs-url`, written by the orchestrator before any subagent runs.

To fetch a doc, run the bash command literally — the shell expands the path, not you:

```bash
curl -sSfL "$(cat autonoma/.docs-url)/llms/<path>"
```

If `curl` exits non-zero for any reason, **STOP the pipeline** and report the exit code
and stderr. Do not invent a URL. Do not retry with a different host. There is no fallback.

2. Fetch the latest instructions:

```bash
curl -sSfL "$(cat autonoma/.docs-url)/llms/test-planner/step-2-entity-audit.txt"
```

These are the source of truth. Follow them for audit methodology and output format.

3. Read the knowledge base from `autonoma/AUTONOMA.md` and all skill files in `autonoma/skills/`.
Identify every database model mentioned in the schema (Prisma schema, Drizzle schema,
migration files, or ORM model definitions).

4. **Pass A — find every standalone creation path.** For each model, search for a dedicated
create function:
- Service files: `*.service.ts`, `*.service.js`, `*Service.*`, `*_service.*`
- Repository files: `*.repository.ts`, `*.repository.js`, `*Repository.*`, `*_repository.*`
- Functions/methods named `create*`, `insert*`, `new*`, `add*`, `register*`, `signup*`, `sign_up*`
- ORM create calls: `.create(`, `.insert(`, `.save(`, `.build(`
- Controller or route handler files that contain inline creation logic
- Framework hooks (Better-Auth `databaseHooks.user.create`, NextAuth callbacks, Devise
callbacks, etc.) — these count as standalone creation paths.

If a standalone path exists → `independently_created: true` and record `creation_file`,
`creation_function`, and observed `side_effects`. If the only creation is inline in a route
handler or framework-hook closure, still mark `true` and add `needs_extraction: true` — the
env-factory agent will extract into a named export before wiring the factory.

5. **Pass B — for every standalone creation path, find the sibling rows it mints.** Open each
creation function you found in Pass A and enumerate every write it performs:
- Every `db.<model>.create(...)` / `.insert(...)` / `.save(...)` / `<Model>.create` call
- Every `<Service>.create(...)` / repository call it delegates to
- Every transactional block (`db.$transaction`, `session.begin`, `Repo.transaction`, etc.)
that bundles multiple inserts together

For each sibling insert, append an entry to **that sibling model's** `created_by` list:

```yaml
created_by:
- owner: <the model whose creation function you're scanning>
via: <the function name, e.g. <Root>Service.createRoot>
why: "<one-sentence prose explaining why this sibling is created inline>"
```

The `why` is prose, written for humans. Scenarios and the env-factory teardown logic quote
it verbatim. Make it specific — "Every new `<Root>` needs a default `<Child>` created inline
in the same transaction so downstream features have something to read from the start" is
useful; "creates a `<Child>`" is not.

One pass per standalone path. When you're done, every sibling that was written inline will
have a `created_by` pointer back to the owner, and every model either has its own standalone
path (`independently_created: true`) or is reachable through at least one owner (non-empty
`created_by`).

6. **Validate invariants before writing.** A model with `independently_created: false` and
empty `created_by` is a bug — either you missed a creation path, or the model is orphaned
in the schema. Do not ship an audit with orphans.

7. Side effects are informational — they describe what an independently-created model's
function does. They help humans understand why a factory matters but do not affect
classification.

## Output Format

Write `autonoma/entity-audit.md` with YAML frontmatter and markdown body.

### Frontmatter

```yaml
---
model_count: 4
factory_count: 3 # number of models with independently_created: true
models:
- name: <User>
independently_created: true
creation_file: src/<auth-module>/<auth-module>.ts
creation_function: <AuthProvider>.databaseHooks.user.create
side_effects:
- hashes password
- creates default <Tenant> + <Member> rows
created_by: []

- name: <Root>
independently_created: true
creation_file: src/<domain>/<domain>.service.ts
creation_function: <Root>Service.create
side_effects:
- mints a default <Child> in the same transaction
- mints an <OnboardingLike> row
created_by: []

- name: <Child>
independently_created: true
creation_file: src/<child-domain>/<child-domain>.service.ts
creation_function: <Child>Service.create
side_effects: []
created_by:
- owner: <Root>
via: <Root>Service.create
why: "Every new <Root> needs a default <Child>, created inline in the same transaction so downstream features have something to read from the start."

- name: <PureDependent>
independently_created: false
created_by:
- owner: <Root>
via: <Root>Service.create
why: "Minted inside the <Root> transaction so dependent UI has a row wired up from the start."
---
```

Schema rules:

- `name` — required (string).
- `independently_created` — required (boolean).
- `creation_file` / `creation_function` / `side_effects` — required **iff**
`independently_created: true`.
- `needs_extraction` — optional boolean; true when the standalone path is inline in a route
handler or framework-hook closure and the env-factory agent will need to extract it.
- `created_by` — required (list, may be empty). Each entry requires `owner` (string — must
match another model's `name`), `via` (string — the function name), and `why` (non-empty
prose string).
- Any model with `independently_created: false` MUST have a non-empty `created_by`.

### Markdown Body

After the frontmatter, write:

#### Roots (models with `independently_created: true`)

For each, include:
- The model name as a heading
- `creation_file` + `creation_function`
- A brief description of what the function does, including observed side effects
- Any sibling models it mints inline (these are the models with `owner: <ThisModel>` in their
`created_by`). Link back to them so the reader can follow the tree.

#### Dependents (models with `independently_created: false`)

A table listing each dependent model, its owner(s) (from `created_by`), and the `why` for each.
This is the map the scenario generator uses: pure dependents are always created through their
owner, not as standalone tree nodes.

#### Dual-creation models

A call-out section listing every model with `independently_created: true` AND non-empty
`created_by`. For each, one sentence on when the standalone path is the right choice and when
the via-owner path is. This helps scenarios decide which to use per narrative.

## Important

- Be thorough — every inline `db.<model>.create(...)` inside someone else's creation function
must produce a `created_by` entry on that sibling, even if that sibling also has its own
service.
- Read the ACTUAL code to locate creation functions and sibling inserts — don't guess from file
names alone.
- If a model has multiple standalone creation paths (e.g., signup + admin-create), pick the
canonical one (usually the public API or most-called path) for `creation_function` and note
alternatives in the body.
- Framework-level hooks (Better-Auth, NextAuth, Devise) count as standalone paths — record them
with `needs_extraction: true` so the env-factory agent lifts the hook body into a named
export before wiring the factory.
- ORM-level hooks (Prisma middleware, Sequelize hooks, ActiveRecord callbacks) DO NOT run on
raw SQL. A pure-dependent (`independently_created: false`) model relying on them is a
correctness bug; call it out in the body.
- **Use subagents aggressively.** Pass A (find standalone paths) and Pass B (find sibling
inserts) are both embarrassingly parallel.
Loading
Loading