Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,15 @@ Run a deterministic RAG environment episode and trajectory report:
bun run --filter './examples/rag-environment-episode' dev
```

Run that episode through the local CLI and write trajectory reports:

```bash
bun run --filter '@ignitionai/agent-trainer-cli' dev -- environment run ./examples/rag-environment-episode/src/index.ts \
--json reports/rag-trajectory.json \
--markdown reports/rag-trajectory.md \
--offline-records
```

---

## Minimal usage
Expand Down
2 changes: 2 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion docs/ALPHA_READINESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
| `@ignitionai/agent-trainer-rl` | ready | ready | partial | ready | prototype | Deterministic policy helpers, trajectory recorder, fixed-strategy bandits, offline policy evaluation, GRPO-style selection and PPO interface skeletons are tested; deeper RL is intentionally absent. |
| `@ignitionai/agent-trainer` | ready | ready | ready | ready | ready | Deterministic recommendation, candidate evaluation and grid search are tested. |

## Verified Through PR #49
## Verified Through PR #50

- Package names use the `@ignitionai/*` scope.
- Root and workspace package versions are aligned on `0.1.0-alpha.0`.
Expand All @@ -56,6 +56,7 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
- Ecosystem adapters have a deterministic grouped example covering LangChain, LangGraph, Mastra and Vercel AI SDK-style shapes.
- Core runtime validation helpers cover datasets, variants, run results, usage metrics, traces, metric/reward results, normalized scores and JSON-compatible fields.
- npm alpha publishing policy is manual-only for `v0.1.0-alpha.x`; automated publishing is disabled until a dedicated Trusted Publishing/OIDC PR exists.
- The CLI can run deterministic environment episode modules, export trajectory JSON/Markdown reports and print offline policy record counts.

## Known Work After Internal Alpha

Expand Down
9 changes: 4 additions & 5 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@ Status values:

Current snapshot:

- Completed through PR #48.
- PR #49 is the current release-policy PR.
- PR #50 is the next planned CLI/environment ergonomics PR.
- Completed through PR #49.
- PR #50 is the current CLI/environment ergonomics PR.
- The dogfood-driven policy optimization loop stays blocked until dogfood or representative trajectory fixtures exist.

## Stable PR sequence
Expand Down Expand Up @@ -3041,7 +3040,7 @@ Next PR:

Status:

- current
- completed

Branch:

Expand Down Expand Up @@ -3099,7 +3098,7 @@ Next PR:

Status:

- planned
- current

Branch:

Expand Down
10 changes: 5 additions & 5 deletions docs/PROJECT_AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
| context engineering example | done | `examples/context-engineering` | Yes | Mocked strategy comparison plus CLI module. |
| exporters | done | `packages/exporters` | Yes | Stable JSON/Markdown export shape and local report bundle writer. |
| typed experiment definitions | done | `packages/experiments/src/definition.ts` | Yes | Used by the CLI example. |
| CLI runner | partial | `packages/cli` | Yes | Runs experiments, reports, bundles, local history, baseline selection and regression checks. |
| CLI runner | partial | `packages/cli` | Yes | Runs experiments, reports, bundles, local history, baseline selection, regression checks and environment episode trajectory reports. |
| regression gates | done | `packages/experiments/src/regression-gates.ts` | Yes | Tested comparison helpers plus a copyable CI example. |
| alpha dogfood workflow | done | `examples/alpha-dogfood` | Yes | Deterministic IgnitionRAG-style document assistant evaluation with report exports and a regression gate. |
| ecosystem adapters | partial | `packages/adapter-*` | No | Structural adapters exist with tests/docs and a grouped mocked example; deeper framework internals remain out of scope. |
Expand All @@ -250,9 +250,9 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
| IgnitionRAG evaluation bridge prototype | prototype | `examples/ignitionrag-evaluation-bridge` | No | Deterministic record mapping only; no database, hosted worker, auth or real provider calls. |
| file-based history | done | `packages/experiments/src/history.ts` and `packages/cli` | Yes | JSONL local history helpers plus CLI list/show/record workflows. |
| policy abstraction | partial | `packages/rl/src/policy.ts` | No | Deterministic static and score-based selection only; no training loop. |
| environment episodes | partial | `packages/environment` | No | Tested episode runner with max-step safety, seed forwarding, final state and metadata. |
| trajectory recorder | partial | `packages/rl/src/trajectory.ts` | No | Local state/action/reward/outcome records with deterministic summaries. |
| trajectory reports | partial | `packages/rl/src/episode-trajectory.ts` | No | Environment episodes can become trajectories, Markdown reports and offline policy records. |
| environment episodes | partial | `packages/environment` | No | Tested episode runner with max-step safety, seed forwarding, final state, metadata and reusable CLI-loadable episode definitions. |
| trajectory recorder | partial | `packages/rl/src/trajectory.ts` | No | Local state/action/reward/outcome records with deterministic summaries and CLI report export support. |
| trajectory reports | partial | `packages/rl/src/episode-trajectory.ts` | No | Environment episodes can become trajectories, Markdown reports, JSON reports and offline policy records. |
| npm publishing policy | done | `docs/NPM_ALPHA_PUBLISHING.md` | Yes | Alpha publishing is manual-only with `--tag alpha`; future automation must use Trusted Publishing/OIDC and manual approval. |
| bandit prototype | prototype | `packages/rl/src/strategy-bandit.ts` | No | Clearly experimental, fixed arms only, no PPO. |
| contextual bandit prototype | prototype | `packages/rl/src/contextual-bandit.ts` | No | Deterministic fixed-feature scoring over task type, citation need, cost sensitivity, latency sensitivity and risk level. |
Expand All @@ -270,7 +270,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
- IgnitionRAG integration is design-only.
- Core still needs full experiment report schema validation before alpha-stable status.
- Lightweight policy optimization remains blocked until dogfood or representative trajectory fixtures exist.
- Environment episodes still need real dogfood data and CLI ergonomics before stable status.
- Environment episodes still need real dogfood data before stable status.
- Ecosystem adapters are minimal and structural.
- CLI still lacks watch mode, remote execution and hosted history.
- Bandit support is prototype-only.
Expand Down
13 changes: 13 additions & 0 deletions examples/rag-environment-episode/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,17 @@ Each action returns a reward. The completed episode is converted into a trajecto
bun run --filter './examples/rag-environment-episode' dev
```

Run the same episode through the local CLI and write trajectory reports:

```bash
bun run --filter '@ignitionai/agent-trainer-cli' dev -- environment run ./examples/rag-environment-episode/src/index.ts \
--seed 7 \
--max-steps 10 \
--policy-id scripted-rag-policy \
--trajectory-id rag-environment-episode \
--json reports/rag-trajectory.json \
--markdown reports/rag-trajectory.md \
--offline-records
```

The example is fully mocked. It does not call LLM providers, vector databases or external tools.
85 changes: 53 additions & 32 deletions examples/rag-environment-episode/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import { pathToFileURL } from "node:url";
import {
type AgentEnvironment,
defineEnvironmentEpisode,
type EnvironmentAction,
type EnvironmentState,
type EnvironmentStepResult,
type Policy,
runEpisode,
} from "@ignitionai/agent-trainer-environment";
import {
createOfflinePolicyRecordsFromTrajectories,
Expand All @@ -16,7 +17,7 @@ import {

type RagActionName = "search" | "rerank" | "verify" | "answer";

class ScriptedRagPolicy implements Policy {
export class ScriptedRagPolicy implements Policy {
private readonly sequence: RagActionName[] = ["search", "rerank", "verify", "answer"];
private index = 0;

Expand All @@ -32,7 +33,7 @@ class ScriptedRagPolicy implements Policy {
}
}

class MockRagEnvironment implements AgentEnvironment {
export class MockRagEnvironment implements AgentEnvironment {
private stage: RagActionName | "start" | "done" = "start";

async reset(seed?: number): Promise<EnvironmentState> {
Expand Down Expand Up @@ -116,38 +117,53 @@ class MockRagEnvironment implements AgentEnvironment {
}
}

const episode = await runEpisode(new MockRagEnvironment(), new ScriptedRagPolicy(), {
seed: 7,
policyId: "scripted-rag-policy",
metadata: { example: "rag-environment-episode" },
const definition = defineEnvironmentEpisode({
name: "rag-environment-episode",
environment: () => new MockRagEnvironment(),
policy: () => new ScriptedRagPolicy(),
options: {
seed: 7,
policyId: "scripted-rag-policy",
metadata: { example: "rag-environment-episode" },
},
});

const trajectory = recordEpisodeTrajectory(episode, {
id: "rag-environment-episode",
startedAt: "2026-06-30T00:00:00.000Z",
endedAt: "2026-06-30T00:00:04.000Z",
});
const summary = summarizeTrajectory(trajectory);
const offlineRecords = createOfflinePolicyRecordsFromTrajectories([trajectory], {
experimentName: "rag-environment-episode",
});
const report = exportTrajectoryReport(trajectory, {
generatedAt: "2026-06-30T00:00:05.000Z",
});
export default definition;

export async function runExample(): Promise<void> {
const episode = await definition.run();

const trajectory = recordEpisodeTrajectory(episode, {
id: "rag-environment-episode",
startedAt: "2026-06-30T00:00:00.000Z",
endedAt: "2026-06-30T00:00:04.000Z",
});
const summary = summarizeTrajectory(trajectory);
const offlineRecords = createOfflinePolicyRecordsFromTrajectories([trajectory], {
experimentName: "rag-environment-episode",
});
const report = exportTrajectoryReport(trajectory, {
generatedAt: "2026-06-30T00:00:05.000Z",
});

console.log("RAG environment episode");
console.table(
episode.steps.map((step) => ({
action: step.action.name,
reward: (step.reward.score * step.reward.weight).toFixed(3),
done: step.done,
})),
);
console.log(`Total reward: ${episode.totalReward.toFixed(3)}`);
console.log(`Trajectory steps: ${summary.stepCount}`);
console.log(`Offline records: ${offlineRecords.length}`);
console.log("");
console.log(toMarkdownTrajectoryReport(report));
console.log("RAG environment episode");
console.table(
episode.steps.map((step) => ({
action: step.action.name,
reward: (step.reward.score * step.reward.weight).toFixed(3),
done: step.done,
})),
);
console.log(`Total reward: ${episode.totalReward.toFixed(3)}`);
console.log(`Trajectory steps: ${summary.stepCount}`);
console.log(`Offline records: ${offlineRecords.length}`);
console.log("");
console.log(toMarkdownTrajectoryReport(report));
}

if (isMainModule()) {
await runExample();
}

function state(
id: string,
Expand All @@ -160,3 +176,8 @@ function state(
...(done ? { done: true } : {}),
};
}

function isMainModule(): boolean {
const entrypoint = process.argv[1];
return entrypoint !== undefined && import.meta.url === pathToFileURL(entrypoint).href;
}
31 changes: 30 additions & 1 deletion packages/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,35 @@ bun run --filter '@ignitionai/agent-trainer-cli' dev -- eval run ./examples/cont

Use `--variant <id>` one or more times when only specific variants should be checked.

## Run Environment Episodes

Use `environment run` for deterministic `@ignitionai/agent-trainer-environment` episode modules:

```bash
bun run --filter '@ignitionai/agent-trainer-cli' dev -- environment run ./examples/rag-environment-episode/src/index.ts \
--seed 7 \
--max-steps 10 \
--policy-id scripted-rag-policy \
--trajectory-id rag-environment-episode \
--json reports/rag-trajectory.json \
--markdown reports/rag-trajectory.md \
--offline-records
```

The episode file must default export an `EnvironmentEpisodeDefinition` created with `defineEnvironmentEpisode()`:

```ts
import { defineEnvironmentEpisode } from "@ignitionai/agent-trainer-environment";

export default defineEnvironmentEpisode({
name: "rag-environment-episode",
environment: () => environment,
policy: () => policy,
});
```

The command prints step rewards, total reward, average reward and trajectory summary. `--json` writes the stable `ignition.trajectory-report.v1` shape. `--markdown` writes the Markdown trajectory report. `--offline-records` prints how many offline policy records can be created from the trajectory.

## Non-goals

The CLI does not implement watch mode, remote execution, hosted dashboards, auth, provider keys, hosted history or provider-backed regression scoring.
The CLI does not implement watch mode, remote execution, hosted dashboards, auth, provider keys, hosted history, policy optimization, model training or provider-backed regression scoring.
4 changes: 3 additions & 1 deletion packages/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,10 @@
"dependencies": {
"@ignitionai/agent-trainer": "0.1.0-alpha.1",
"@ignitionai/agent-trainer-core": "0.1.0-alpha.1",
"@ignitionai/agent-trainer-environment": "0.1.0-alpha.1",
"@ignitionai/agent-trainer-experiments": "0.1.0-alpha.1",
"@ignitionai/agent-trainer-exporters": "0.1.0-alpha.1"
"@ignitionai/agent-trainer-exporters": "0.1.0-alpha.1",
"@ignitionai/agent-trainer-rl": "0.1.0-alpha.1"
},
"devDependencies": {
"tsup": "latest",
Expand Down
Loading
Loading