feat: add ecosystem adapter examples

README.md

@@ -125,6 +125,12 @@ Run the sample CI regression gate:
 bun run --filter './examples/ci-regression-gate' dev
 ```
 
+Run mocked ecosystem adapter examples:
+
+```bash
+bun run --filter './examples/ecosystem-adapters' dev
+```
+
 Run the alpha dogfood document-assistant experiment:
 
 ```bash

docs/ALPHA_READINESS.md

@@ -23,10 +23,10 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
 |---|---|---|---|---|---|---|
 | `@ignitionai/agent-trainer-adapter-callable` | ready | ready | ready | ready | ready | Generic callable adapter has docs, tests and an example. |
 | `@ignitionai/agent-trainer-adapter-ignitionrag` | ready | ready | ready | ready | partial | Type-level contract only; no runtime IgnitionRAG integration. |
-| `@ignitionai/agent-trainer-adapter-langchain` | ready | ready | ready | ready | partial | Structural adapter only; no dedicated example or deep framework coverage. |
-| `@ignitionai/agent-trainer-adapter-langgraph` | ready | ready | ready | ready | partial | Structural adapter only; no persistence, streaming or graph internals. |
-| `@ignitionai/agent-trainer-adapter-mastra` | ready | ready | ready | ready | partial | Structural adapter only; no memory, tool or full Mastra coverage. |
-| `@ignitionai/agent-trainer-adapter-vercel-ai` | ready | ready | ready | ready | partial | Structural adapter only; no streaming, tools or live provider calls. |
+| `@ignitionai/agent-trainer-adapter-langchain` | ready | ready | ready | ready | partial | Structural adapter with mocked example coverage; no deep LangChain internals. |
+| `@ignitionai/agent-trainer-adapter-langgraph` | ready | ready | ready | ready | partial | Structural adapter with mocked example coverage; no persistence, streaming or graph internals. |
+| `@ignitionai/agent-trainer-adapter-mastra` | ready | ready | ready | ready | partial | Structural adapter with mocked example coverage; no memory, tool or full Mastra coverage. |
+| `@ignitionai/agent-trainer-adapter-vercel-ai` | ready | ready | ready | ready | partial | Structural adapter with mocked example coverage; no streaming, tools or live provider calls. |
 | `@ignitionai/agent-trainer-cli` | ready | ready | ready | ready | partial | Runs typed experiments, writes reports/bundles, records local history, selects baselines and runs regression checks; no watch mode or remote execution. |
 | `@ignitionai/agent-trainer-core` | ready | ready | ready | ready | partial | Foundational helpers have dedicated tests; runtime schema validation remains outside the current helper surface. |
 | `@ignitionai/agent-trainer-environment` | ready | ready | ready | ready | partial | Tested episode runner with safety guards and a deterministic RAG episode example; no production runtime or optimization loop. |
@@ -55,7 +55,6 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
 
 ## Known Work After Internal Alpha
 
-- Add deeper mocked examples for ecosystem adapters.
 - Dogfood the alpha inside IgnitionRAG and collect trajectory/reward evidence.
 - Add a lightweight policy optimization loop after real dogfood produces trajectory data.
 - Decide any next-step publishing automation policy before a future public release workflow.

docs/BACKLOG.md

@@ -2850,7 +2850,7 @@ Next PR:
 
 Status:
 
-- current
+- completed
 
 Branch:
 
@@ -2912,7 +2912,7 @@ Next PR:
 
 Status:
 
-- planned
+- current
 
 Branch:
 

docs/PROJECT_AUDIT.md

@@ -43,7 +43,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 - Main exports: `createLangChainAdapter`, `langChainAdapter`, `LangChainRunnableLike`.
 - Stability level: partial.
 - Tests present: yes.
-- Example present: no dedicated example.
+- Example present: yes, `examples/ecosystem-adapters`.
 - Known limitations: does not require or inspect LangChain internals, does not stream, does not call providers, and only expects `invoke()`.
 
 ### `@ignitionai/agent-trainer-adapter-langgraph`
@@ -52,7 +52,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 - Main exports: `createLangGraphAdapter`, `langGraphAdapter`, `LangGraphLike`.
 - Stability level: partial.
 - Tests present: yes.
-- Example present: no dedicated example.
+- Example present: yes, `examples/ecosystem-adapters`.
 - Known limitations: does not persist graph state, stream events, inspect graph internals or call providers.
 
 ### `@ignitionai/agent-trainer-adapter-mastra`
@@ -61,7 +61,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 - Main exports: `createMastraAdapter`, `mastraAdapter`, `MastraAgentLike`.
 - Stability level: partial.
 - Tests present: yes.
-- Example present: no dedicated example.
+- Example present: yes, `examples/ecosystem-adapters`.
 - Known limitations: only supports `generate()` or `run()` style objects, no tools, memory, persistence or full Mastra API coverage.
 
 ### `@ignitionai/agent-trainer-adapter-vercel-ai`
@@ -70,7 +70,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 - Main exports: `createVercelAiAdapter`, `vercelAiAdapter`, `VercelAiGenerateLike`.
 - Stability level: partial.
 - Tests present: yes.
-- Example present: no dedicated example.
+- Example present: yes, `examples/ecosystem-adapters`.
 - Known limitations: no real provider calls, no auth, no streaming, no tool integration and no prompt generation.
 
 ### `@ignitionai/agent-trainer-cli`
@@ -187,6 +187,14 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 - Mocked or live: mocked.
 - Product concept: context engineering strategy comparison and CLI-ready typed experiments.
 
+### `examples/ecosystem-adapters`
+
+- Demonstrates: LangChain, LangGraph, Mastra and Vercel AI SDK-style adapters wired into `createExperiment()`.
+- Command: `bun run --filter './examples/ecosystem-adapters' dev`.
+- Test command: `bun test examples/ecosystem-adapters/src/example.test.ts`.
+- Mocked or live: deterministic mocked framework objects.
+- Product concept: adapter portability with trace, usage and metadata preserved through standard experiment results.
+
 ### `examples/ci-regression-gate`
 
 - Demonstrates: comparing a current experiment result against a committed baseline in CI.
@@ -221,7 +229,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 
 | Capability | Status | Package/File | Stable? | Notes |
 |---|---|---|---|---|
-| core primitives | partial | `packages/core` | No | Core types and helpers exist, but package lacks dedicated tests. |
+| core primitives | partial | `packages/core` | No | Core types and helpers have dedicated tests; runtime schema validation remains out of scope. |
 | IgnitionRAG adapter contract | partial | `packages/adapter-ignitionrag` | No | Type-level contract only; no runtime IgnitionRAG integration. |
 | evals/rewards | partial | `packages/evals` | No | Tests and README exist, but reward set is intentionally small. |
 | RAG presets | partial | `packages/preset-rag` | No | Deterministic presets compose text, citation, latency, cost and tool-use rewards. |
@@ -232,15 +240,15 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 | context engineering example | done | `examples/context-engineering` | Yes | Mocked strategy comparison plus CLI module. |
 | exporters | done | `packages/exporters` | Yes | Stable JSON/Markdown export shape and local report bundle writer. |
 | typed experiment definitions | done | `packages/experiments/src/definition.ts` | Yes | Used by the CLI example. |
-| CLI runner | partial | `packages/cli` | Yes | Current command works; history, baselines and regression gates are not CLI flags yet. |
+| CLI runner | partial | `packages/cli` | Yes | Runs experiments, reports, bundles, local history, baseline selection and regression checks. |
 | regression gates | done | `packages/experiments/src/regression-gates.ts` | Yes | Tested comparison helpers plus a copyable CI example. |
 | alpha dogfood workflow | done | `examples/alpha-dogfood` | Yes | Deterministic IgnitionRAG-style document assistant evaluation with report exports and a regression gate. |
-| ecosystem adapters | partial | `packages/adapter-*` | No | Structural adapters exist with tests/docs, but dedicated examples and deeper framework coverage are missing. |
+| ecosystem adapters | partial | `packages/adapter-*` | No | Structural adapters exist with tests/docs and a grouped mocked example; deeper framework internals remain out of scope. |
 | simple search optimization | done | `packages/trainer/src/search.ts` | Yes | Deterministic grid search over manual parameter grids. |
 | IgnitionRAG design | done | `docs/10-ignitionrag-integration-design.md` | Yes | Design-only; no IgnitionRAG runtime integration. |
 | IgnitionRAG Evaluation Center checklist | done | `docs/IGNITIONRAG_EVALUATION_CENTER_CHECKLIST.md` | Yes | Actionable checklist only; no IgnitionRAG app code, migrations or frontend. |
 | IgnitionRAG evaluation bridge prototype | prototype | `examples/ignitionrag-evaluation-bridge` | No | Deterministic record mapping only; no database, hosted worker, auth or real provider calls. |
-| file-based history | done | `packages/experiments/src/history.ts` | Yes | JSONL local history helpers, no CLI flag yet. |
+| file-based history | done | `packages/experiments/src/history.ts` and `packages/cli` | Yes | JSONL local history helpers plus CLI list/show/record workflows. |
 | policy abstraction | partial | `packages/rl/src/policy.ts` | No | Deterministic static and score-based selection only; no training loop. |
 | environment episodes | partial | `packages/environment` | No | Tested episode runner with max-step safety, seed forwarding, final state and metadata. |
 | trajectory recorder | partial | `packages/rl/src/trajectory.ts` | No | Local state/action/reward/outcome records with deterministic summaries. |

examples/ecosystem-adapters/README.md

@@ -0,0 +1,24 @@
+# Ecosystem Adapter Examples
+
+This example runs deterministic mocked adapters for:
+
+- LangChain Runnable-like objects,
+- LangGraph graph-like objects,
+- Mastra agent-like objects,
+- Vercel AI SDK-style generate functions.
+
+Each adapter plugs into `createExperiment()` as an `AgentVariant`. The mocks return trace, usage and metadata so the example can validate adapter observability without provider credentials or network calls.
+
+Run it:
+
+```bash
+bun run --filter './examples/ecosystem-adapters' dev
+```
+
+Run the example test:
+
+```bash
+bun test examples/ecosystem-adapters/src/example.test.ts
+```
+
+This is not deep framework coverage. It proves the adapter contracts work with public package APIs and documents the shape each adapter expects.

examples/ecosystem-adapters/package.json

@@ -0,0 +1,20 @@
+{
+  "name": "ecosystem-adapters",
+  "version": "0.0.0",
+  "private": true,
+  "type": "module",
+  "scripts": {
+    "dev": "bun src/index.ts",
+    "test": "vitest run"
+  },
+  "dependencies": {
+    "@ignitionai/agent-trainer-adapter-langchain": "workspace:*",
+    "@ignitionai/agent-trainer-adapter-langgraph": "workspace:*",
+    "@ignitionai/agent-trainer-adapter-mastra": "workspace:*",
+    "@ignitionai/agent-trainer-adapter-vercel-ai": "workspace:*",
+    "@ignitionai/agent-trainer-core": "workspace:*",
+    "@ignitionai/agent-trainer-evals": "workspace:*",
+    "@ignitionai/agent-trainer-experiments": "workspace:*",
+    "vitest": "latest"
+  }
+}

examples/ecosystem-adapters/src/example.test.ts

@@ -0,0 +1,34 @@
+import { describe, expect, it } from "vitest";
+import { dataset, runEcosystemAdapterExperiment, variants } from "./example";
+
+describe("ecosystem adapter example", () => {
+  it("runs mocked ecosystem adapters through createExperiment with trace, usage and metadata", async () => {
+    const result = await runEcosystemAdapterExperiment();
+
+    expect(result.name).toBe("ecosystem-adapter-comparison");
+    expect(result.failedCases).toEqual([]);
+    expect(result.leaderboard.map((row) => row.variantId).sort()).toEqual(
+      variants.map((variant) => variant.id).sort(),
+    );
+    expect(result.cases).toHaveLength(dataset.items.length * variants.length);
+
+    for (const caseResult of result.cases) {
+      expect(caseResult.score).toBeGreaterThan(0.8);
+      expect(caseResult.trace.steps.length).toBeGreaterThan(0);
+      expect(caseResult.usage?.latencyMs).toBeGreaterThan(0);
+      expect(caseResult.usage?.costUsd).toBeGreaterThan(0);
+      expect(caseResult.metadata?.framework).toBeDefined();
+    }
+
+    expect(result.cases.map((caseResult) => caseResult.metadata?.framework).sort()).toEqual([
+      "langchain",
+      "langchain",
+      "langgraph",
+      "langgraph",
+      "mastra",
+      "mastra",
+      "vercel-ai",
+      "vercel-ai",
+    ]);
+  });
+});