feat(core): add runtime validation helpers

docs/ALPHA_READINESS.md

@@ -28,7 +28,7 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
 | `@ignitionai/agent-trainer-adapter-mastra` | ready | ready | ready | ready | partial | Structural adapter with mocked example coverage; no memory, tool or full Mastra coverage. |
 | `@ignitionai/agent-trainer-adapter-vercel-ai` | ready | ready | ready | ready | partial | Structural adapter with mocked example coverage; no streaming, tools or live provider calls. |
 | `@ignitionai/agent-trainer-cli` | ready | ready | ready | ready | partial | Runs typed experiments, writes reports/bundles, records local history, selects baselines and runs regression checks; no watch mode or remote execution. |
-| `@ignitionai/agent-trainer-core` | ready | ready | ready | ready | partial | Foundational helpers have dedicated tests; runtime schema validation remains outside the current helper surface. |
+| `@ignitionai/agent-trainer-core` | ready | ready | ready | ready | partial | Foundational helpers and runtime validation helpers have dedicated tests; full experiment report schema validation remains outside the current helper surface. |
 | `@ignitionai/agent-trainer-environment` | ready | ready | ready | ready | partial | Tested episode runner with safety guards and a deterministic RAG episode example; no production runtime or optimization loop. |
 | `@ignitionai/agent-trainer-evals` | ready | ready | ready | ready | partial | Current rewards are tested; RAG presets and richer scoring are still missing. |
 | `@ignitionai/agent-trainer-experiments` | ready | ready | ready | ready | ready | Local runner, definitions, gates and JSONL history are tested and documented. |

docs/BACKLOG.md

@@ -11,9 +11,9 @@ Status values:
 Current snapshot:
 
 - Completed through PR #47.
-- No unblocked implementation PR is currently selected.
-- PR #48 is the next RL/product loop, but it stays blocked until dogfood or representative trajectory fixtures exist.
-- Useful unblocked maintenance can still happen as small focused PRs for docs, publish automation decisions or issues found while dogfooding.
+- PR #48 is the current unblocked core validation PR.
+- PR #49 and PR #50 are the next planned release-policy and CLI/environment ergonomics PRs.
+- The dogfood-driven policy optimization loop stays blocked until dogfood or representative trajectory fixtures exist.
 
 ## Stable PR sequence
 
@@ -2972,9 +2972,192 @@ Definition of done:
 
 Next PR:
 
-- PR #48 - `feat: add dogfood-driven policy optimization loop`
+- PR #48 - `feat(core): add runtime validation helpers`
 
-### PR #48 - `feat: add dogfood-driven policy optimization loop`
+### PR #48 - `feat(core): add runtime validation helpers`
+
+Status:
+
+- current
+
+Branch:
+
+```txt
+feat/core-runtime-validation-helpers
+```
+
+Goal:
+
+Move `@ignitionai/agent-trainer-core` closer to alpha-stable status by validating core runtime shapes.
+
+Scope:
+
+- add runtime assertions and non-throwing validators for datasets, dataset items, variants, adapters, run results, usage metrics, traces, metric results, reward results, normalized scores and JSON-compatible fields,
+- wire high-risk core entry points through the validators where compatible,
+- document the validation surface in `packages/core/README.md`,
+- update audit/readiness docs to remove the stale "no runtime schema validation" limitation for the covered core surface,
+- add focused tests for positive and failure paths.
+
+Out of scope:
+
+- external schema libraries,
+- full experiment report validation,
+- public API redesign,
+- provider calls,
+- database or hosted validation services.
+
+Required APIs / files:
+
+- `packages/core/src/validation.ts`,
+- `packages/core/src/dataset.ts`,
+- `packages/core/src/adapter.ts`,
+- `packages/core/src/index.ts`,
+- `packages/core/src/index.test.ts`,
+- `packages/core/README.md`,
+- alpha readiness and project audit docs.
+
+Acceptance:
+
+```bash
+bun run lint
+bun run typecheck
+bun run test
+bun run build
+bun run pack:check
+```
+
+Definition of done:
+
+- developers can validate or assert common core runtime values without adding dependencies,
+- invalid usage, traces, scores and serialized JSON fields fail with clear errors,
+- existing public helpers keep working,
+- docs explain the validation limits.
+
+Next PR:
+
+- PR #49 - `docs(release): decide npm publish automation policy`
+
+### PR #49 - `docs(release): decide npm publish automation policy`
+
+Status:
+
+- planned
+
+Branch:
+
+```txt
+docs/npm-publish-automation-policy
+```
+
+Goal:
+
+Make the alpha npm publishing policy explicit before any future public release automation.
+
+Scope:
+
+- decide manual vs GitHub Actions publication policy,
+- document dist-tag policy, especially `alpha` and no accidental `latest`,
+- document npm provenance posture,
+- document OTP/2FA and org permission expectations,
+- document what must pass before any publish attempt.
+
+Out of scope:
+
+- actually publishing packages,
+- introducing automatic npm publish in this PR,
+- changing package names or versions unless required by the policy doc,
+- release dashboard or hosted workflow.
+
+Required APIs / files:
+
+- `docs/NPM_ALPHA_PUBLISHING.md`,
+- `docs/ALPHA_RELEASE.md`,
+- `docs/CODEX_RUNBOOK.md` if runbook steps change,
+- alpha readiness/backlog docs.
+
+Acceptance:
+
+```bash
+bun run lint
+bun run typecheck
+bun run test
+bun run build
+bun run pack:check
+```
+
+Definition of done:
+
+- the repo clearly says whether alpha publishing is manual or automated,
+- npm tag, provenance, OTP and permission rules are explicit,
+- no workflow can publish accidentally.
+
+Next PR:
+
+- PR #50 - `feat(cli): add environment episode trajectory commands`
+
+### PR #50 - `feat(cli): add environment episode trajectory commands`
+
+Status:
+
+- planned
+
+Branch:
+
+```txt
+feat/cli-environment-trajectory-commands
+```
+
+Goal:
+
+Expose environment episode and trajectory report ergonomics through the CLI without adding model training.
+
+Scope:
+
+- add a CLI command that loads a deterministic environment episode module,
+- run `runEpisode()` with seed, max steps, policy id and metadata options,
+- print episode steps, total reward and trajectory summary,
+- optionally write JSON and Markdown trajectory reports,
+- optionally print offline policy record counts,
+- add focused CLI parser/runtime tests and docs.
+
+Out of scope:
+
+- PPO, GRPO or neural policy training,
+- live provider calls,
+- production routing,
+- database or hosted trajectory store,
+- policy optimization loop.
+
+Required APIs / files:
+
+- `packages/cli`,
+- `packages/environment`,
+- `packages/rl`,
+- `examples/rag-environment-episode` if a reusable module export is needed,
+- CLI/environment/rl READMEs and audit docs.
+
+Acceptance:
+
+```bash
+bun run lint
+bun run typecheck
+bun run test
+bun run build
+bun run pack:check
+```
+
+Definition of done:
+
+- a developer can run an episode module from the CLI,
+- JSON and Markdown trajectory reports are writable from CLI flags,
+- offline record count is visible,
+- docs explicitly say this is not a training loop.
+
+Next PR:
+
+- PR #51 - `feat: add dogfood-driven policy optimization loop`
+
+### PR #51 - `feat: add dogfood-driven policy optimization loop`
 
 Status:
 

docs/PROJECT_AUDIT.md

@@ -229,7 +229,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 
 | Capability | Status | Package/File | Stable? | Notes |
 |---|---|---|---|---|
-| core primitives | partial | `packages/core` | No | Core types and helpers have dedicated tests; runtime schema validation remains out of scope. |
+| core primitives | partial | `packages/core` | No | Core types, helpers and runtime validation helpers have dedicated tests; full experiment report schema validation remains out of scope. |
 | IgnitionRAG adapter contract | partial | `packages/adapter-ignitionrag` | No | Type-level contract only; no runtime IgnitionRAG integration. |
 | evals/rewards | partial | `packages/evals` | No | Tests and README exist, but reward set is intentionally small. |
 | RAG presets | partial | `packages/preset-rag` | No | Deterministic presets compose text, citation, latency, cost and tool-use rewards. |
@@ -267,7 +267,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
 - No real LLM calls by default.
 - No hosted IgnitionRAG integration code.
 - IgnitionRAG integration is design-only.
-- Core still needs runtime schema validation before alpha-stable status.
+- Core still needs full experiment report schema validation before alpha-stable status.
 - Lightweight policy optimization remains blocked until dogfood or representative trajectory fixtures exist.
 - Environment episodes still need real dogfood data and CLI ergonomics before stable status.
 - Ecosystem adapters are minimal and structural.

packages/core/README.md

@@ -8,28 +8,33 @@ Use this package when defining datasets, agent adapters, traces, usage metrics,
 
 ```ts
 import {
+  assertRunResult,
   createDataset,
   createMockAdapter,
   normalizeRunResult,
   toAgentInput,
+  validateDataset,
   weightedAverage,
 } from "@ignitionai/agent-trainer-core";
 ```
 
 Main exports:
 
 - dataset helpers: `createDataset`, `assertDatasetItem`,
+- runtime validation helpers: `assertDataset`, `validateDataset`, `assertAgentVariant`, `validateAgentVariant`, `assertRunResult`, `validateRunResult`, `assertUsageMetrics`, `validateUsageMetrics`, `assertTrace`, `validateTrace`, `assertMetricResult`, `validateMetricResult`, `assertRewardResult`, `assertNormalizedScore`, `assertJsonValue`,
 - adapter helpers: `createMockAdapter`, `normalizeRunResult`, `toAgentInput`,
 - score helpers: `clampScore`, `weightedAverage`,
 - shared types for datasets, adapters, traces, rewards, cases, leaderboards and experiment reports.
 
+The `assert*` helpers throw clear errors and narrow TypeScript types. The `validate*` helpers return `{ ok: true, value }` or `{ ok: false, error }` when callers need non-throwing validation.
+
 ## Alpha Readiness Status
 
 This package is foundational and covered by dedicated package-level tests for the current helper surface.
 
 Known gaps:
 
-- no runtime schema validation for serialized reports,
+- runtime validation covers core datasets, variants, run results, usage, traces and scores; full serialized experiment report schema validation remains outside the current helper surface,
 - no compatibility policy beyond the current monorepo usage.
 
 ## Non-goals

packages/core/src/adapter.ts

@@ -11,6 +11,7 @@ import type {
   Trace,
   UsageMetrics,
 } from "./types";
+import { assertRunResult, assertUsageMetrics } from "./validation";
 
 export type MockAdapterHandler =
   | AgentAdapterResult
@@ -26,6 +27,9 @@ export function createMockAdapter(
   handler: MockAdapterHandler,
   options: MockAdapterOptions = {},
 ): AgentAdapter {
+  if (options.trace !== undefined) assertRunResult({ output: "", trace: options.trace });
+  if (options.usage !== undefined) assertUsageMetrics(options.usage);
+
   const adapter: AgentAdapter = {
     async run(input, context) {
       const value = typeof handler === "function" ? await handler(input, context) : handler;
@@ -56,12 +60,14 @@ export function toAgentInput(item: DatasetItem): AgentInput {
 
 export function normalizeRunResult(value: AgentAdapterResult): RunResult {
   if (isRunResultLike(value)) {
-    return {
+    const result = {
       output: value.output,
       trace: value.trace ?? { steps: [] },
       ...(value.usage !== undefined ? { usage: value.usage } : {}),
       ...(value.metadata !== undefined ? { metadata: value.metadata } : {}),
     };
+    assertRunResult(result);
+    return result;
   }
 
   return {

packages/core/src/dataset.ts

@@ -1,31 +1,12 @@
 import type { Dataset, DatasetItem } from "./types";
+import { assertDataset } from "./validation";
 
 export function createDataset(items: DatasetItem[]): Dataset;
 export function createDataset(input: Dataset): Dataset;
 export function createDataset(input: Dataset | DatasetItem[]): Dataset {
   const dataset = Array.isArray(input) ? { name: "dataset", items: input } : input;
 
-  if (!dataset.name.trim()) {
-    throw new Error("Dataset name is required.");
-  }
-
-  const seen = new Set<string>();
-  for (const item of dataset.items) {
-    assertDatasetItem(item);
-    if (seen.has(item.id)) {
-      throw new Error(`Duplicate dataset item id: ${item.id}`);
-    }
-    seen.add(item.id);
-  }
+  assertDataset(dataset);
 
   return dataset;
 }
-
-export function assertDatasetItem(item: DatasetItem): void {
-  if (!item.id.trim()) {
-    throw new Error("Dataset item id is required.");
-  }
-  if (!item.input.trim()) {
-    throw new Error(`Dataset item ${item.id} input is required.`);
-  }
-}