Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/ALPHA_READINESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
| `@ignitionai/agent-trainer-rl` | ready | ready | partial | ready | prototype | Deterministic policy helpers, trajectory recorder, fixed-strategy bandits, offline policy evaluation, GRPO-style selection and PPO interface skeletons are tested; deeper RL is intentionally absent. |
| `@ignitionai/agent-trainer` | ready | ready | ready | ready | ready | Deterministic recommendation, candidate evaluation and grid search are tested. |

## Verified Through PR #47
## Verified Through PR #49

- Package names use the `@ignitionai/*` scope.
- Root and workspace package versions are aligned on `0.1.0-alpha.0`.
Expand All @@ -54,12 +54,14 @@ All packages declare `license: MIT`, matching the root `LICENSE` file.
- `@ignitionai/agent-trainer-core` has dedicated package-level tests for dataset helpers, adapter helpers and score helpers.
- The CLI can record local experiment history, select a baseline and run regression checks.
- Ecosystem adapters have a deterministic grouped example covering LangChain, LangGraph, Mastra and Vercel AI SDK-style shapes.
- Core runtime validation helpers cover datasets, variants, run results, usage metrics, traces, metric/reward results, normalized scores and JSON-compatible fields.
- npm alpha publishing policy is manual-only for `v0.1.0-alpha.x`; automated publishing is disabled until a dedicated Trusted Publishing/OIDC PR exists.

## Known Work After Internal Alpha

- Dogfood the alpha inside IgnitionRAG and collect trajectory/reward evidence.
- Add a lightweight policy optimization loop after real dogfood produces trajectory data.
- Decide any next-step publishing automation policy before a future public release workflow.
- Add a future npm Trusted Publishing/OIDC workflow only if manual alpha consumption proves automation is needed.

## Explicit Non-goals

Expand Down
2 changes: 2 additions & 0 deletions docs/ALPHA_RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ This document defines the internal `v0.1.0-alpha.0` tag criteria and release pro

The alpha tag is a repository milestone, not an npm publication.

For npm alpha publishing policy, see `docs/NPM_ALPHA_PUBLISHING.md`. Current npm alpha publishing is manual only; no GitHub Actions workflow should publish packages automatically.

## Target Tag

```txt
Expand Down
10 changes: 5 additions & 5 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ Status values:

Current snapshot:

- Completed through PR #47.
- PR #48 is the current unblocked core validation PR.
- PR #49 and PR #50 are the next planned release-policy and CLI/environment ergonomics PRs.
- Completed through PR #48.
- PR #49 is the current release-policy PR.
- PR #50 is the next planned CLI/environment ergonomics PR.
- The dogfood-driven policy optimization loop stays blocked until dogfood or representative trajectory fixtures exist.

## Stable PR sequence
Expand Down Expand Up @@ -2978,7 +2978,7 @@ Next PR:

Status:

- current
- completed

Branch:

Expand Down Expand Up @@ -3041,7 +3041,7 @@ Next PR:

Status:

- planned
- current

Branch:

Expand Down
10 changes: 10 additions & 0 deletions docs/CODEX_RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,16 @@ PR #42 packaging readiness

PR #42 must not publish to npm during the PR and must not add runtime features.

Current npm alpha publication policy is documented in `docs/NPM_ALPHA_PUBLISHING.md`:

```txt
manual alpha publish only
no GitHub Actions auto-publish workflow
no long-lived NPM_TOKEN
publish prereleases with --tag alpha
future automation must use npm Trusted Publishing/OIDC and manual approval
```

After the npm alpha is published, the next phase is dogfooding inside IgnitionRAG:

```txt
Expand Down
44 changes: 41 additions & 3 deletions docs/NPM_ALPHA_PUBLISHING.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,45 @@
# npm Alpha Publishing

This document defines the manual npm alpha publishing process for `v0.1.0-alpha.1`.
This document defines the npm alpha publishing policy and manual publishing process for `v0.1.0-alpha.1`.

The goal is to validate Ignition Agent Trainer as a real external dependency before dogfooding it inside IgnitionRAG. Do not publish these packages as `latest`.

## Publishing Policy

Current decision:

- `v0.1.0-alpha.x` publishing is manual only.
- This repository must not contain a GitHub Actions workflow that can publish to npm automatically.
- Alpha packages must be published with the `alpha` dist-tag.
- Never run `npm publish` without `--tag alpha` for an alpha or prerelease version.
- Do not create or store long-lived `NPM_TOKEN` secrets for this repo.
- Do not use granular access tokens for alpha publication unless a future security review explicitly changes this policy.
- Manual publishers must use an npm account with 2FA enabled and publish interactively with OTP when npm requests it.
- If package-level npm settings are configured after first publish, prefer requiring 2FA for publishing/settings and disallowing token-based publish for the manual-alpha phase.

Future automation decision:

- A publish workflow may be added only in a dedicated future PR.
- Any future GitHub Actions publish workflow must use npm Trusted Publishing / OIDC, not long-lived npm tokens.
- Any future publish workflow must use a protected GitHub environment or equivalent manual approval gate.
- Any future prerelease publish workflow must pass `--tag alpha` or another explicit prerelease tag.
- Any future stable publish workflow must be a separate explicit release process and may use `latest` only for a stable version.
- Any future automation PR must include a dry-run or pack verification path and must prove it cannot publish from ordinary pushes.

Provenance policy:

- Manual local alpha publishing does not claim CI provenance.
- If npm Trusted Publishing is introduced later, provenance must be enabled through the trusted-publisher flow and verified after publish.
- Do not add `--provenance` to local manual publish commands.

Official npm references for this policy:

- [npm publish docs](https://docs.npmjs.com/cli/v11/commands/npm-publish/) document that `npm publish` uses `latest` by default unless `--tag` is provided.
- [npm dist-tag docs](https://docs.npmjs.com/cli/v11/commands/npm-dist-tag/) recommend using tags such as prerelease tags for unstable versions.
- [npm Trusted Publishing docs](https://docs.npmjs.com/trusted-publishers/) describe OIDC-based publishing without long-lived npm tokens.
- [npm provenance docs](https://docs.npmjs.com/generating-provenance-statements/) note that Trusted Publishing generates provenance automatically.
- [npm 2FA docs](https://docs.npmjs.com/configuring-two-factor-authentication/) describe the current 2FA/token requirement for publishing.

## Package Names

The npm alpha uses Agent Trainer-specific package names because `@ignitionai/core` already exists on npm for another IgnitionAI package.
Expand Down Expand Up @@ -76,6 +112,8 @@ Confirm npm authentication:
npm whoami
```

Use an npm account that belongs to the IgnitionAI npm organization, has publish access to the packages below, and has 2FA enabled. Keep OTP entry interactive unless npm requires `--otp <code>` for the local environment.

Publish in dependency order:

```bash
Expand All @@ -91,7 +129,7 @@ cd ../adapter-ignitionrag && npm publish --access public --tag alpha
cd ../cli && npm publish --access public --tag alpha
```

Do not publish with the `latest` dist-tag.
Do not publish with the `latest` dist-tag. The explicit `--tag alpha` flag is mandatory because `npm publish` otherwise updates `latest` by default.

## Post-Publish Verification

Expand Down Expand Up @@ -131,4 +169,4 @@ bunx --package @ignitionai/agent-trainer-cli@alpha ignition-agent-trainer eval r

## Later Automation

GitHub Actions publishing with npm provenance is intentionally out of scope for `v0.1.0-alpha.1`. Add it only after the manual alpha has been consumed successfully from IgnitionRAG.
GitHub Actions publishing with npm provenance is intentionally disabled for `v0.1.0-alpha.1`. Add it only after the manual alpha has been consumed successfully from IgnitionRAG, and only through a dedicated PR that follows the policy above.
1 change: 1 addition & 0 deletions docs/PROJECT_AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,7 @@ If a package exists but is intentionally narrow, minimal or untested, it is part
| environment episodes | partial | `packages/environment` | No | Tested episode runner with max-step safety, seed forwarding, final state and metadata. |
| trajectory recorder | partial | `packages/rl/src/trajectory.ts` | No | Local state/action/reward/outcome records with deterministic summaries. |
| trajectory reports | partial | `packages/rl/src/episode-trajectory.ts` | No | Environment episodes can become trajectories, Markdown reports and offline policy records. |
| npm publishing policy | done | `docs/NPM_ALPHA_PUBLISHING.md` | Yes | Alpha publishing is manual-only with `--tag alpha`; future automation must use Trusted Publishing/OIDC and manual approval. |
| bandit prototype | prototype | `packages/rl/src/strategy-bandit.ts` | No | Clearly experimental, fixed arms only, no PPO. |
| contextual bandit prototype | prototype | `packages/rl/src/contextual-bandit.ts` | No | Deterministic fixed-feature scoring over task type, citation need, cost sensitivity, latency sensitivity and risk level. |
| offline policy evaluation | prototype | `packages/rl/src/offline-policy-evaluation.ts` | No | Deterministic replay over local records or observed trajectory steps; no live traffic path. |
Expand Down
Loading