Skip to content

Complete teaching corrections, candidate review and explicit adoption - #4

Merged
steph4n-gh merged 2 commits into
mainfrom
codex/teaching-lifecycle
Sep 22, 2026
Merged

steph4n-gh merged 2 commits into
mainfrom
codex/teaching-lifecycle

Conversation

@steph4n-gh

Copy link
Copy Markdown
Owner

What changes

Correcting a taught skill previously required manual candidate management, and editing the document demo's lessons could disable its working skill. This adds a shared TeachingSession workflow: record an explicit correction, assess a separate candidate, inspect the before/after results, and deliberately adopt the exact passing artifact. The approved skill remains available during edits and failed assessments.

  • Add the Python helper, system1 teach CLI and legacy reflex import parity for one text ChoiceField with uncertainty review enabled. Reject stale or altered evidence and incompatible schemas.
  • Connect the document workspace to the same lifecycle, including user-written lessons, persistent approved skills, inspectable errors/reviews/regressions and explicit adoption.
  • Synchronize README, guides, architecture, papers, examples and the workload evidence index. Keep the new workflow marked unreleased and distinguish empirical adoption checks, uncertainty estimates and independent quality evidence.

Existing prediction algorithms, compiler/engine APIs, .s1m format, required dependencies and package version remain unchanged.

Real-document evidence

The frozen BBC experiment separates initial lessons, calibration, adoption checks, feedback and a 400-article final holdout, with duplicate grouping before splitting and an ordinary-extra-lessons control. Adding 64 targeted feedback lessons reduced accepted mistakes from 16 to 8 while increasing reviews from 27 to 63. Raw accuracy rose from 92.25% to 93.25%, but its paired interval includes no gain.

All three candidates failed the unchanged zero-error development adoption gate; none was adopted. The evidence supports an error/review tradeoff, not general classifier superiority or successful adoption on real data. The protocol, full text-free outcomes, split manifest and exact measured source snapshot are retained in benchmarks/quality/document_workflow/.

Validation

  • Full local suite: 1,325 passed, 22 skipped; includes unchanged-artifact adoption, rejection, split isolation, disabled-review schema rejection and stale CLI status regressions.
  • Documentation: 95 Markdown files, 668 local links, 9 version labels checked; prose independently reviewed across the product and research documents.
  • Lockfile, wheel/source contents, strict Twine metadata and isolated wheel installation passed. The lifecycle and reflex alias work outside the checkout without SciPy.
  • Browser checks covered successful adoption, failed correction rejection, persistence/restart, own-text teaching and multilingual input.
  • Independent audit reproduced the BBC metrics from stored predictions and verified split/artifact/source hashes without rerunning the holdout.

Scope is a single-writer local workspace for one text-choice skill. Default adoption thresholds are demonstration defaults, not a production accuracy guarantee. This PR does not bump or publish a release.

@steph4n-gh
steph4n-gh merged commit 96cdc10 into main Sep 22, 2026
6 checks passed
@steph4n-gh steph4n-gh mentioned this pull request Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant