feat(course): publish quiz assignments verbatim and verify by read-back - #166
Conversation
Implementation plan for publishing quiz-envelope assignments from the CLI, plus a CONCEPTS.md entry for the Quiz Envelope. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cepted write A write the gateway accepted but whose read-back differs, or came back degraded, is neither a success nor a server failure: the module WAS modified. Kind "verify" names that outcome. It shares exit 1 per the main.go rule that kinds already distinguishable by name do not get their own code. Documented in help exit-codes, README, the context doc and the CLAUDE.md Failure Contract. (#165) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ad of an empty assignment.md tiptapToMarkdown matched no node of a quiz envelope and wrote an empty assignment.md, which a later course import published as the assignment. A non-doc content_json is now written pretty-printed to assignment.quiz.json and the stale counterpart file is removed on re-export, so export followed by import cannot destroy a quiz or trip import's both-files error. (#165, #59) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… fixtures Add internal/quiz, a dependency-free package that mirrors the Andamio app's quiz-envelope.ts: Recognize (isQuizContentEnvelope) classifies a content_json value as doc / quiz / other / not-an-object, Validate (validateQuizDefinition) collects every violated rule with the app's exact issue codes, and Summarize digests question count, pass threshold and question ids for --output json. Three CLI-additional checks the app does not enforce get their own codes: malformed-prompt, malformed-help, malformed-intro. Validate never panics on malformed elements and never stops at the first issue. testdata/quiz pins the rule set: 3 valid fixtures, 23 invalid fixtures each with a .issues sidecar labeled source: app (same code set the app emits; every validateQuizDefinition test case is mirrored) or source: cli-additional. SOURCE.md records the upstream commits the mirror was read from. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e assignment A module directory may hold assignment.quiz.json in place of assignment.md. The file is validated as a v1 quiz envelope through internal/quiz (every violated rule listed, no bypass), sent as content_json without re-encoding, and the module's existing assignment title and metadata are preserved by the existing merge. Both files present is a parse-time error before any request. --dry-run reports the quiz digest and --output json gains an additive assignment_quiz object; the schema golden now scans internal/quiz so those tags are pinned. fetchExistingModule surfaces the gateway's meta.warning so a degraded 206 read is never mistaken for a module without an assignment. (#165) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d verifies it by read-back andamio course import-assignment <course-id> <module-code> <file.json> replaces the hand-built curl a teacher needed to publish a quiz assignment. The file is validated before any request (every violated rule listed, no bypass), only the assignment key is sent, and the existing title, description, image and video URLs are carried because db-api overwrites every field it is not given. After the POST the module is re-fetched: a differing stored value or a degraded 206 read-back is kind verify (exit 1), a failed re-fetch keeps its own kind, and every message says the update was accepted. --dry-run / --show-payload follow the #61 convention; --output json emits the ImportAssignmentEnvelope pinned in the schema golden. Joins the expired-JWT fail-fast table as the eighth hand-rolled PreRunE. (#165) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ify kind README command list and Quiz assignments section, COURSE-LIFECYCLE step 9, CLAUDE.md and context-doc command tables, and the CHANGELOG Unreleased entries. The published-module statement rests on gateway and db-api source; the changelog says the live preprod check was not run. (#165) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…efusal, not_found mapping Review findings on #165: - The merged module list carries content only from db-api, so a db-api outage returns chain-only modules that fetchExistingModule treated as 'not found' and an Andamioscan outage tripped the meta.warning guard on a complete module. fetchExistingModule now returns errDegradedRead when the module is missing from a degraded list; import-assignment maps it to the R7a refusal before the write and to kind verify after, course import refuses to send (and never creates), and a found module is trusted regardless of the warning. Tests use the gateway's real chain-only shape. - course import with assignment.quiz.json on a module that has no assignment refuses with the R5 title-required error instead of storing an empty title. - A module missing from the list is kind not_found (exit 2) in import-assignment, not the generic kind. - CLAUDE.md exit-code table row order; docs name the FCB Fan Campus app as the validator the CLI mirrors; new text-summary test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review — findings only, no verdict recorded (self-authored PR)Conformance to andamio-cli#165: every acceptance criterion is met. CI green; Verified outside the diff:
Decisions before merge
Defects worth fixing in this PR
Minor
Follow-up candidates (not this PR)
Refuted during review: DeepEqual dropping explicit nulls on read-back (db-api stores |
…e, pre-create refusal, warning-aware degraded reads Addresses the devkit review on #166. Decisions taken as recommended: - Validate against the union of both apps' rules. andamio-app-v2 adds empty-prompt (trimmed), empty-option-label and empty-option-value; any course is viewable in app.andamio.io, so a quiz the CLI accepts must render there too. Fixtures labeled source: app-v2; SOURCE.md records ca30c2a (the file's commit). - Every failure after the accepted POST is kind verify, with the cause reachable through VerifyError.Unwrap. A reset connection after a 200 is not "the request never reached the service". Defects: - course import --create with assignment.quiz.json refuses before the create POST instead of leaving an empty module behind. - errDegradedRead only when the warning names the db side. An Andamioscan-only outage leaves the db list authoritative, so a missing module is not_found and --create keeps working. - Only ENOENT means "no assignment file"; unreadable files fail loudly. A zero-byte assignment.md never conflicts with a quiz file. - Export removes whichever assignment file it did not write, including both when the remote assignment is gone, and says so on stderr. - Export classifies through quiz.Classify (one classifier); a type-less object is preserved verbatim rather than flattened to an H1-only file. Residuals move from docs/residual-review-findings/ to todos/039 and todos/040 per CONTRIBUTING. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Addressed in Decisions — both taken as recommended
Defects
Minor
Left as follow-ups per your note: the unguarded list scans in |
…8bb3b8) 88bb3b8 landed only the fixtures and todos: its git add named the residual-findings directory that the same command had just deleted, so git rejected the pathspec and staged nothing else. This is the rest of that change — the union validator (empty-prompt, empty-option-label, empty-option-value), VerifyError.Err/Unwrap and verify for every post-write failure, the pre-create refusal, warning-aware degraded reads, read-error handling, quiz.Classify in export with stale-file cleanup outside the nil guard, and the doc wording. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Correction: |
The quiz-assignment work (#165, #166) adds a command, a module-directory file and an error kind — all additive, so this is a minor bump, not the patch it might look like from "one feature". A 1.0.1 tag advertising a new command would misread the policy line at the top of this file and leave the next genuine bugfix release without a natural number. Promoting the section is also what unblocks `scripts/release.sh`: its preflight hard-exits on a non-empty [Unreleased] with no versioned heading, which is exactly this state. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
Teachers can now publish a quiz assignment from the CLI.
andamio course import-assignment <course-id> <module-code> <file.json>takes the quiz envelope the FCB Fan Campus app grades client-side, validates it with the same rules that app enforces, sends only theassignmentkey of the module update, and then reads the module back to prove the stored value matches. Until now the only path was a hand-builtcurlwith the JWT copied out of~/.andamio/config.json, the same gap #62 closed for module creation.The export/import round trip also stops destroying quizzes.
course exportused to run the Markdown converter over the envelope, match nothing, and write an emptyassignment.md, which a latercourse importpublished as the assignment. A non-docassignment is now written verbatim toassignment.quiz.json,course import <dir>sends that file back verbatim, and export followed by import of a quiz module is a server-side no-op.Closes #165. Related: #59, #61, #62.
What a reviewer cannot see in the diff
processAssignmentUpdateoverwritestitle,description,image_urlandvideo_urlfrom the input unconditionally, so an omitted field is nulled, not preserved. The command fetches the module first and carries those fields;--title/--descriptionoverride. A module with no assignment yet requires--title, andcourse importwith a quiz file on such a module refuses for the same reason rather than storing an empty title.jsonb, so equality isreflect.DeepEqualover decoded values, never bytes. The teacher module list is a merged read that answers 206 withmeta.warningwhen one backend is down, and it carriescontentonly from db-api. A module missing from a degraded list is therefore a distinct outcome (errDegradedRead) from "not found": before the write it refuses to send, after the write it reportskind: verifynaming the warning, and a module found with content is trusted regardless of the warning.course importrefuses (and never creates) on the same signal.verify, exit 1. A write the gateway accepted but whose read-back did not confirm the stored value. Success would hide that the stored value is wrong;serverwould hide that the module was modified. Additive; documented inhelp exit-codes, README, the context doc and CLAUDE.md.internal/quizreuses the app's issue codes and control flow;testdata/quiz/pins 26 fixtures labeledapporcli-additional, andSOURCE.mdrecords the app commits they were copied from. Drift in the app is re-mirrored by hand, not detected by CI.assignment.mdandassignment.quiz.jsontogether fail at parse time; export removes the stale counterpart on a--forcere-export so it does not manufacture that state.Open decision for the author
andamio-app-v2's quiz validator enforces three rules the FCB app does not:
empty-prompt(trimmed),empty-option-label,empty-option-value. The issue names the FCB app as the reference, so the CLI does not enforce them and the docs now say "FCB Fan Campus app" rather than "the Andamio app". If quizzes published here must also render in app.andamio.io, those three rules should be added withsource: appfixtures. Recorded with the other review residuals indocs/residual-review-findings/feat-course-import-assignment-quiz.md.Validation
go test ./...is green (cmd/andamio ~88s). New coverage: 8 quiz-package tests over 26 golden fixtures; export writesassignment.quiz.jsonand removes the stale counterpart; the export→import round trip yields a payload deep-equal to the stored quiz;import-assignmentunit tests cover verbatim publish, metadata carry,--titleoverride, title-required, every rejection before any request, dry-run, read-back mismatch (content and metadata), degraded pre-fetch and read-back in the gateway's real chain-only shape, Andamioscan-only warning still publishing, module-not-in-list asnot_found, 503 read-back keeping its kind; binary-level tests pin the single-documentkind: verifyJSON, the dry-run envelope, stdout/stderr split and the expired-JWT fail-fast. Goldens refreshed for the new command, its flags and the new envelopes.Not run: the live preprod check against an
ON_CHAINmodule. The stored session targets mainnet and is expired. The "assignments are editable in any module status" statement rests on the gateway and db-api source (the aggregate update soft-skips only SLTs on non-DRAFT modules); CHANGELOG anddocs/COURSE-LIFECYCLE.mdsay so.Plan:
docs/plans/2026-09-05-001-feat-course-import-assignment-quiz-plan.md.New concepts
Write-then-read-back verification against an opaque store. When a service stores a value it does not interpret (here
content_jsonasjsonbbehind a gateway that re-serializes it), a 200 on the write proves only that the request was accepted, not that what was stored is what you meant. Reading the value back through the same read path the consumers use and comparing it turns an assumption into a check. Two things make it honest here: the comparison is structural (decode both sides,reflect.DeepEqual) because byte identity is not a property the store offers, and every failure after the POST says the write was applied, so a caller never mistakes "unconfirmed" for "nothing happened". The read-back must also know when it is blind: a degraded merged read that omits the module is reported as unconfirmed, not as a mismatch and not as success.Do not use it when the write returns the canonical stored representation itself, or when the read path is eventually consistent with the write path; then a read-back either duplicates the response or produces false mismatches.