diff --git a/CHANGELOG.md b/CHANGELOG.md index e17e47b..64aa38a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,22 @@ The format follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/ ## [Unreleased] +### Added + +- **`andamio course import-assignment `** — publishes a quiz assignment (a `{"type": "quiz", "version": 1, …}` envelope, the format the Andamio app grades client-side) as the module's `assignment.content_json`, verbatim, sending only the `assignment` key. Until now this was a hand-built `curl` against the module-update endpoint with the JWT copied out of `~/.andamio/config.json` — the gap #62 closed for module creation, reopened for quizzes. + + The envelope is validated before any request with the union of the rules the two Andamio apps enforce (fcb-fan-engagement-app and andamio-app-v2, each `src/lib/quiz/quiz-envelope.ts`, pinned by fixtures under `testdata/quiz/`) — any course on the gateway is viewable in app.andamio.io, so a quiz the CLI accepts renders in both. Every violated rule is listed, and there is no bypass flag. The existing assignment's title, description, image and video URLs are preserved unless `--title` / `--description` override them. After the update the module is re-fetched and the stored value deep-compared to the file, so the command proves the opaque-`jsonb` assumption on the live gateway rather than trusting it. `--dry-run` prints the summary (question count, pass threshold, question ids, title source) and sends nothing; `--show-payload` adds the payload; `--output json` emits `{course_id, module_code, module_status, assignment: {title, title_source, question_count, pass_threshold, question_ids}, verified}`. + + Works on published modules too: db-api's aggregate update soft-skips only SLTs on a non-DRAFT module and edits assignments in any status. That statement rests on the gateway and db-api source as of this change; the live preprod check on an `ON_CHAIN` module had not been run when it was written. (#165) + +- **`kind: verify` in the `--output json` error envelope** — a write the gateway accepted but whose read-back did not confirm the stored value: it differs from what was sent, the read-back was degraded (206), or the read-back request itself failed. Every failure after the accepted write classifies as `verify`; the cause stays inspectable through `errors.Unwrap`. It shares exit 1 with the other kinds that are already distinguishable by name. The distinction matters because the alternatives both mislead: success would hide that the stored value is wrong, `server` would hide that the module *was* modified. Emitted by `course import-assignment`. Additive — no existing kind changes. (#165) + +- **`assignment.quiz.json` in the module directory format.** `course import ` sends it verbatim as the assignment's `content_json` after validating it as a v1 quiz, preserving the existing title; a directory holding both `assignment.md` and `assignment.quiz.json` is refused before any request, and so is `--create` for a module that does not exist yet (a quiz file carries no title; publish it with `import-assignment --title` after the module exists). `--dry-run` reports `Assignment: quiz (N questions, threshold M)` and `--output json` gains an additive `assignment_quiz` summary object. (#165) + +### Fixed + +- **`course export` no longer destroys a quiz assignment.** It ran the Markdown converter over the envelope, matched no node type, and wrote an empty `assignment.md` — which a later `course import` of that directory published as the assignment, replacing the quiz with an empty text document. A non-`doc` assignment is now written verbatim to `assignment.quiz.json` and no `assignment.md` is produced, so export followed by import of a quiz module is a server-side no-op. Re-exporting into the same directory with `--force` removes whichever assignment file no longer matches the module, both when the assignment is gone, and says so on stderr. (#165, #59) + ## [1.0.0] - 2026-08-27 **Andamio CLI 1.0 is a developer tool for the people who author work and assess it: course Owners and Teachers, and project Managers.** diff --git a/CLAUDE.md b/CLAUDE.md index c182995..e34faf3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -100,6 +100,7 @@ Export and import are the two complex commands. They follow a different pattern: 7. **SLT locking** — import checks module status; skips sending SLTs for non-DRAFT modules to avoid `SLT_LOCKED` errors. 8. **Tiptap node types** — standalone images use `imageBlock` (with `width: "600"`, `align: "center"` attrs), not `image`. Matches app's `markdown-to-tiptap.ts`. 9. **Goldmark TextBlock** — tight list items use `ast.TextBlock`, not `ast.Paragraph`. Both are handled identically in the converter. +10. **Quiz assignments** — an assignment whose `content_json.type` is not `doc` (a quiz envelope, see CONCEPTS.md) is exported verbatim to `assignment.quiz.json` with no `assignment.md`, and import sends `assignment.quiz.json` back verbatim after validating it as a v1 quiz through `internal/quiz`. Both files present is a parse-time error. The validator enforces the union of both apps' `validateQuizDefinition` rules (fcb-fan-engagement-app and andamio-app-v2); `testdata/quiz/SOURCE.md` records the commits the fixtures were copied from, and a rule change in either app is re-mirrored by hand. ### Auth Flow @@ -108,7 +109,7 @@ Three auth slots coexist in config: - **User JWT** (`user login`) — browser-based wallet signing flow: starts ephemeral local HTTP server, opens browser to `{appURL}/auth/cli?redirect_uri=...&state=...`, user connects Cardano wallet and signs nonce, receives JWT via callback. CSRF protection via random state parameter. Required for edit operations on course/project commands. Headless variant: `user login --skey --alias --address`. - **Developer JWT** (`dev login`) — supports two modes. **Browser mode** (default, `dev login` with no args) opens `{appURL}/auth/dev-cli` and waits for a wallet-signed nonce from Eternl/Lace/Nami via an ephemeral localhost callback — the typical developer journey, since browser wallets don't expose `.skey` files. The browser-flow callback is **POST + JSON** (`Content-Type: application/json`) per `andamio-app-v2#699`'s `DevCliSuccessPayload` / `DevCliErrorPayload`, with an `OPTIONS` preflight serving CORS + `Access-Control-Allow-Private-Network: true` so Chrome's PNA spec permits the HTTPS-origin → 127.0.0.1 POST. The listener enforces an exact-string `Origin` allow-list (derived from `cfg.BaseURL` via `.api.`→`.app.` swap) on browser-originated POSTs; loopback diagnostics without an `Origin` header are still accepted. The user-login browser flow (`/auth/cli`) deliberately stays on **GET + query params** (lower-sensitivity payload, no 30-day refresh token); the two flows have intentionally different wire formats. **Headless mode** (`dev login --skey --alias --address`) signs locally for CI/CD, ops, and devkit. Both call andamio-api's CIP-30 signature-verified login endpoints (#410). Two-step flow: `POST /v2/auth/developer/login/session` opens a 5-min session keyed to `(alias, wallet_address)` and returns a nonce; the CLI signs the nonce locally with `internal/cardano.SignMessage`; `POST /v2/auth/developer/login/complete` submits the signature and receives a 60-minute RS256 JWT plus a 30-day single-use rotation refresh token. The dev JWT is required for `/v2/keys`, `/api/v2/apikey/developer/*`, and other developer-portal endpoints — the gateway's `developerJWTAuth` middleware does not accept wallet/user JWTs and vice versa. These surfaces are **dual-credential**: the gateway's `V2AuthMiddleware` requires `X-API-Key` and the inner `developerJWTAuth` requires `Authorization: Bearer `. The CLI's `devKeysClient` helper (`cmd/andamio/dev_keys.go`) is the shared routing for any dual-credential dev-portal surface — preserves `APIKey`, promotes `DevJWT` into the JWT slot, both headers ride on the wire. `dev keys` and `apikey usage`/`profile` both route through it; new dev-portal commands should too. Distinct config slot (`dev_jwt` + `dev_refresh_token`) so the two JWTs don't clobber each other. `dev refresh` rotates without re-signing (uses the refresh token); a 401 from refresh clears the dev slot and instructs re-login. `dev logout` clears the entire dev slot whenever **either** `dev_jwt` **or** `dev_refresh_token` is persisted (the durable 30-day refresh token gets cleared even when the 60-min JWT is empty). Override at runtime via `ANDAMIO_DEV_JWT` and/or `ANDAMIO_DEV_REFRESH_TOKEN` env vars (parallel to `ANDAMIO_JWT` for the user slot — the refresh-token override is the path for ephemeral CI/CD agents that want to rotate without committing tokens to the image). **Ephemeral by design:** env-sourced credentials (`ANDAMIO_JWT` / `ANDAMIO_DEV_JWT` / `ANDAMIO_DEV_REFRESH_TOKEN`) are NOT persisted to disk on `Save` — `Load` snapshots the env values and `Save` strips fields whose current value still matches the snapshot. Rotation works normally: `dev refresh` mutates the in-memory token to the gateway-rotated value (which differs from the snapshot) and that new value IS persisted, so subsequent CLI commands in the same job pick it up. The legacy lookup-only `/v2/auth/developer/account/login` is intentionally not used — it returns 410 Gone behind the gateway's kill-switch flag and does not prove wallet ownership. -**Local JWT expiry handling (#134).** `internal/config/jwt.go` decodes a token's `exp` claim locally (payload base64 only, no signature verification — the decoded value drives a send/don't-send decision; the gateway stays the authority) with a conservative 30s skew: expired means `now >= exp - 30s`, so the CLI never sends a token the gateway might already reject. Four enforcement points, in request order: (1) `client.New` drops a locally-expired **user-slot** JWT from its own field snapshot (config is never mutated, so no `Save` can persist the clear) and prints one stderr warning per process — resettable `warnOnce`, emitted in every output mode, env-aware wording (`ANDAMIO_JWT` vs `user login`); (2) `requireUserAuth` (helpers.go) fails fast with exit 3 / `kind: auth` + expiry timestamp on all JWT-required commands — `jwtAuthPreRunE` parents AND the seven hand-rolled PreRunEs (`course export/import/import-all/create-module`, `tx build/register/run`); (3) `devKeysClient` fail-fasts on an expired **dev** JWT with a `dev refresh` hint *before* promoting it into the shared UserJWT slot — ordering is load-bearing: it guarantees the client-level drop can never silently strip a dev JWT on a dual-credential surface (the 0.12.x regression class); (4) either-auth course reads route teacher-vs-user endpoints via `HasFreshUserAuth`, not `HasUserAuth`, so an expired JWT + API key lands on the user endpoint and succeeds. **Fail open on undecodable tokens everywhere** — non-JWT strings (incl. the `"test-jwt"` test fixtures) are sent as-is, never treated as expired. Login flows are self-healing: the browser guard treats expired as unauthenticated, and headless login builds its client from a cfg copy with `UserJWT` blanked (login never needs prior user auth — this covers even tokens the CLI cannot decode). Headless login persists `jwt_expires_at` from the decoded `exp`; `user status` falls back to decoding `exp` when the stored field is empty and computes `session_expired` with the same skew predicate as enforcement (probe and enforcement must agree). +**Local JWT expiry handling (#134).** `internal/config/jwt.go` decodes a token's `exp` claim locally (payload base64 only, no signature verification — the decoded value drives a send/don't-send decision; the gateway stays the authority) with a conservative 30s skew: expired means `now >= exp - 30s`, so the CLI never sends a token the gateway might already reject. Four enforcement points, in request order: (1) `client.New` drops a locally-expired **user-slot** JWT from its own field snapshot (config is never mutated, so no `Save` can persist the clear) and prints one stderr warning per process — resettable `warnOnce`, emitted in every output mode, env-aware wording (`ANDAMIO_JWT` vs `user login`); (2) `requireUserAuth` (helpers.go) fails fast with exit 3 / `kind: auth` + expiry timestamp on all JWT-required commands — `jwtAuthPreRunE` parents AND the eight hand-rolled PreRunEs (`course export/import/import-all/import-assignment/create-module`, `tx build/register/run`); (3) `devKeysClient` fail-fasts on an expired **dev** JWT with a `dev refresh` hint *before* promoting it into the shared UserJWT slot — ordering is load-bearing: it guarantees the client-level drop can never silently strip a dev JWT on a dual-credential surface (the 0.12.x regression class); (4) either-auth course reads route teacher-vs-user endpoints via `HasFreshUserAuth`, not `HasUserAuth`, so an expired JWT + API key lands on the user endpoint and succeeds. **Fail open on undecodable tokens everywhere** — non-JWT strings (incl. the `"test-jwt"` test fixtures) are sent as-is, never treated as expired. Login flows are self-healing: the browser guard treats expired as unauthenticated, and headless login builds its client from a cfg copy with `UserJWT` blanked (login never needs prior user auth — this covers even tokens the CLI cannot decode). Headless login persists `jwt_expires_at` from the decoded `exp`; `user status` falls back to decoding `exp` when the stored field is empty and computes `session_expired` with the same skew predicate as enforcement (probe and enforcement must agree). The app URL is derived from the API URL by replacing `.api.` with `.app.` in the hostname. @@ -141,6 +142,7 @@ Every failure carries an exit code **and**, under `--output json`, a `kind` fiel |------|--------|------| | 0 | — | Success, including an empty but valid result set | | 1 | `error` / `server` / `backpressure` / `canceled` | Unexpected, 5xx, retry-later, interrupted | +| 1 | `verify` | A write was accepted but the read-back did not confirm the stored value (differs, or degraded 206). Emitted by `course import-assignment`. Shares exit 1 per the main.go rule; the module WAS modified | | 2 | `not_found` | 404 | | 3 | `auth` | No credentials, or 401/403 | | 4 | `removed_command` | Retired in 1.0 | @@ -211,6 +213,7 @@ Exit codes 0–3 predate 1.0 and are fixed. `conflict` moved from 1 to 6 in 1.0. | `course teacher commitments` | `/v2/course/teacher/assignment-commitments/list` | jwt | List pending reviews. `--course-id` | | `course credential verify-hash ` | `/api/v2/course/user/modules/{id}` | either | Verify credential hashes match computed SLT hashes | | `course credential compute-hash` | local | none | Compute SLT hash from `--slt` flags or `--file` (outline.md). No auth required | +| `course import-assignment ` | `/v2/course/teacher/course-modules/list` + `/v2/course/teacher/course-module/update` | jwt | Publish a quiz envelope (`{"type":"quiz","version":1,…}`) verbatim as the module's `assignment.content_json`, sending only the `assignment` key. Validates before any request (no bypass flag), preserves existing title/description/image_url/video_url (`--title`/`--description` override), then re-fetches and deep-compares; any failure after the accepted write (mismatch, degraded read-back, failed re-fetch) is `kind: verify`. `--course`, `--dry-run`, `--show-payload`. Assignments are editable in any module status; only SLTs lock | ### project — Project data | Command | Endpoint | Auth | Description | diff --git a/CONCEPTS.md b/CONCEPTS.md index df57bac..79eb1cf 100644 --- a/CONCEPTS.md +++ b/CONCEPTS.md @@ -22,6 +22,11 @@ The content-derived identity of a Course Module: a digest computed over the orde Both the CLI and the chain compute it independently from the same content, which is what makes it a linkage key rather than a checksum: two records agree that they describe the same module precisely when their SLT Hashes match. A mismatch is therefore not corruption to repair but a statement that these are different modules. +### Quiz Envelope +An assignment whose `content_json` is a `{"type": "quiz", "version": 1, ...}` object instead of a Tiptap `doc`. The gateway and db-api store it as opaque JSON; only the Andamio app's render layer interprets it, grading client-side and storing a self-contained evidence snapshot on commit. Its validity rules are owned by the app (`src/lib/quiz/quiz-envelope.ts` in fcb-fan-engagement-app); the CLI mirrors them so an envelope it publishes is one the app can render. + +On disk a quiz assignment is `assignment.quiz.json`, never `assignment.md`: converting the envelope to Markdown loses it, so export and import carry it verbatim. + ## Project ### Task diff --git a/README.md b/README.md index fa74e5b..c436aac 100644 --- a/README.md +++ b/README.md @@ -145,6 +145,7 @@ Both come from the same classification, so they never disagree. |------|--------|------| | 0 | — | Success, **including an empty but valid result set** | | 1 | `error` / `server` / `backpressure` / `canceled` | Unexpected, 5xx, retry-later, or interrupted | +| 1 | `verify` | The update was accepted, but the read-back did not confirm the stored value (`course import-assignment`) — inspect, don't retry blindly | | 2 | `not_found` | Resource doesn't exist | | 3 | `auth` | No credentials, or 401 / 403 | | 4 | `removed_command` | Command was retired in 1.0 | @@ -201,6 +202,7 @@ Author: - `course export ` — Export module to local directory - `course import --course-id ` — Import module from local directory +- `course import-assignment ` — Publish a quiz assignment (JSON envelope) and verify it by read-back - `course owner create|update|register` — Create and register a course - `course owner teachers --course-id --alias --skey --add ` — Manage teachers (on-chain transaction) - `course teacher register-module|publish-module|update-module-status` — Module lifecycle @@ -349,6 +351,30 @@ A blockchain is a distributed ledger... **introduction.md** / **assignment.md** — Same format as lessons (H1 = title). +**assignment.quiz.json** — A quiz assignment instead of `assignment.md`. The file is the quiz envelope (`{"type": "quiz", "version": 1, "passThreshold": N, "questions": [...]}`) exactly as the Andamio app stores and grades it; format contract in the app's `docs/quiz-content-format.md`. Export writes this file for a quiz module and no `assignment.md`; import validates it and sends it back verbatim, keeping the module's existing assignment title. A directory holding both `assignment.md` and `assignment.quiz.json` is refused. + +### Quiz assignments + +Publish a quiz envelope directly, without a module directory: + +```bash +# Validate, preview the summary, send nothing +andamio course import-assignment 101 quiz.json --dry-run + +# Publish, then read back and verify. Title/description come from the existing +# assignment unless overridden; a module with no assignment yet needs --title. +andamio course import-assignment 101 quiz.json --title "Module Quiz" + +# Scripting +andamio course import-assignment 101 quiz.json --output json +# {"course_id":"…","module_code":"101","module_status":"ON_CHAIN", +# "assignment":{"title":"Module Quiz","title_source":"flag","question_count":5, +# "pass_threshold":4,"question_ids":["q1","q2","q3","q4","q5"]}, +# "verified":true} +``` + +The command validates the envelope with the union of the rules the two Andamio apps enforce — fcb-fan-engagement-app and app.andamio.io, since any course on the gateway is viewable in the latter (`type`, `version: 1`, non-empty `questions`, unique ids, non-blank prompts, at least two options with unique non-empty values and non-blank labels, `correctValue` matching one option, `passThreshold` in `1..len(questions)`, an `intro` that is a Tiptap doc if present). Every violated rule is listed; there is no bypass flag. Only the `assignment` key is sent, so lessons, SLTs, and the introduction are untouched. Assignments are editable in any module status — only SLTs lock after DRAFT. After the update the module is re-fetched and the stored `content_json` is deep-compared to the file; a mismatch, a degraded read-back, or a read-back request that fails for any reason exits 1 with `kind: verify`, which means the update was applied but should be inspected. + ### Image Handling **Exported images:** Downloaded to `assets/` with a `.image-manifest.json` mapping filenames to their original CDN URLs. On re-import, the manifest restores the original URLs — no re-upload needed. diff --git a/cmd/andamio/course_export.go b/cmd/andamio/course_export.go index 399f7de..1fe85e5 100644 --- a/cmd/andamio/course_export.go +++ b/cmd/andamio/course_export.go @@ -18,6 +18,7 @@ import ( "github.com/Andamio-Platform/andamio-cli/internal/client" "github.com/Andamio-Platform/andamio-cli/internal/config" "github.com/Andamio-Platform/andamio-cli/internal/output" + "github.com/Andamio-Platform/andamio-cli/internal/quiz" "github.com/spf13/cobra" ) @@ -32,9 +33,16 @@ This creates a directory structure that can be edited locally and re-imported: ├── introduction.md # Module introduction (if present) ├── lesson-1.md # Lesson for SLT 1 ├── lesson-N.md # Lesson for SLT N - ├── assignment.md # Module assignment (if present) + ├── assignment.md # Module assignment (Tiptap doc, if present) + ├── assignment.quiz.json # Module assignment when it is a quiz envelope └── assets/ # Downloaded images +A quiz assignment (content_json type "quiz") is written verbatim to +assignment.quiz.json and no assignment.md is produced — converting it to +Markdown would lose it. Import sends assignment.quiz.json back verbatim +after validating it as a v1 quiz; a non-quiz or non-v1 file in that slot +blocks re-import of the whole directory until it is a valid quiz. + The course can be specified by ID (first arg) or by name (--course flag): andamio course export andamio course export --course "Intro to Cardano" @@ -395,16 +403,52 @@ func writeCompiledModule(outputDir string, data *ModuleData) (*WriteResult, erro result.Files = append(result.Files, "introduction.md") } - // Write assignment.md if present + // Write the assignment if present. A Tiptap doc becomes assignment.md; + // anything else — a quiz envelope today, a type-less or unknown object + // tomorrow — is preserved verbatim as assignment.quiz.json, because + // tiptapToMarkdown matches none of its nodes and would write an empty + // assignment.md that a later import publishes as the assignment (#165, + // #59). The shape decision is quiz.Classify, the same classifier import + // uses, so the two sides cannot drift. Whatever was NOT written this + // export is removed: a stale counterpart would trip import's + // both-files-present error, and when the remote assignment is gone both + // files go, or the next import would republish it. + var wroteAssignment string if data.Assignment != nil { - assignPath := filepath.Join(absDir, "assignment.md") - assignContent, urls := convertContentToMarkdown(data.Assignment) - imageURLs = append(imageURLs, urls...) + contentJSON, title := unwrapContent(data.Assignment) + if contentJSON != nil && quiz.Classify(contentJSON) != quiz.Doc { + pretty, err := json.MarshalIndent(contentJSON, "", " ") + if err != nil { + return nil, fmt.Errorf("failed to encode assignment.quiz.json: %w", err) + } + if err := writeFileAtomic(filepath.Join(absDir, "assignment.quiz.json"), append(pretty, '\n')); err != nil { + return nil, fmt.Errorf("failed to write assignment.quiz.json: %w", err) + } + wroteAssignment = "assignment.quiz.json" + } else { + assignContent, urls := renderContentMarkdown(contentJSON, title) + imageURLs = append(imageURLs, urls...) - if err := writeFileAtomic(assignPath, []byte(assignContent)); err != nil { - return nil, fmt.Errorf("failed to write assignment.md: %w", err) + if err := writeFileAtomic(filepath.Join(absDir, "assignment.md"), []byte(assignContent)); err != nil { + return nil, fmt.Errorf("failed to write assignment.md: %w", err) + } + wroteAssignment = "assignment.md" + } + result.Files = append(result.Files, wroteAssignment) + } + for _, name := range []string{"assignment.md", "assignment.quiz.json"} { + if name == wroteAssignment { + continue + } + removed, err := removeIfExists(filepath.Join(absDir, name)) + if err != nil { + return nil, err + } + if removed && output.GetFormat() != output.FormatJSON { + // Only reachable under --force. The file may be a local draft, so + // say what happened rather than folding it into "overwriting". + fmt.Fprintf(os.Stderr, "Removed %s: it no longer matches the module's assignment\n", name) } - result.Files = append(result.Files, "assignment.md") } // Download images if any @@ -522,29 +566,52 @@ func sanitizeTitle(s string) string { return strings.TrimSpace(s) } -func convertContentToMarkdown(resp map[string]interface{}) (string, []string) { - // Handle introduction/assignment response structure - // API returns: { "data": { "content": { "content_json": {...}, "title": "..." } } } - var contentJSON map[string]interface{} - var title string - - if data, ok := resp["data"].(map[string]interface{}); ok { - if content, ok := data["content"].(map[string]interface{}); ok { - if cj, ok := content["content_json"].(map[string]interface{}); ok { - contentJSON = cj - } - if t, ok := content["title"].(string); ok { - title = t - } +// unwrapContent pulls content_json and title out of the wrapped +// introduction/assignment shape fetchModuleData builds: +// { "data": { "content": { "content_json": {...}, "title": "..." } } }. +// Falls back to a direct content_json on data in case the shape flattens. +func unwrapContent(resp map[string]interface{}) (contentJSON map[string]interface{}, title string) { + data, ok := resp["data"].(map[string]interface{}) + if !ok { + return nil, "" + } + if content, ok := data["content"].(map[string]interface{}); ok { + if cj, ok := content["content_json"].(map[string]interface{}); ok { + contentJSON = cj } - // Fallback: try direct content_json on data (in case API changes) - if contentJSON == nil { - if cj, ok := data["content_json"].(map[string]interface{}); ok { - contentJSON = cj - } + if t, ok := content["title"].(string); ok { + title = t + } + } + if contentJSON == nil { + if cj, ok := data["content_json"].(map[string]interface{}); ok { + contentJSON = cj } } + return contentJSON, title +} + +// removeIfExists deletes path when present and tolerates its absence. It +// reports whether a file was actually removed so the caller can say so. +func removeIfExists(path string) (bool, error) { + err := os.Remove(path) + if err == nil { + return true, nil + } + if os.IsNotExist(err) { + return false, nil + } + return false, fmt.Errorf("failed to remove stale %s: %w", filepath.Base(path), err) +} + +func convertContentToMarkdown(resp map[string]interface{}) (string, []string) { + contentJSON, title := unwrapContent(resp) + return renderContentMarkdown(contentJSON, title) +} +// renderContentMarkdown converts an already-unwrapped content_json to Markdown +// with the title as an H1. A nil contentJSON renders nothing. +func renderContentMarkdown(contentJSON map[string]interface{}, title string) (string, []string) { if contentJSON == nil { return "", nil } diff --git a/cmd/andamio/course_export_quiz_test.go b/cmd/andamio/course_export_quiz_test.go new file mode 100644 index 0000000..7d6f088 --- /dev/null +++ b/cmd/andamio/course_export_quiz_test.go @@ -0,0 +1,224 @@ +package main + +import ( + "encoding/json" + "os" + "path/filepath" + "reflect" + "slices" + "strings" + "testing" +) + +// quizEnvelope is a valid v1 quiz used across the export/import round-trip +// tests. Numbers are float64 because that is what encoding/json decodes into, +// and the comparisons below are structural, never byte-for-byte (KTD3). +func quizEnvelope() map[string]interface{} { + return map[string]interface{}{ + "type": "quiz", + "version": float64(1), + "passThreshold": float64(2), + "questions": []interface{}{ + map[string]interface{}{ + "id": "q1", + "prompt": "What is a wallet?", + "options": []interface{}{ + map[string]interface{}{"value": "a", "label": "A key manager"}, + map[string]interface{}{"value": "b", "label": "A bank account"}, + }, + "correctValue": "a", + }, + map[string]interface{}{ + "id": "q2", + "prompt": "What is a credential?", + "options": []interface{}{ + map[string]interface{}{"value": "a", "label": "A sticker"}, + map[string]interface{}{"value": "b", "label": "An on-chain record"}, + }, + "correctValue": "b", + }, + }, + } +} + +// wrapAssignment mirrors the shape fetchModuleData hands writeCompiledModule. +func wrapAssignment(contentJSON map[string]interface{}, title string) map[string]interface{} { + return map[string]interface{}{ + "data": map[string]interface{}{ + "content": map[string]interface{}{ + "content_json": contentJSON, + "title": title, + }, + }, + } +} + +func exportModuleData(assignment map[string]interface{}) *ModuleData { + return &ModuleData{ + CourseID: "course-1", + CourseSlug: "course", + ModuleCode: "101", + Title: "Module 101", + Status: "DRAFT", + SLTs: []SLTData{{Index: 1, Text: "Do a thing"}}, + Assignment: assignment, + } +} + +func fileExists(t *testing.T, path string) bool { + t.Helper() + _, err := os.Stat(path) + if err == nil { + return true + } + if os.IsNotExist(err) { + return false + } + t.Fatalf("stat %s: %v", path, err) + return false +} + +// A non-doc assignment is preserved verbatim on disk as assignment.quiz.json; +// converting it to Markdown matched no node type and produced an empty +// assignment.md that a later import would publish as the assignment (#165). +func TestWriteCompiledModule_QuizAssignmentWritesQuizJSON(t *testing.T) { + dir := t.TempDir() + quiz := quizEnvelope() + + result, err := writeCompiledModule(dir, exportModuleData(wrapAssignment(quiz, "Module Quiz"))) + if err != nil { + t.Fatalf("writeCompiledModule: %v", err) + } + + quizPath := filepath.Join(dir, "assignment.quiz.json") + if !fileExists(t, quizPath) { + t.Fatal("assignment.quiz.json was not written") + } + if fileExists(t, filepath.Join(dir, "assignment.md")) { + t.Error("assignment.md must not be written for a quiz assignment") + } + if !slices.Contains(result.Files, "assignment.quiz.json") { + t.Errorf("Files = %v, want assignment.quiz.json listed", result.Files) + } + if slices.Contains(result.Files, "assignment.md") { + t.Errorf("Files = %v, must not list assignment.md", result.Files) + } + + raw, err := os.ReadFile(quizPath) + if err != nil { + t.Fatal(err) + } + var got map[string]interface{} + if err := json.Unmarshal(raw, &got); err != nil { + t.Fatalf("assignment.quiz.json is not JSON: %v", err) + } + if !reflect.DeepEqual(got, quiz) { + t.Errorf("assignment.quiz.json content differs from the stored envelope\n got: %v\nwant: %v", got, quiz) + } +} + +func TestWriteCompiledModule_DocAssignmentUnchanged(t *testing.T) { + dir := t.TempDir() + doc := map[string]interface{}{ + "type": "doc", + "content": []interface{}{ + map[string]interface{}{ + "type": "paragraph", + "content": []interface{}{map[string]interface{}{"type": "text", "text": "Write an essay."}}, + }, + }, + } + + result, err := writeCompiledModule(dir, exportModuleData(wrapAssignment(doc, "Essay"))) + if err != nil { + t.Fatalf("writeCompiledModule: %v", err) + } + if fileExists(t, filepath.Join(dir, "assignment.quiz.json")) { + t.Error("assignment.quiz.json must not be written for a doc assignment") + } + md, err := os.ReadFile(filepath.Join(dir, "assignment.md")) + if err != nil { + t.Fatalf("assignment.md missing: %v", err) + } + if want := "# Essay\n\nWrite an essay."; strings.TrimSpace(string(md)) != want { + t.Errorf("assignment.md = %q, want H1 title then body", string(md)) + } + if !slices.Contains(result.Files, "assignment.md") { + t.Errorf("Files = %v, want assignment.md listed", result.Files) + } +} + +func TestWriteCompiledModule_NoAssignmentWritesNeither(t *testing.T) { + dir := t.TempDir() + if _, err := writeCompiledModule(dir, exportModuleData(nil)); err != nil { + t.Fatalf("writeCompiledModule: %v", err) + } + if fileExists(t, filepath.Join(dir, "assignment.md")) || fileExists(t, filepath.Join(dir, "assignment.quiz.json")) { + t.Error("no assignment file expected when the module has no assignment") + } +} + +// When the remote assignment is gone, both stale assignment files go too — +// otherwise the next import would republish the deleted assignment. +func TestWriteCompiledModule_NoAssignmentRemovesStaleFiles(t *testing.T) { + dir := t.TempDir() + for _, name := range []string{"assignment.md", "assignment.quiz.json"} { + if err := os.WriteFile(filepath.Join(dir, name), []byte("stale"), 0644); err != nil { + t.Fatal(err) + } + } + if _, err := writeCompiledModule(dir, exportModuleData(nil)); err != nil { + t.Fatal(err) + } + for _, name := range []string{"assignment.md", "assignment.quiz.json"} { + if fileExists(t, filepath.Join(dir, name)) { + t.Errorf("stale %s survived an export of a module with no assignment", name) + } + } +} + +// A type-less object is not a Tiptap doc: exporting it as Markdown would be +// lossy (an H1-only file), so it takes the verbatim path like any non-doc. +func TestWriteCompiledModule_TypelessObjectExportsVerbatim(t *testing.T) { + dir := t.TempDir() + if _, err := writeCompiledModule(dir, exportModuleData(wrapAssignment(map[string]interface{}{"foo": "bar"}, "Odd"))); err != nil { + t.Fatal(err) + } + if !fileExists(t, filepath.Join(dir, "assignment.quiz.json")) || fileExists(t, filepath.Join(dir, "assignment.md")) { + t.Error("a type-less content_json must be preserved verbatim, not flattened to Markdown") + } +} + +// A re-export into the same directory (only reachable under --force) must not +// leave the previous assignment file behind: both files present is the R11 +// ambiguity error on the next import, and export would be the tool that +// produced it. +func TestWriteCompiledModule_ReexportRemovesStaleCounterpart(t *testing.T) { + dir := t.TempDir() + stale := filepath.Join(dir, "assignment.md") + if err := os.WriteFile(stale, []byte("# Old\n\nstale"), 0644); err != nil { + t.Fatal(err) + } + + if _, err := writeCompiledModule(dir, exportModuleData(wrapAssignment(quizEnvelope(), "Quiz"))); err != nil { + t.Fatalf("writeCompiledModule: %v", err) + } + if fileExists(t, stale) { + t.Error("stale assignment.md survived a quiz export") + } + if !fileExists(t, filepath.Join(dir, "assignment.quiz.json")) { + t.Fatal("assignment.quiz.json was not written") + } + + // Reverse transition: quiz back to doc removes the stale quiz file. + doc := map[string]interface{}{"type": "doc", "content": []interface{}{}} + if _, err := writeCompiledModule(dir, exportModuleData(wrapAssignment(doc, "Essay"))); err != nil { + t.Fatalf("writeCompiledModule: %v", err) + } + if fileExists(t, filepath.Join(dir, "assignment.quiz.json")) { + t.Error("stale assignment.quiz.json survived a doc export") + } + if !fileExists(t, stale) { + t.Error("assignment.md was not written on the reverse transition") + } +} diff --git a/cmd/andamio/course_import.go b/cmd/andamio/course_import.go index a1e1a61..7493469 100644 --- a/cmd/andamio/course_import.go +++ b/cmd/andamio/course_import.go @@ -22,6 +22,7 @@ import ( "github.com/Andamio-Platform/andamio-cli/internal/client" "github.com/Andamio-Platform/andamio-cli/internal/config" "github.com/Andamio-Platform/andamio-cli/internal/output" + "github.com/Andamio-Platform/andamio-cli/internal/quiz" "github.com/adrg/frontmatter" "github.com/spf13/cobra" "github.com/yuin/goldmark" @@ -36,6 +37,14 @@ var ( sltLineRe = regexp.MustCompile(`^\d+[\.\)]\s+(.+)$`) lessonNumRe = regexp.MustCompile(`lesson-(\d+)\.md`) errModuleNotFound = errors.New("module not found") + // errDegradedRead is returned by fetchExistingModule when the module was + // not in the list AND the gateway flagged the list as degraded (206 with + // meta.warning). The merged list carries `content` only from db-api, so + // on a db-api outage every module comes back chain-only with no content + // and looks absent. That is not "not found": a write would null the + // module's metadata and --create would try to duplicate it. Callers must + // refuse to send. Deliberately does NOT wrap errModuleNotFound. + errDegradedRead = errors.New("module list was degraded") ) func init() { @@ -57,7 +66,9 @@ The directory should contain: - outline.md (with YAML frontmatter: title, code) - lesson-N.md files (one per SLT) - introduction.md (optional) - - assignment.md (optional) + - assignment.md (optional) — or assignment.quiz.json for a quiz assignment, + never both. A quiz file is validated as a v1 quiz envelope and sent + verbatim; the module's existing assignment title is preserved. Examples: andamio course import ./compiled/my-course/101 --course-id abc123 @@ -76,14 +87,17 @@ Requires user authentication via 'andamio user login'.`, // ImportData holds the parsed module content for import type ImportData struct { - Title string - ModuleCode string - SLTs []string - Lessons []LessonImport - Introduction *ContentSection - Assignment *ContentSection - ImageWarnings []string - ImageManifest map[string]string // filename → original URL from .image-manifest.json + Title string + ModuleCode string + SLTs []string + Lessons []LessonImport + Introduction *ContentSection + Assignment *ContentSection + // AssignmentQuiz is set when the assignment came from assignment.quiz.json: + // the validated envelope's digest for the dry-run summary and the result. + AssignmentQuiz *quiz.Summary + ImageWarnings []string + ImageManifest map[string]string // filename → original URL from .image-manifest.json } // LessonImport holds a single lesson's data @@ -93,10 +107,17 @@ type LessonImport struct { TiptapJSON map[string]interface{} } -// ContentSection holds parsed content with title extracted from H1 +// ContentSection holds parsed content with title extracted from H1. +// +// Exactly one of TiptapJSON and RawJSON is set. TiptapJSON is the converted +// Markdown; RawJSON is the bytes of assignment.quiz.json, sent as +// content_json without re-encoding so the envelope reaches the gateway as +// the author wrote it (byte-for-byte modulo encoding/json's compaction — +// equality with the stored value is structural, see KTD3 in the #165 plan). type ContentSection struct { Title string TiptapJSON map[string]interface{} + RawJSON json.RawMessage } // OutlineFrontmatter is the YAML frontmatter structure in outline.md @@ -107,16 +128,19 @@ type OutlineFrontmatter struct { // ImportResult holds the result of an import operation for structured output type ImportResult struct { - CourseID string `json:"course_id"` - ModuleCode string `json:"module_code"` - Title string `json:"title"` - ModuleStatus string `json:"module_status"` - SLTsLocked bool `json:"slts_locked"` - SLTCount int `json:"slt_count"` - SltHash string `json:"slt_hash,omitempty"` - LessonCount int `json:"lesson_count"` - HasIntro bool `json:"has_introduction"` - HasAssignment bool `json:"has_assignment"` + CourseID string `json:"course_id"` + ModuleCode string `json:"module_code"` + Title string `json:"title"` + ModuleStatus string `json:"module_status"` + SLTsLocked bool `json:"slts_locked"` + SLTCount int `json:"slt_count"` + SltHash string `json:"slt_hash,omitempty"` + LessonCount int `json:"lesson_count"` + HasIntro bool `json:"has_introduction"` + HasAssignment bool `json:"has_assignment"` + // AssignmentQuiz is present only when the assignment came from + // assignment.quiz.json (additive; Markdown assignments omit it). + AssignmentQuiz *quiz.Summary `json:"assignment_quiz,omitempty"` ManifestUsed int `json:"manifest_images"` ImagesUploaded int `json:"images_uploaded,omitempty"` FailedImages []string `json:"failed_images,omitempty"` @@ -197,8 +221,19 @@ func importModule(p ImportParams) (*ImportResult, error) { // Fetch current module state to determine SLT lock status and preserve metadata existing, err := fetchExistingModule(p.Ctx, p.Client, p.CourseID, data.ModuleCode) if err != nil { + if errors.Is(err, errDegradedRead) { + return nil, fmt.Errorf("%w; nothing was sent — retry when the gateway is healthy", err) + } // Only trigger creation for "not found" errors, not auth/network failures if p.CreateMode && errors.Is(err, errModuleNotFound) { + // A quiz file carries no title and a brand-new module has no + // assignment to take one from, so the update after the create + // would be refused (R5). Refuse here, before the create POST, + // rather than leaving an empty module behind — the condition is + // fully known before any request. + if data.Assignment != nil && data.Assignment.RawJSON != nil { + return nil, fmt.Errorf("title required for a module with no existing assignment: assignment.quiz.json carries no title and module %s does not exist yet. Create the module first (import the directory without assignment.quiz.json, or 'andamio course create-module'), then publish the quiz with 'andamio course import-assignment %s %s assignment.quiz.json --title '", data.ModuleCode, p.CourseID, data.ModuleCode) + } if p.DryRun { if !p.Quiet { fmt.Printf("Dry-run: would create module %s (%s) with sort_order %d\n", data.Title, data.ModuleCode, p.SortOrder) @@ -208,16 +243,17 @@ func importModule(p ImportParams) (*ImportResult, error) { earlyHash = cardano.ComputeSltHash(data.SLTs) } return &ImportResult{ - CourseID: p.CourseID, - ModuleCode: data.ModuleCode, - Title: data.Title, - DryRun: true, - SLTCount: len(data.SLTs), - SltHash: earlyHash, - LessonCount: len(data.Lessons), - HasIntro: data.Introduction != nil, - HasAssignment: data.Assignment != nil, - Changes: map[string]interface{}{"would_create_module": true}, + CourseID: p.CourseID, + ModuleCode: data.ModuleCode, + Title: data.Title, + DryRun: true, + SLTCount: len(data.SLTs), + SltHash: earlyHash, + LessonCount: len(data.Lessons), + HasIntro: data.Introduction != nil, + HasAssignment: data.Assignment != nil, + AssignmentQuiz: data.AssignmentQuiz, + Changes: map[string]interface{}{"would_create_module": true}, }, nil } if !p.Quiet { @@ -283,6 +319,7 @@ func importModule(p ImportParams) (*ImportResult, error) { LessonCount: len(data.Lessons), HasIntro: data.Introduction != nil, HasAssignment: data.Assignment != nil, + AssignmentQuiz: data.AssignmentQuiz, ManifestUsed: len(data.ImageManifest), ImagesUploaded: imagesUploaded, FailedImages: data.ImageWarnings, @@ -393,7 +430,9 @@ func runCourseImport(cmd *cobra.Command, args []string) error { if r.HasIntro { fmt.Printf(" Introduction: yes\n") } - if r.HasAssignment { + if r.AssignmentQuiz != nil { + fmt.Printf(" Assignment: quiz (%d questions, threshold %d)\n", r.AssignmentQuiz.QuestionCount, r.AssignmentQuiz.PassThreshold) + } else if r.HasAssignment { fmt.Printf(" Assignment: yes\n") } @@ -572,9 +611,29 @@ func readCompiledModule(dir string) (*ImportData, error) { data.Introduction = &ContentSection{Title: title, TiptapJSON: tiptap} } - // Read assignment.md if exists — H1 → title, rest → content_json - assignPath := filepath.Join(dir, "assignment.md") - if assignBytes, err := os.ReadFile(assignPath); err == nil && len(assignBytes) > 0 { + // The assignment is either assignment.md (Markdown → Tiptap, H1 → title) + // or assignment.quiz.json (a quiz envelope sent verbatim). Both present is + // an error here, before any request: the ambiguity is never resolved by + // picking one (#165). + // Only a missing file means "no assignment": an unreadable one (EACCES, + // a directory in its place) must fail, not silently drop the assignment + // from the payload. A zero-byte assignment.md is ignored, as the Markdown + // path always has, so it never conflicts with a quiz file. + assignBytes, mdErr := os.ReadFile(filepath.Join(dir, "assignment.md")) + if mdErr != nil && !os.IsNotExist(mdErr) { + return nil, fmt.Errorf("failed to read assignment.md: %w", mdErr) + } + quizBytes, quizErr := os.ReadFile(filepath.Join(dir, "assignment.quiz.json")) + if quizErr != nil && !os.IsNotExist(quizErr) { + return nil, fmt.Errorf("failed to read assignment.quiz.json: %w", quizErr) + } + hasMD := mdErr == nil && len(assignBytes) > 0 + hasQuiz := quizErr == nil + if hasMD && hasQuiz { + return nil, fmt.Errorf("both assignment.md and assignment.quiz.json exist in %s — a module has one assignment; remove the file that is not the assignment you mean to publish", dir) + } + + if hasMD { title, body := extractH1Title(string(assignBytes)) if title == "" && output.GetFormat() != output.FormatJSON { fmt.Printf("Warning: assignment.md has no # title heading\n") @@ -586,9 +645,49 @@ func readCompiledModule(dir string) (*ImportData, error) { data.Assignment = &ContentSection{Title: title, TiptapJSON: tiptap} } + if hasQuiz { + _, summary, err := parseQuizFile(quizBytes, "assignment.quiz.json") + if err != nil { + return nil, err + } + // Title stays empty on purpose: the update merge then keeps the + // module's existing assignment title, which is what export relies on. + data.Assignment = &ContentSection{RawJSON: json.RawMessage(quizBytes)} + data.AssignmentQuiz = &summary + } + return data, nil } +// parseQuizFile recognizes and validates a quiz envelope read from name, +// returning the decoded envelope and its summary. Every failure is an error +// that names the file: a Tiptap doc points the author at the Markdown path, +// any other non-quiz type is refused, and an invalid quiz lists every +// violated rule on its own line. There is no bypass (R16). +func parseQuizFile(raw []byte, name string) (map[string]interface{}, quiz.Summary, error) { + env, kind, issues, err := quiz.Parse(raw) + if err != nil { + return nil, quiz.Summary{}, fmt.Errorf("%s: %w", name, err) + } + switch kind { + case quiz.Doc: + return nil, quiz.Summary{}, fmt.Errorf("%s is a Tiptap document, not a quiz. Tiptap assignments are authored as assignment.md in a module directory and published with 'andamio course import <dir>'", name) + case quiz.Quiz: + // validated below + default: + return nil, quiz.Summary{}, fmt.Errorf("%s is not a quiz envelope: expected an object with \"type\": \"quiz\", \"version\": 1 and a \"questions\" array", name) + } + if len(issues) > 0 { + lines := make([]string, 0, len(issues)+1) + lines = append(lines, fmt.Sprintf("%s is not a valid quiz (%d issue(s)):", name, len(issues))) + for _, is := range issues { + lines = append(lines, " "+is.String()) + } + return nil, quiz.Summary{}, errors.New(strings.Join(lines, "\n")) + } + return env, quiz.Summarize(env), nil +} + // loadImageManifest reads .image-manifest.json from the assets directory. // Returns an empty map if the file doesn't exist. // Warns on parse errors rather than silently degrading. @@ -1220,9 +1319,28 @@ func fetchExistingModule(ctx context.Context, c *client.Client, courseID, module return existing, nil } + // A module with content comes from db-api, so a found module is complete + // even when Andamioscan was down. Not found on a degraded list depends on + // WHICH backend was missing: with db-api down the module may exist with + // content the list could not show; with only Andamioscan down the db list + // is complete and authoritative, so absent means absent (and --create + // must keep working through an Andamioscan outage). + if warning := metaWarning(resp); warning != "" && warningHidesDBContent(warning) { + return nil, fmt.Errorf("%w: %s (module '%s' in course '%s' may exist but was not returned)", errDegradedRead, warning, moduleCode, courseID) + } return nil, fmt.Errorf("%w: '%s' in course '%s'", errModuleNotFound, moduleCode, courseID) } +// warningHidesDBContent reports whether a merged-read meta.warning means the +// db-api side — the only source of module content — was unavailable. The +// gateway's strings (andamio-api course_orchestrator.go) begin "DB API +// unavailable, ..." or "Andamioscan unavailable, ...". Only the Andamioscan +// form leaves the db list authoritative; unknown text is treated as hiding +// content, the conservative reading. +func warningHidesDBContent(warning string) bool { + return !strings.HasPrefix(strings.ToLower(strings.TrimSpace(warning)), "andamioscan unavailable") +} + func updateModuleContent(ctx context.Context, c *client.Client, courseID string, data *ImportData, existing *ExistingModuleData, sltsLocked bool, dryRun bool, showPayload bool) (map[string]interface{}, error) { isJSON := output.GetFormat() == output.FormatJSON @@ -1351,10 +1469,16 @@ func updateModuleContent(ctx context.Context, c *client.Client, courseID string, payload["introduction"] = intro } - // Build assignment — H1 title from file, preserve existing metadata + // Build assignment — H1 title from file, preserve existing metadata. A quiz + // assignment carries its file bytes as content_json (RawJSON) instead of a + // converted Tiptap document. if data.Assignment != nil { + var contentJSON interface{} = data.Assignment.TiptapJSON + if data.Assignment.RawJSON != nil { + contentJSON = data.Assignment.RawJSON + } assign := map[string]interface{}{ - "content_json": data.Assignment.TiptapJSON, + "content_json": contentJSON, } if data.Assignment.Title != "" { assign["title"] = data.Assignment.Title @@ -1363,6 +1487,12 @@ func updateModuleContent(ctx context.Context, c *client.Client, courseID string, assign["title"] = v } } + // A quiz file carries no title. db-api stores Title as a plain + // string, so omitting it here would silently publish an untitled + // assignment; refuse, as import-assignment does (R5). + if _, ok := assign["title"]; !ok && data.Assignment.RawJSON != nil { + return nil, fmt.Errorf("title required for a module with no existing assignment: assignment.quiz.json carries no title. Publish it first with 'andamio course import-assignment %s %s assignment.quiz.json --title <title>', then import the directory", courseID, data.ModuleCode) + } if existing.Assignment != nil { for _, field := range []string{"description", "image_url", "video_url"} { if v, ok := existing.Assignment[field]; ok && v != nil && v != "" { diff --git a/cmd/andamio/course_import_assignment.go b/cmd/andamio/course_import_assignment.go new file mode 100644 index 0000000..3d6cddb --- /dev/null +++ b/cmd/andamio/course_import_assignment.go @@ -0,0 +1,344 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "reflect" + "strings" + + "github.com/Andamio-Platform/andamio-cli/internal/apierr" + "github.com/Andamio-Platform/andamio-cli/internal/client" + "github.com/Andamio-Platform/andamio-cli/internal/config" + "github.com/Andamio-Platform/andamio-cli/internal/output" + "github.com/spf13/cobra" +) + +func init() { + courseCmd.AddCommand(courseImportAssignmentCmd) + courseImportAssignmentCmd.Flags().String("course", "", "Course name or substring (alternative to the course-id argument)") + courseImportAssignmentCmd.Flags().String("title", "", "Assignment title (default: the module's existing assignment title; required when the module has no assignment yet)") + courseImportAssignmentCmd.Flags().String("description", "", "Assignment description (default: the existing description)") + courseImportAssignmentCmd.Flags().Bool("dry-run", false, "Validate and print the summary without sending anything") + courseImportAssignmentCmd.Flags().Bool("show-payload", false, "With --dry-run, also print the full API payload on stderr") +} + +var courseImportAssignmentCmd = &cobra.Command{ + Use: "import-assignment <course-id> <module-code> <file.json>", + Short: "Publish a quiz assignment from a JSON envelope, verbatim, and verify it by read-back", + Long: `Publish a quiz assignment to a course module without a module directory. + +The file is a quiz envelope — {"type": "quiz", "version": 1, "passThreshold": N, +"questions": [...]} — exactly as the Andamio app stores and grades it. It is +validated before any request with the union of the rules the FCB Fan Campus app and +app.andamio.io enforce: type and +version, a non-empty questions array, unique question ids, non-blank prompts, at least two options +per question with unique non-empty values and non-blank labels, a correctValue matching one option, a +passThreshold in 1..len(questions), and an intro that is a Tiptap doc if present. +Every violated rule is listed. There is no bypass flag. A Tiptap document is +refused: author it as assignment.md in a module directory and use 'course import'. + +Only the assignment is sent. Lessons, SLTs and the introduction are untouched. +The existing assignment's title, description, image and video URLs are kept +unless --title / --description override them; a module with no assignment yet +requires --title. Assignments are editable in any module status — only SLTs +lock after DRAFT — so this works on DRAFT, APPROVED, PENDING_TX and ON_CHAIN +modules alike. + +After the update the module is re-fetched and the stored content_json is +deep-compared to the file. A mismatch, a degraded (206) read-back, or a +read-back request that fails for any reason exits 1 with kind "verify" under +--output json: the update WAS applied and should be inspected. + +Examples: + andamio course import-assignment <course-id> 101 quiz.json --dry-run + andamio course import-assignment <course-id> 101 quiz.json --title "Module Quiz" + andamio course import-assignment 101 quiz.json --course "Fan Campus" + andamio course import-assignment <course-id> 101 quiz.json --output json + +Requires user authentication via 'andamio user login'.`, + Args: cobra.RangeArgs(2, 3), + PreRunE: func(cmd *cobra.Command, args []string) error { + return requireUserAuth() + }, + RunE: runCourseImportAssignment, +} + +// ImportAssignmentEnvelope is the stable --output json contract for +// `course import-assignment`. Verified is true only after the read-back +// deep-compare passed; a dry run reports DryRun true and Verified false. +type ImportAssignmentEnvelope struct { + CourseID string `json:"course_id"` + ModuleCode string `json:"module_code"` + ModuleStatus string `json:"module_status"` + Assignment ImportAssignmentSummary `json:"assignment"` + DryRun bool `json:"dry_run,omitempty"` + Verified bool `json:"verified"` +} + +// ImportAssignmentSummary digests what was (or would be) published. TitleSource +// is "flag" when --title supplied the title and "existing" when it came from +// the module's current assignment. +type ImportAssignmentSummary struct { + Title string `json:"title"` + TitleSource string `json:"title_source"` + QuestionCount int `json:"question_count"` + PassThreshold int `json:"pass_threshold"` + QuestionIDs []string `json:"question_ids"` +} + +// assignmentUpdatePayload is the whole request body: the module identity and +// the assignment, nothing else. Omitted top-level fields are left unchanged by +// the gateway, which is what keeps lessons, SLTs and the introduction out of +// reach of this command. +type assignmentUpdatePayload struct { + CourseID string `json:"course_id"` + CourseModuleCode string `json:"course_module_code"` + Assignment assignmentInput `json:"assignment"` +} + +// assignmentInput mirrors db-api's AggregateAssignmentInput. Every metadata +// field is carried because processAssignmentUpdate overwrites all of them +// from the input — an omitted description is nulled, not preserved (KTD5 in +// the #165 plan). ContentJSON is the file bytes: no re-encoding on the way out. +type assignmentInput struct { + Title string `json:"title"` + Description *string `json:"description,omitempty"` + ImageURL *string `json:"image_url,omitempty"` + VideoURL *string `json:"video_url,omitempty"` + ContentJSON json.RawMessage `json:"content_json"` +} + +type importAssignmentOptions struct { + CourseID string + ModuleCode string + FilePath string + Title string + Description string + DryRun bool + ShowPayload bool + Quiet bool // suppress stderr progress (JSON mode) +} + +func runCourseImportAssignment(cmd *cobra.Command, args []string) error { + ctx := cmd.Context() + isJSON := output.GetFormat() == output.FormatJSON + + cfg, err := config.Load() + if err != nil { + return err + } + c := client.New(cfg) + + opts := importAssignmentOptions{Quiet: isJSON} + opts.Title, _ = cmd.Flags().GetString("title") + opts.Description, _ = cmd.Flags().GetString("description") + opts.DryRun, _ = cmd.Flags().GetBool("dry-run") + opts.ShowPayload, _ = cmd.Flags().GetBool("show-payload") + + if len(args) == 3 { + opts.CourseID, opts.ModuleCode, opts.FilePath = args[0], args[1], args[2] + } else { + // import-assignment <module-code> <file.json> --course "Name". + // resolveCourseID owns the "no course given" error, as it does for + // course export's two-arg form. + opts.ModuleCode, opts.FilePath = args[0], args[1] + opts.CourseID, err = resolveCourseID(ctx, c, "", cmd) + if err != nil { + return err + } + } + + env, err := runImportAssignment(ctx, c, opts) + if err != nil { + return err + } + + if isJSON { + return output.PrintJSON(env) + } + a := env.Assignment + ids := strings.Join(a.QuestionIDs, ", ") + if env.DryRun { + fmt.Printf("Dry-run: would publish quiz assignment %q (title from %s) to module %s: %d questions, threshold %d, ids %s. Nothing sent.\n", + a.Title, a.TitleSource, env.ModuleCode, a.QuestionCount, a.PassThreshold, ids) + return nil + } + fmt.Printf("Published quiz assignment %q to module %s (%s): %d questions, threshold %d, ids %s — verified by read-back.\n", + a.Title, env.ModuleCode, env.ModuleStatus, a.QuestionCount, a.PassThreshold, ids) + return nil +} + +// runImportAssignment is the command body, split from cobra so tests can drive +// it against an httptest gateway. Order matters: parse and validate the file +// first (no request for a bad file), fetch the module, resolve title and +// metadata, then either stop (dry run) or POST and read back. +func runImportAssignment(ctx context.Context, c *client.Client, opts importAssignmentOptions) (*ImportAssignmentEnvelope, error) { + raw, err := os.ReadFile(opts.FilePath) + if err != nil { + return nil, fmt.Errorf("quiz file: %w", err) + } + env, summary, err := parseQuizFile(raw, opts.FilePath) + if err != nil { + return nil, err + } + + if !opts.Quiet { + fmt.Fprintf(os.Stderr, "Fetching module %s from course %s...\n", opts.ModuleCode, opts.CourseID) + } + existing, err := fetchExistingModule(ctx, c, opts.CourseID, opts.ModuleCode) + if err != nil { + switch { + case errors.Is(err, errDegradedRead): + // db-api may be the missing backend: the module may exist with an + // assignment the list could not show. Inferring "no existing + // assignment" would null its metadata on the write (R7a). + return nil, fmt.Errorf("module state could not be read reliably: %w; not sending. Retry when the gateway is healthy", err) + case errors.Is(err, errModuleNotFound): + return nil, &apierr.NotFoundError{Message: err.Error() + ". Run 'andamio course modules " + opts.CourseID + " --output json' to list the course's modules"} + } + return nil, err + } + + title, titleSource := opts.Title, "flag" + if title == "" { + titleSource = "existing" + if existing.Assignment != nil { + title, _ = existing.Assignment["title"].(string) + } + if title == "" { + return nil, fmt.Errorf("title required for a module with no existing assignment: pass --title") + } + } + + input := assignmentInput{ + Title: title, + Description: firstNonEmpty(opts.Description, existingString(existing.Assignment, "description")), + ImageURL: firstNonEmpty(existingString(existing.Assignment, "image_url")), + VideoURL: firstNonEmpty(existingString(existing.Assignment, "video_url")), + ContentJSON: json.RawMessage(raw), + } + payload := assignmentUpdatePayload{ + CourseID: opts.CourseID, + CourseModuleCode: opts.ModuleCode, + Assignment: input, + } + + result := &ImportAssignmentEnvelope{ + CourseID: opts.CourseID, + ModuleCode: opts.ModuleCode, + ModuleStatus: existing.Status, + Assignment: ImportAssignmentSummary{ + Title: title, + TitleSource: titleSource, + QuestionCount: summary.QuestionCount, + PassThreshold: summary.PassThreshold, + QuestionIDs: summary.QuestionIDs, + }, + } + + if opts.DryRun { + result.DryRun = true + if opts.ShowPayload && !opts.Quiet { + pretty, err := json.MarshalIndent(payload, "", " ") + if err != nil { + return nil, fmt.Errorf("failed to marshal payload: %w", err) + } + fmt.Fprintln(os.Stderr, "Dry-run payload (not sent):") + fmt.Fprintln(os.Stderr, string(pretty)) + } + return result, nil + } + + if !opts.Quiet { + fmt.Fprintf(os.Stderr, "Publishing quiz assignment (%d questions, threshold %d)...\n", summary.QuestionCount, summary.PassThreshold) + } + var resp map[string]interface{} + if err := c.Post(ctx, "/api/v2/course/teacher/course-module/update", payload, &resp); err != nil { + return nil, fmt.Errorf("failed to update module: %w", err) + } + + // Read-back. From here on the module has been modified, so every failure + // message says so — a caller must never read one as "nothing happened". + if !opts.Quiet { + fmt.Fprintln(os.Stderr, "Reading back to verify...") + } + stored, err := fetchExistingModule(ctx, c, opts.CourseID, opts.ModuleCode) + if err != nil { + // Every failure after the accepted POST is kind verify: "applied but + // unconfirmed" is the outcome a caller must branch on, whatever the + // cause. A connection reset here is not "the request never reached + // the service" and an expired token is not "unauthenticated" — the + // module changed. The cause stays inspectable via Unwrap. + msg := fmt.Sprintf("the read-back request failed (%v); the stored value could not be confirmed", err) + switch { + case errors.Is(err, errDegradedRead): + msg = fmt.Sprintf("the read-back was degraded (%v); the stored value could not be confirmed", err) + case errors.Is(err, errModuleNotFound): + msg = fmt.Sprintf("the read-back did not return the module (%v); the stored value could not be confirmed", err) + } + return nil, &apierr.VerifyError{Path: "assignment.content_json", Message: msg, Err: err} + } + if err := verifyStoredAssignment(env, input, stored.Assignment); err != nil { + return nil, err + } + + result.Verified = true + return result, nil +} + +// verifyStoredAssignment deep-compares what was sent with what the gateway now +// returns. content_json is compared structurally (the gateway re-serializes +// from jsonb, so bytes are not a property it offers); the metadata fields are +// compared as strings with absent and empty treated alike. +func verifyStoredAssignment(sent map[string]interface{}, input assignmentInput, stored map[string]interface{}) error { + if stored == nil { + return &apierr.VerifyError{Path: "assignment", Message: "the module has no assignment after the update"} + } + if !reflect.DeepEqual(stored["content_json"], sent) { + return &apierr.VerifyError{Path: "assignment.content_json", Message: "the stored value did not read back identical to the file"} + } + fields := []struct { + name string + sent *string + }{ + {"title", &input.Title}, + {"description", input.Description}, + {"image_url", input.ImageURL}, + {"video_url", input.VideoURL}, + } + for _, f := range fields { + want := "" + if f.sent != nil { + want = *f.sent + } + got, _ := stored[f.name].(string) + if got != want { + return &apierr.VerifyError{Path: "assignment." + f.name, Message: fmt.Sprintf("sent %q, stored %q", want, got)} + } + } + return nil +} + +// existingString reads a non-empty string field from the existing assignment +// (nil-safe), or "" when absent. +func existingString(assignment map[string]interface{}, field string) string { + if assignment == nil { + return "" + } + v, _ := assignment[field].(string) + return v +} + +// firstNonEmpty returns a pointer to the first non-empty candidate, or nil when +// all are empty so the JSON field is omitted. +func firstNonEmpty(candidates ...string) *string { + for _, s := range candidates { + if s != "" { + return &s + } + } + return nil +} diff --git a/cmd/andamio/course_import_assignment_test.go b/cmd/andamio/course_import_assignment_test.go new file mode 100644 index 0000000..3f4631d --- /dev/null +++ b/cmd/andamio/course_import_assignment_test.go @@ -0,0 +1,553 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "reflect" + "strings" + "sync" + "testing" + + "github.com/Andamio-Platform/andamio-cli/internal/apierr" + "github.com/Andamio-Platform/andamio-cli/internal/client" + "github.com/Andamio-Platform/andamio-cli/internal/config" +) + +// assignmentStub is a gateway stand-in for the two routes import-assignment +// uses: the teacher module list (served twice — pre-fetch and read-back) and +// the module update. listBodies are served in order; the last one repeats. +// updateStatus/updateBody shape the POST answer. Every POST body is captured. +type assignmentStub struct { + mu sync.Mutex + listBodies []string + listStatus []int // parallel to listBodies; 0 means 200 + updateStatus int + updateBody string + listCalls int + posts []map[string]interface{} +} + +func (s *assignmentStub) handler(t *testing.T) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + s.mu.Lock() + defer s.mu.Unlock() + w.Header().Set("Content-Type", "application/json") + switch { + case r.URL.Path == "/api/v2/course/teacher/course-modules/list" && r.Method == http.MethodPost: + i := s.listCalls + if i >= len(s.listBodies) { + i = len(s.listBodies) - 1 + } + s.listCalls++ + status := http.StatusOK + if i < len(s.listStatus) && s.listStatus[i] != 0 { + status = s.listStatus[i] + } + w.WriteHeader(status) + _, _ = w.Write([]byte(s.listBodies[i])) + case r.URL.Path == "/api/v2/course/teacher/course-module/update" && r.Method == http.MethodPost: + var body map[string]interface{} + _ = json.NewDecoder(r.Body).Decode(&body) + s.posts = append(s.posts, body) + status := s.updateStatus + if status == 0 { + status = http.StatusOK + } + w.WriteHeader(status) + body2 := s.updateBody + if body2 == "" { + body2 = `{"changes":{"assignment_updated":true}}` + } + _, _ = w.Write([]byte(body2)) + default: + http.Error(w, "unexpected route "+r.Method+" "+r.URL.Path, http.StatusNotFound) + } + } +} + +func (s *assignmentStub) serve(t *testing.T) (*client.Client, string) { + t.Helper() + srv := httptest.NewServer(s.handler(t)) + t.Cleanup(srv.Close) + return client.New(&config.Config{BaseURL: srv.URL, UserJWT: "test-jwt"}), srv.URL +} + +// listBody renders the teacher module list with one module 101 and the given +// assignment object (nil → module has no assignment). warning adds meta.warning. +func listBody(t *testing.T, assignment map[string]interface{}, warning string) string { + t.Helper() + content := map[string]interface{}{ + "course_module_code": "101", + "module_status": "ON_CHAIN", + "slts": []interface{}{map[string]interface{}{"slt_index": 1, "slt_text": "Do a thing"}}, + } + if assignment != nil { + content["assignment"] = assignment + } + env := map[string]interface{}{ + "data": []interface{}{map[string]interface{}{"content": content}}, + } + if warning != "" { + env["meta"] = map[string]interface{}{"warning": warning} + } + b, err := json.Marshal(env) + if err != nil { + t.Fatal(err) + } + return string(b) +} + +// chainOnlyListBody is the shape the merged endpoint really produces when +// db-api is down: a 206 with meta.warning and every module chain-only — +// slt_hash, course_id, source, and no `content` key at all +// (MergedCourseModuleItem.Content is omitempty and only db-api fills it). +func chainOnlyListBody(t *testing.T, warning string) string { + t.Helper() + env := map[string]interface{}{ + "data": []interface{}{map[string]interface{}{ + "slt_hash": "c28e2bad6ef905179a5d81eb1ebdb9198db87f067fe867ed3c34b566d9c5f6c5", + "course_id": "course-1", + "source": "chain_only", + }}, + "meta": map[string]interface{}{"warning": warning}, + } + b, err := json.Marshal(env) + if err != nil { + t.Fatal(err) + } + return string(b) +} + +func docAssignment(title string) map[string]interface{} { + return map[string]interface{}{ + "title": title, + "content_json": map[string]interface{}{"type": "doc", "content": []interface{}{}}, + } +} + +func writeQuizFile(t *testing.T, env interface{}) string { + t.Helper() + b, err := json.MarshalIndent(env, "", " ") + if err != nil { + t.Fatal(err) + } + p := filepath.Join(t.TempDir(), "quiz.json") + if err := os.WriteFile(p, append(b, '\n'), 0644); err != nil { + t.Fatal(err) + } + return p +} + +func TestImportAssignment_PublishesVerbatimAndVerifies(t *testing.T) { + quiz := quizEnvelope() + existing := docAssignment("Quiz") + existing["description"] = "Pass to continue" + existing["image_url"] = "https://cdn/x.png" + existing["video_url"] = "https://cdn/x.mp4" + stored := map[string]interface{}{"title": "Quiz", "description": "Pass to continue", "image_url": "https://cdn/x.png", "video_url": "https://cdn/x.mp4", "content_json": quiz} + stub := &assignmentStub{listBodies: []string{listBody(t, existing, ""), listBody(t, stored, "")}} + c, _ := stub.serve(t) + + env, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quiz), + }) + if err != nil { + t.Fatalf("runImportAssignment: %v", err) + } + if !env.Verified || env.Assignment.QuestionCount != 2 || env.Assignment.PassThreshold != 2 || env.Assignment.Title != "Quiz" || env.Assignment.TitleSource != "existing" { + t.Errorf("envelope = %+v", env) + } + if env.ModuleStatus != "ON_CHAIN" { + t.Errorf("module_status = %q, want ON_CHAIN", env.ModuleStatus) + } + + if len(stub.posts) != 1 { + t.Fatalf("POSTs = %d, want 1", len(stub.posts)) + } + post := stub.posts[0] + keys := make([]string, 0, len(post)) + for k := range post { + keys = append(keys, k) + } + if len(post) != 3 || post["course_id"] != "course-1" || post["course_module_code"] != "101" || post["assignment"] == nil { + t.Errorf("POST body keys = %v, want exactly course_id, course_module_code, assignment", keys) + } + assign := post["assignment"].(map[string]interface{}) + if !reflect.DeepEqual(assign["content_json"], quiz) { + t.Errorf("content_json differs from the file (structural compare)\n got: %v", assign["content_json"]) + } + if assign["title"] != "Quiz" || assign["description"] != "Pass to continue" || assign["image_url"] != "https://cdn/x.png" || assign["video_url"] != "https://cdn/x.mp4" { + t.Errorf("existing metadata not carried: %v", assign) + } + if stub.listCalls != 2 { + t.Errorf("list calls = %d, want 2 (pre-fetch + read-back)", stub.listCalls) + } +} + +func TestImportAssignment_TitleFlagOverrides(t *testing.T) { + quiz := quizEnvelope() + stub := &assignmentStub{listBodies: []string{ + listBody(t, docAssignment("Old"), ""), + listBody(t, map[string]interface{}{"title": "New", "content_json": quiz}, ""), + }} + c, _ := stub.serve(t) + env, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quiz), Title: "New", + }) + if err != nil { + t.Fatal(err) + } + if env.Assignment.Title != "New" || env.Assignment.TitleSource != "flag" { + t.Errorf("title = %q source = %q", env.Assignment.Title, env.Assignment.TitleSource) + } + if stub.posts[0]["assignment"].(map[string]interface{})["title"] != "New" { + t.Error("POST did not carry the --title override") + } +} + +func TestImportAssignment_NoExistingAssignmentRequiresTitle(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, nil, "")}} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if err == nil || !strings.Contains(err.Error(), "title required for a module with no existing assignment") { + t.Fatalf("err = %v", err) + } + if len(stub.posts) != 0 { + t.Error("no POST may be sent when the title cannot be resolved") + } +} + +func TestImportAssignment_RejectsBeforeAnyRequest(t *testing.T) { + cases := []struct { + name string + file interface{} + want string + }{ + {"tiptap doc", map[string]interface{}{"type": "doc", "content": []interface{}{}}, "course import"}, + {"quiz-evidence", map[string]interface{}{"type": "quiz-evidence", "version": 1}, "not a quiz"}, + {"array", []interface{}{1, 2}, "not a quiz"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, tc.file), + }) + if err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatalf("err = %v, want mention of %q", err, tc.want) + } + if stub.listCalls != 0 || len(stub.posts) != 0 { + t.Error("a rejected file must not cause any request") + } + }) + } +} + +func TestImportAssignment_InvalidQuizListsEveryRule(t *testing.T) { + broken := quizEnvelope() + broken["passThreshold"] = float64(9) + qs := broken["questions"].([]interface{}) + qs[1].(map[string]interface{})["correctValue"] = "zzz" + qs[0].(map[string]interface{})["prompt"] = "" + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, broken), + }) + if err == nil { + t.Fatal("expected validation error") + } + for _, want := range []string{"threshold-exceeds-questions", "dangling-correct-value", "empty-prompt"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error should list %q: %v", want, err) + } + } + if stub.listCalls != 0 { + t.Error("validation must run before any request") + } +} + +func TestImportAssignment_DryRunSendsNothing(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + c, _ := stub.serve(t) + env, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), DryRun: true, + }) + if err != nil { + t.Fatal(err) + } + if !env.DryRun || env.Verified { + t.Errorf("dry run envelope = %+v, want dry_run true, verified false", env) + } + if len(stub.posts) != 0 || stub.listCalls != 1 { + t.Errorf("dry run must fetch once and POST never; list=%d posts=%d", stub.listCalls, len(stub.posts)) + } +} + +func TestImportAssignment_ReadBackMismatchIsVerifyError(t *testing.T) { + quiz := quizEnvelope() + stub := &assignmentStub{listBodies: []string{ + listBody(t, docAssignment("Quiz"), ""), + listBody(t, docAssignment("Quiz"), ""), // stored value unchanged + }} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quiz), + }) + var verify *apierr.VerifyError + if !errors.As(err, &verify) { + t.Fatalf("err = %v, want VerifyError", err) + } + if apierr.Kind(err) != apierr.KindVerify || !strings.Contains(err.Error(), "accepted") { + t.Errorf("kind = %q, err = %v", apierr.Kind(err), err) + } + if len(stub.posts) != 1 { + t.Error("the update must have been sent before the mismatch was detected") + } +} + +func TestImportAssignment_ReadBackMetadataMismatchIsVerifyError(t *testing.T) { + quiz := quizEnvelope() + stub := &assignmentStub{listBodies: []string{ + listBody(t, docAssignment("Quiz"), ""), + listBody(t, map[string]interface{}{"title": "Something Else", "content_json": quiz}, ""), + }} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quiz), + }) + if apierr.Kind(err) != apierr.KindVerify || !strings.Contains(err.Error(), "title") { + t.Fatalf("err = %v, want verify error naming title", err) + } +} + +func TestImportAssignment_DegradedPreFetchIsErrorNotTitleError(t *testing.T) { + stub := &assignmentStub{ + listBodies: []string{chainOnlyListBody(t, "DB API unavailable, showing on-chain data only")}, + listStatus: []int{http.StatusPartialContent}, + } + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if err == nil || !strings.Contains(err.Error(), "DB API unavailable") || strings.Contains(err.Error(), "title required") { + t.Fatalf("err = %v, want an error naming the warning, not the title error", err) + } + if apierr.Kind(err) == apierr.KindNotFound { + t.Errorf("a degraded read must not classify as not_found: %v", err) + } + if len(stub.posts) != 0 { + t.Error("no POST may follow a degraded pre-fetch") + } +} + +// An Andamioscan outage also produces a 206 with meta.warning, but a module +// that came back with content came from db-api and is complete: publishing +// must proceed and verify normally. +func TestImportAssignment_AndamioscanOnlyWarningStillPublishes(t *testing.T) { + quiz := quizEnvelope() + stub := &assignmentStub{ + listBodies: []string{ + listBody(t, docAssignment("Quiz"), "Andamioscan unavailable, showing database data only"), + listBody(t, map[string]interface{}{"title": "Quiz", "content_json": quiz}, "Andamioscan unavailable, showing database data only"), + }, + listStatus: []int{http.StatusPartialContent, http.StatusPartialContent}, + } + c, _ := stub.serve(t) + env, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quiz), + }) + if err != nil { + t.Fatalf("a found module with content is complete regardless of the warning: %v", err) + } + if !env.Verified || len(stub.posts) != 1 { + t.Errorf("verified=%v posts=%d, want a verified publish", env.Verified, len(stub.posts)) + } +} + +func TestImportAssignment_ModuleNotInListIsNotFound(t *testing.T) { + stub := &assignmentStub{listBodies: []string{`{"data":[{"content":{"course_module_code":"999","module_status":"DRAFT"}}]}`}} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if apierr.Kind(err) != apierr.KindNotFound { + t.Fatalf("kind = %q, want not_found; err = %v", apierr.Kind(err), err) + } + if len(stub.posts) != 0 { + t.Error("no POST for an unknown module") + } +} + +// Only Andamioscan being down leaves the db list complete, so a module that is +// absent from it is genuinely absent: not_found, not a degraded read — and +// course import --create keeps working through such an outage. +func TestImportAssignment_AndamioscanOnlyWarningMissingModuleIsNotFound(t *testing.T) { + stub := &assignmentStub{ + listBodies: []string{`{"data":[{"content":{"course_module_code":"999","module_status":"DRAFT"}}],"meta":{"warning":"Andamioscan unavailable, showing DB data only"}}`}, + listStatus: []int{http.StatusPartialContent}, + } + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if apierr.Kind(err) != apierr.KindNotFound { + t.Fatalf("kind = %q, want not_found; err = %v", apierr.Kind(err), err) + } +} + +func TestImportAssignment_ReadBackOmittingModuleSaysAccepted(t *testing.T) { + stub := &assignmentStub{listBodies: []string{ + listBody(t, docAssignment("Quiz"), ""), + `{"data":[{"content":{"course_module_code":"999","module_status":"DRAFT"}}]}`, + }} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if apierr.Kind(err) != apierr.KindVerify || !strings.Contains(err.Error(), "accepted") { + t.Fatalf("err = %v, want kind verify with a message saying the update was accepted", err) + } + if !errors.Is(err, errModuleNotFound) { + t.Errorf("the cause must stay inspectable through Unwrap: %v", err) + } +} + +func TestImportAssignment_DegradedReadBackIsVerifyErrorNamingWarning(t *testing.T) { + stub := &assignmentStub{ + listBodies: []string{listBody(t, docAssignment("Quiz"), ""), chainOnlyListBody(t, "DB API unavailable, showing on-chain data only")}, + listStatus: []int{0, http.StatusPartialContent}, + } + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if apierr.Kind(err) != apierr.KindVerify { + t.Fatalf("kind = %q, err = %v", apierr.Kind(err), err) + } + if !strings.Contains(err.Error(), "DB API unavailable") || strings.Contains(err.Error(), "did not read back identical") { + t.Errorf("degraded read-back must name the warning, not claim a mismatch: %v", err) + } +} + +func TestImportAssignment_ReadBackFailureIsVerifyWithCause(t *testing.T) { + stub := &assignmentStub{ + listBodies: []string{listBody(t, docAssignment("Quiz"), ""), `{"message":"down"}`}, + listStatus: []int{0, http.StatusServiceUnavailable}, + } + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + // Every failure after the accepted POST is verify: the module changed, + // and "server" would tell a script the write did not happen. The cause + // stays inspectable through Unwrap. + if apierr.Kind(err) != apierr.KindVerify { + t.Fatalf("kind = %q, want verify; err = %v", apierr.Kind(err), err) + } + var se *apierr.ServerError + if !errors.As(err, &se) { + t.Errorf("underlying 503 must remain reachable via errors.As: %v", err) + } + if !strings.Contains(err.Error(), "accepted") { + t.Errorf("message must say the update was accepted: %v", err) + } +} + +func TestImportAssignment_UpdateErrorsKeepTheirKind(t *testing.T) { + for _, tc := range []struct { + status int + kind string + }{{http.StatusNotFound, apierr.KindNotFound}, {http.StatusUnauthorized, apierr.KindAuth}} { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}, updateStatus: tc.status, updateBody: `{"message":"stub"}`} + c, _ := stub.serve(t) + _, err := runImportAssignment(context.Background(), c, importAssignmentOptions{ + CourseID: "course-1", ModuleCode: "101", FilePath: writeQuizFile(t, quizEnvelope()), + }) + if apierr.Kind(err) != tc.kind { + t.Errorf("status %d: kind = %q, want %q (%v)", tc.status, apierr.Kind(err), tc.kind, err) + } + } +} + +// End-to-end through the binary: the JSON error document for a read-back +// mismatch is exactly one document carrying kind verify, exit 1, and the +// dry-run success envelope has the documented shape. +func TestImportAssignment_CLIJSONContract(t *testing.T) { + bin := buildTestBinary(t) + quiz := quizEnvelope() + file := writeQuizFile(t, quiz) + + t.Run("verify mismatch", func(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), ""), listBody(t, docAssignment("Quiz"), "")}} + _, url := stub.serve(t) + stdout, stderr, code := runCLI(t, bin, url, "course", "import-assignment", "course-1", "101", file, "--output", "json") + if code != 1 { + t.Errorf("exit = %d, want 1", code) + } + var parsed map[string]string + if err := json.Unmarshal([]byte(stdout), &parsed); err != nil { + t.Fatalf("stdout must be exactly one JSON document: %v\n%q", err, stdout) + } + if parsed["kind"] != "verify" { + t.Errorf("kind = %q, want verify (stderr %q)", parsed["kind"], stderr) + } + }) + + t.Run("dry run envelope", func(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + _, url := stub.serve(t) + stdout, _, code := runCLI(t, bin, url, "course", "import-assignment", "course-1", "101", file, "--dry-run", "--output", "json") + if code != 0 { + t.Fatalf("exit = %d, want 0; stdout %q", code, stdout) + } + var env ImportAssignmentEnvelope + if err := json.Unmarshal([]byte(stdout), &env); err != nil { + t.Fatalf("stdout is not the envelope: %v\n%q", err, stdout) + } + if !env.DryRun || env.Verified || env.Assignment.QuestionCount != 2 || env.CourseID != "course-1" || env.ModuleCode != "101" { + t.Errorf("envelope = %+v", env) + } + if len(stub.posts) != 0 { + t.Error("dry run sent a POST") + } + }) + + t.Run("text dry run summary on stdout, payload on stderr", func(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + _, url := stub.serve(t) + stdout, stderr, code := runCLI(t, bin, url, "course", "import-assignment", "course-1", "101", file, "--dry-run", "--show-payload") + if code != 0 { + t.Fatalf("exit = %d; stderr %q", code, stderr) + } + if !strings.Contains(stdout, "2 questions") || !strings.Contains(stdout, "threshold 2") { + t.Errorf("stdout summary = %q", stdout) + } + if !strings.Contains(stderr, `"course_module_code"`) { + t.Errorf("--show-payload should print the payload on stderr: %q", stderr) + } + if strings.Contains(stdout, `"course_module_code"`) { + t.Error("payload leaked to stdout") + } + }) + + t.Run("missing file", func(t *testing.T) { + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + _, url := stub.serve(t) + _, stderr, code := runCLI(t, bin, url, "course", "import-assignment", "course-1", "101", filepath.Join(t.TempDir(), "nope.json")) + if code != 1 || !strings.Contains(stderr, "nope.json") { + t.Errorf("exit = %d, stderr = %q", code, stderr) + } + if stub.listCalls != 0 { + t.Error("a missing file must not cause a request") + } + }) +} diff --git a/cmd/andamio/course_import_quiz_test.go b/cmd/andamio/course_import_quiz_test.go new file mode 100644 index 0000000..346c7b7 --- /dev/null +++ b/cmd/andamio/course_import_quiz_test.go @@ -0,0 +1,353 @@ +package main + +import ( + "context" + "encoding/json" + "net/http" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/Andamio-Platform/andamio-cli/internal/config" +) + +const quizOutlineMD = "---\ntitle: Module 101\ncode: \"101\"\n---\n\n## SLTs\n\n1. Do a thing\n" + +// writeQuizModuleDir builds a minimal compiled module directory with the given +// assignment files. quizJSON nil means no assignment.quiz.json; withMD adds an +// assignment.md. +func writeQuizModuleDir(t *testing.T, quizJSON []byte, withMD bool) string { + t.Helper() + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "outline.md"), []byte(quizOutlineMD), 0644); err != nil { + t.Fatal(err) + } + if quizJSON != nil { + if err := os.WriteFile(filepath.Join(dir, "assignment.quiz.json"), quizJSON, 0644); err != nil { + t.Fatal(err) + } + } + if withMD { + if err := os.WriteFile(filepath.Join(dir, "assignment.md"), []byte("# Essay\n\nWrite it.\n"), 0644); err != nil { + t.Fatal(err) + } + } + return dir +} + +func prettyQuiz(t *testing.T) []byte { + t.Helper() + b, err := json.MarshalIndent(quizEnvelope(), "", " ") + if err != nil { + t.Fatal(err) + } + return append(b, '\n') +} + +// payloadAssignment decodes the dry-run payload's assignment through JSON so +// RawMessage and map values compare structurally (KTD3). +func payloadAssignment(t *testing.T, resp map[string]interface{}) map[string]interface{} { + t.Helper() + b, err := json.Marshal(resp["payload"]) + if err != nil { + t.Fatal(err) + } + var payload map[string]interface{} + if err := json.Unmarshal(b, &payload); err != nil { + t.Fatal(err) + } + assign, ok := payload["assignment"].(map[string]interface{}) + if !ok { + t.Fatalf("payload has no assignment object: %v", payload) + } + return assign +} + +func TestReadCompiledModule_QuizAssignment(t *testing.T) { + raw := prettyQuiz(t) + dir := writeQuizModuleDir(t, raw, false) + + data, err := readCompiledModule(dir) + if err != nil { + t.Fatalf("readCompiledModule: %v", err) + } + if data.Assignment == nil { + t.Fatal("Assignment is nil for a directory with assignment.quiz.json") + } + if string(data.Assignment.RawJSON) != string(raw) { + t.Errorf("RawJSON differs from the file bytes") + } + if data.Assignment.Title != "" { + t.Errorf("Title = %q, want empty so the existing title is preserved on import", data.Assignment.Title) + } + if data.Assignment.TiptapJSON != nil { + t.Error("TiptapJSON must be nil for a quiz assignment") + } + if data.AssignmentQuiz == nil { + t.Fatal("AssignmentQuiz summary is nil") + } + if data.AssignmentQuiz.QuestionCount != 2 || data.AssignmentQuiz.PassThreshold != 2 || !reflect.DeepEqual(data.AssignmentQuiz.QuestionIDs, []string{"q1", "q2"}) { + t.Errorf("summary = %+v, want 2 questions, threshold 2, ids q1 q2", *data.AssignmentQuiz) + } +} + +// Both files present is never resolved by picking one (R11). +func TestReadCompiledModule_BothAssignmentFilesIsError(t *testing.T) { + dir := writeQuizModuleDir(t, prettyQuiz(t), true) + _, err := readCompiledModule(dir) + if err == nil { + t.Fatal("expected an error when both assignment.md and assignment.quiz.json exist") + } + if !strings.Contains(err.Error(), "assignment.md") || !strings.Contains(err.Error(), "assignment.quiz.json") { + t.Errorf("error should name both files: %v", err) + } +} + +func TestReadCompiledModule_InvalidQuizListsEveryIssue(t *testing.T) { + broken := quizEnvelope() + broken["passThreshold"] = float64(9) + broken["questions"].([]interface{})[1].(map[string]interface{})["correctValue"] = "zzz" + raw, _ := json.Marshal(broken) + dir := writeQuizModuleDir(t, raw, false) + + _, err := readCompiledModule(dir) + if err == nil { + t.Fatal("expected validation error") + } + msg := err.Error() + for _, want := range []string{"assignment.quiz.json", "threshold-exceeds-questions", "dangling-correct-value", "q2"} { + if !strings.Contains(msg, want) { + t.Errorf("error %q should mention %q", msg, want) + } + } +} + +func TestReadCompiledModule_DocInQuizSlotIsError(t *testing.T) { + dir := writeQuizModuleDir(t, []byte(`{"type":"doc","content":[]}`), false) + _, err := readCompiledModule(dir) + if err == nil || !strings.Contains(err.Error(), "assignment.md") { + t.Fatalf("a Tiptap doc in assignment.quiz.json should be refused with a pointer at assignment.md, got %v", err) + } +} + +func TestReadCompiledModule_NonQuizTypeInQuizSlotIsError(t *testing.T) { + dir := writeQuizModuleDir(t, []byte(`{"type":"quiz-evidence","version":1}`), false) + if _, err := readCompiledModule(dir); err == nil { + t.Fatal("a non-quiz envelope in assignment.quiz.json should be refused") + } +} + +func TestUpdateModuleContent_QuizPayloadPreservesMetadata(t *testing.T) { + raw := prettyQuiz(t) + data, err := readCompiledModule(writeQuizModuleDir(t, raw, false)) + if err != nil { + t.Fatal(err) + } + existing := &ExistingModuleData{ + Status: "ON_CHAIN", + SLTCount: 1, + Lessons: map[int]map[string]interface{}{}, + Assignment: map[string]interface{}{ + "title": "Module Quiz", + "description": "Pass to continue", + "image_url": "https://cdn/x.png", + "content_json": map[string]interface{}{ + "type": "doc", "content": []interface{}{}, + }, + }, + } + + resp, err := updateModuleContent(context.Background(), nil, "course-1", data, existing, true, true, false) + if err != nil { + t.Fatalf("updateModuleContent: %v", err) + } + assign := payloadAssignment(t, resp) + if !reflect.DeepEqual(assign["content_json"], quizEnvelope()) { + t.Errorf("content_json differs from the quiz file\n got: %v", assign["content_json"]) + } + if assign["title"] != "Module Quiz" || assign["description"] != "Pass to continue" || assign["image_url"] != "https://cdn/x.png" { + t.Errorf("existing metadata not preserved: %v", assign) + } + payload := resp["payload"].(map[string]interface{}) + if _, ok := payload["lessons"]; ok { + t.Error("payload must not carry lessons when the directory has none") + } +} + +// The property #59 asks for: export then import of a quiz module produces a +// payload whose assignment is what the server already holds. +func TestExportImportRoundTrip_QuizIsServerNoOp(t *testing.T) { + dir := t.TempDir() + quiz := quizEnvelope() + if _, err := writeCompiledModule(dir, exportModuleData(wrapAssignment(quiz, "Module Quiz"))); err != nil { + t.Fatal(err) + } + + data, err := readCompiledModule(dir) + if err != nil { + t.Fatalf("readCompiledModule after export: %v", err) + } + existing := &ExistingModuleData{ + Status: "ON_CHAIN", + SLTCount: 1, + Lessons: map[int]map[string]interface{}{}, + Assignment: map[string]interface{}{"title": "Module Quiz", "content_json": quiz}, + } + resp, err := updateModuleContent(context.Background(), nil, "course-1", data, existing, true, true, false) + if err != nil { + t.Fatal(err) + } + assign := payloadAssignment(t, resp) + if !reflect.DeepEqual(assign["content_json"], quiz) { + t.Errorf("round trip changed the quiz\n got: %v\nwant: %v", assign["content_json"], quiz) + } + if assign["title"] != "Module Quiz" { + t.Errorf("title = %v, want the existing title", assign["title"]) + } +} + +// Markdown-only directories must produce the same payload as before: the quiz +// path is additive. +func TestUpdateModuleContent_MarkdownAssignmentUnchanged(t *testing.T) { + dir := writeQuizModuleDir(t, nil, true) + data, err := readCompiledModule(dir) + if err != nil { + t.Fatal(err) + } + if data.AssignmentQuiz != nil { + t.Error("AssignmentQuiz must be nil for a Markdown assignment") + } + resp, err := updateModuleContent(context.Background(), nil, "course-1", data, &ExistingModuleData{Status: "DRAFT", Lessons: map[int]map[string]interface{}{}}, false, true, false) + if err != nil { + t.Fatal(err) + } + assign := payloadAssignment(t, resp) + if assign["title"] != "Essay" { + t.Errorf("title = %v, want H1 title", assign["title"]) + } + cj := assign["content_json"].(map[string]interface{}) + if cj["type"] != "doc" { + t.Errorf("content_json type = %v, want doc", cj["type"]) + } +} + +// A quiz file carries no title. On a module with no assignment the update +// would store an empty title (db-api's Title is a plain string), so the +// import refuses the way import-assignment does (R5). +func TestUpdateModuleContent_QuizWithoutExistingAssignmentRequiresTitle(t *testing.T) { + data, err := readCompiledModule(writeQuizModuleDir(t, prettyQuiz(t), false)) + if err != nil { + t.Fatal(err) + } + existing := &ExistingModuleData{Status: "DRAFT", Lessons: map[int]map[string]interface{}{}} + _, err = updateModuleContent(context.Background(), nil, "course-1", data, existing, false, true, false) + if err == nil || !strings.Contains(err.Error(), "title required") || !strings.Contains(err.Error(), "import-assignment") { + t.Fatalf("err = %v, want the title-required error pointing at import-assignment", err) + } +} + +// A degraded (206) list that does not show the module must refuse to send: +// db-api may be the missing backend and the module may exist with metadata +// the write would null. With --create it must not create a duplicate either. +func TestImportModule_DegradedListRefusesToSend(t *testing.T) { + for _, create := range []bool{false, true} { + stub := &assignmentStub{ + listBodies: []string{chainOnlyListBody(t, "DB API unavailable, showing on-chain data only")}, + listStatus: []int{http.StatusPartialContent}, + } + c, _ := stub.serve(t) + _, err := importModule(ImportParams{ + Ctx: context.Background(), Client: c, Config: &config.Config{}, + ModuleDir: writeQuizModuleDir(t, prettyQuiz(t), false), CourseID: "course-1", + CreateMode: create, DryRun: true, Quiet: true, + }) + if err == nil || !strings.Contains(err.Error(), "DB API unavailable") || !strings.Contains(err.Error(), "nothing was sent") { + t.Fatalf("create=%v: err = %v, want a refusal naming the warning", create, err) + } + if len(stub.posts) != 0 { + t.Errorf("create=%v: a degraded list must never lead to a create or update request", create) + } + } +} + +// The text summary names the quiz digest instead of "Assignment: yes". +func TestCourseImport_TextSummaryReportsQuiz(t *testing.T) { + bin := buildTestBinary(t) + stub := &assignmentStub{listBodies: []string{listBody(t, docAssignment("Quiz"), "")}} + _, url := stub.serve(t) + dir := writeQuizModuleDir(t, prettyQuiz(t), false) + stdout, stderr, code := runCLI(t, bin, url, "course", "import", dir, "--course-id", "course-1", "--dry-run") + if code != 0 { + t.Fatalf("exit = %d; stderr %q", code, stderr) + } + if !strings.Contains(stdout, "Assignment: quiz (2 questions, threshold 2)") { + t.Errorf("stdout = %q, want the quiz summary line", stdout) + } +} + +// Only a missing file means "no assignment". An unreadable one must fail +// rather than silently drop the assignment from the payload. +func TestReadCompiledModule_UnreadableQuizFileIsError(t *testing.T) { + dir := writeQuizModuleDir(t, nil, false) + if err := os.Mkdir(filepath.Join(dir, "assignment.quiz.json"), 0755); err != nil { + t.Fatal(err) + } + _, err := readCompiledModule(dir) + if err == nil || !strings.Contains(err.Error(), "assignment.quiz.json") { + t.Fatalf("a directory in the quiz file's place must fail loudly, got %v", err) + } +} + +// A zero-byte assignment.md has always meant "no assignment" on the Markdown +// path; it must not trip the both-files error beside a real quiz file. +func TestReadCompiledModule_EmptyMarkdownBesideQuizIsNotAConflict(t *testing.T) { + dir := writeQuizModuleDir(t, prettyQuiz(t), false) + if err := os.WriteFile(filepath.Join(dir, "assignment.md"), nil, 0644); err != nil { + t.Fatal(err) + } + data, err := readCompiledModule(dir) + if err != nil { + t.Fatalf("readCompiledModule: %v", err) + } + if data.Assignment == nil || data.Assignment.RawJSON == nil { + t.Fatal("the quiz file must win over an empty assignment.md") + } +} + +// --create with a quiz file and no module would create the module and then +// refuse the update for lack of a title, leaving an empty module behind. The +// condition is known before any request, so the refusal comes first. +func TestImportModule_CreateWithQuizAndNoModuleRefusesBeforeCreate(t *testing.T) { + for _, dry := range []bool{true, false} { + stub := &assignmentStub{listBodies: []string{`{"data":[]}`}} + c, _ := stub.serve(t) + _, err := importModule(ImportParams{ + Ctx: context.Background(), Client: c, Config: &config.Config{}, + ModuleDir: writeQuizModuleDir(t, prettyQuiz(t), false), CourseID: "course-1", + CreateMode: true, DryRun: dry, Quiet: true, + }) + if err == nil || !strings.Contains(err.Error(), "title required") || strings.Contains(err.Error(), "failed to create module") { + t.Fatalf("dry=%v: err = %v, want the title-required refusal before any create", dry, err) + } + if len(stub.posts) != 0 { + t.Errorf("dry=%v: no update may be sent", dry) + } + } +} + +func TestImportResult_AssignmentQuizIsAdditive(t *testing.T) { + b, _ := json.Marshal(ImportResult{Changes: map[string]interface{}{}}) + if strings.Contains(string(b), "assignment_quiz") { + t.Error("assignment_quiz must be omitted for a Markdown assignment") + } + data, err := readCompiledModule(writeQuizModuleDir(t, prettyQuiz(t), false)) + if err != nil { + t.Fatal(err) + } + b, _ = json.Marshal(ImportResult{AssignmentQuiz: data.AssignmentQuiz, Changes: map[string]interface{}{}}) + if !strings.Contains(string(b), `"assignment_quiz":{"question_count":2,"pass_threshold":2,"question_ids":["q1","q2"]}`) { + t.Errorf("assignment_quiz not rendered: %s", b) + } +} diff --git a/cmd/andamio/exitcode_test.go b/cmd/andamio/exitcode_test.go index 451e1c6..e415b8a 100644 --- a/cmd/andamio/exitcode_test.go +++ b/cmd/andamio/exitcode_test.go @@ -43,6 +43,10 @@ const tierLimitBody = `{"error":{"code":"tier_limit_exceeded","message":"maximum // The contract issue #126 turns on: each distinguishable failure gets its own // exit code AND its own kind, and the two never disagree. +// +// Kind "verify" (#165) cannot be produced by `course list` — it needs a write +// followed by a read-back — so its end-to-end pin lives in +// TestImportAssignment_CLIJSONContract (course_import_assignment_test.go). func TestExitCodes_AndKindsAgree(t *testing.T) { bin := buildTestBinary(t) diff --git a/cmd/andamio/exitcodes_help.go b/cmd/andamio/exitcodes_help.go index d42c672..d29cbaf 100644 --- a/cmd/andamio/exitcodes_help.go +++ b/cmd/andamio/exitcodes_help.go @@ -23,6 +23,8 @@ they never disagree — branch on whichever is more convenient. 1 server 5xx response 1 backpressure 408 / 425 / 429 — retry later 1 canceled interrupted, or a --timeout expired + 1 verify the update was accepted, but the read-back did not + confirm the stored value — inspect, don't retry blindly 2 not_found resource doesn't exist (404) 3 auth no credentials, or 401 / 403 4 removed_command command was retired in 1.0 @@ -63,7 +65,14 @@ Exit 7 is new in 1.0. It is classified by the gateway's error code (tier_limit_exceeded), not by HTTP status, so it holds whether the API answers 429 or 403 for that condition. A 429 carrying that code previously reported backpressure; it no longer does. A 429 without it (rate limits, quotas) still -reports backpressure.`, +reports backpressure. + +The verify kind is emitted by commands that read their own write back +(course import-assignment). It is distinct from server: the module WAS +modified. Either the stored value differs from what was sent, or the +read-back was degraded and could not confirm it. Text mode carries no kind, +so a script that must tell "applied but unconfirmed" from "failed" uses +--output json.`, } func init() { diff --git a/cmd/andamio/expired_jwt_test.go b/cmd/andamio/expired_jwt_test.go index e07f4e0..81336d6 100644 --- a/cmd/andamio/expired_jwt_test.go +++ b/cmd/andamio/expired_jwt_test.go @@ -90,7 +90,7 @@ func runCLIInDir(t *testing.T, bin, baseURL, userJWT, workdir string, extraEnv [ return outBuf.String(), errBuf.String(), code } -// Every JWT-required command — jwtAuthPreRunE parents and the seven +// Every JWT-required command — jwtAuthPreRunE parents and the eight // hand-rolled PreRunEs — must reject a locally-expired session with exit 3 / // kind auth BEFORE any request leaves the machine (issue #134). func TestExpiredJWT_FailFastOnJWTRequiredCommands(t *testing.T) { @@ -102,10 +102,11 @@ func TestExpiredJWT_FailFastOnJWTRequiredCommands(t *testing.T) { {"course", "owner", "list"}, {"teacher", "courses"}, {"project", "task", "list", "proj-1"}, - // the seven hand-rolled PreRunEs + // the eight hand-rolled PreRunEs {"course", "export", "course-1", "101"}, {"course", "import", "somedir"}, {"course", "import-all", "somedir"}, + {"course", "import-assignment", "course-1", "101", "quiz.json"}, {"course", "create-module", "--course-id", "course-1"}, {"tx", "build", "/api/v2/tx/x"}, {"tx", "register", "--tx-hash", "h", "--tx-type", "t"}, diff --git a/cmd/andamio/main.go b/cmd/andamio/main.go index b84b239..754d717 100644 --- a/cmd/andamio/main.go +++ b/cmd/andamio/main.go @@ -117,6 +117,8 @@ func init() { // // 0 — success, including an empty but valid result set // 1 error — generic: unexpected, server-side, interrupted, bad input +// 1 verify — a write was accepted but its read-back did not confirm +// the stored value (course import-assignment) // 2 not_found — resource doesn't exist (404) // 3 auth — no credentials, or 401/403 // 4 removed_command — retired in 1.0 (see cmd/andamio/retired.go) @@ -127,8 +129,8 @@ func init() { // // A caller can branch on the exit code alone or on "kind" alone; the two never // disagree. Kinds without a dedicated exit code (server, backpressure, -// canceled) share exit 1 — they are already distinguishable via "kind", and -// splitting them further buys a caller nothing today. +// canceled, verify) share exit 1 — they are already distinguishable via "kind", +// and splitting them further buys a caller nothing today. // // Codes 0-3 predate 1.0 and are load-bearing for existing scripts, so they are // fixed. 4, 5 and 7 are new. 6 is the one change to an existing path: conflicts diff --git a/cmd/andamio/surface_test.go b/cmd/andamio/surface_test.go index fc3fe72..7e60767 100644 --- a/cmd/andamio/surface_test.go +++ b/cmd/andamio/surface_test.go @@ -19,6 +19,10 @@ var update = flag.Bool("update", false, "update golden files instead of comparin var schemaSrcDirs = []string{ ".", "../../internal/config", + // quiz.Summary rides inside ImportResult / ImportAssignmentEnvelope as + // `assignment_quiz` / `assignment`, so its json tags are part of the + // --output json contract and must be pinned here too. + "../../internal/quiz", } // compareOrUpdateGolden either overwrites goldenPath with actual (-update) diff --git a/cmd/andamio/testdata/golden/commands.golden b/cmd/andamio/testdata/golden/commands.golden index b50ba67..2a86468 100644 --- a/cmd/andamio/testdata/golden/commands.golden +++ b/cmd/andamio/testdata/golden/commands.golden @@ -47,6 +47,12 @@ name: create | shorthand: | type: bool name: dry-run | shorthand: | type: bool name: show-payload | shorthand: | type: bool name: sort-order-start | shorthand: | type: int +andamio course import-assignment +name: course | shorthand: | type: string +name: description | shorthand: | type: string +name: dry-run | shorthand: | type: bool +name: show-payload | shorthand: | type: bool +name: title | shorthand: | type: string andamio course intro andamio course lesson andamio course list diff --git a/cmd/andamio/testdata/golden/schema.golden b/cmd/andamio/testdata/golden/schema.golden index e43e3eb..d040e90 100644 --- a/cmd/andamio/testdata/golden/schema.golden +++ b/cmd/andamio/testdata/golden/schema.golden @@ -15,6 +15,12 @@ Config.DevRefreshToken string json:"dev_refresh_token,omitempty" Config.DevRefreshTokenExpiresAt string json:"dev_refresh_token_expires_at,omitempty" Config.SubmitURL string json:"submit_url,omitempty" Config.SubmitHeaders map[string]string json:"submit_headers,omitempty" +Issue.Code string json:"code" +Issue.Message string json:"message" +Issue.QuestionID string json:"question_id,omitempty" +Summary.QuestionCount int json:"question_count" +Summary.PassThreshold int json:"pass_threshold" +Summary.QuestionIDs []string json:"question_ids" CreateModuleResult.CourseID string json:"course_id" CreateModuleResult.ModuleCode string json:"module_code" CreateModuleResult.Title string json:"title" @@ -40,6 +46,7 @@ ImportResult.SltHash string json:"slt_hash,omitempty" ImportResult.LessonCount int json:"lesson_count" ImportResult.HasIntro bool json:"has_introduction" ImportResult.HasAssignment bool json:"has_assignment" +ImportResult.AssignmentQuiz *quiz.Summary json:"assignment_quiz,omitempty" ImportResult.ManifestUsed int json:"manifest_images" ImportResult.ImagesUploaded int json:"images_uploaded,omitempty" ImportResult.FailedImages []string json:"failed_images,omitempty" @@ -50,6 +57,25 @@ ModuleImportSummary.Code string json:"code" ModuleImportSummary.Title string json:"title" ModuleImportSummary.Result *ImportResult json:"result,omitempty" ModuleImportSummary.Error string json:"error,omitempty" +ImportAssignmentEnvelope.CourseID string json:"course_id" +ImportAssignmentEnvelope.ModuleCode string json:"module_code" +ImportAssignmentEnvelope.ModuleStatus string json:"module_status" +ImportAssignmentEnvelope.Assignment ImportAssignmentSummary json:"assignment" +ImportAssignmentEnvelope.DryRun bool json:"dry_run,omitempty" +ImportAssignmentEnvelope.Verified bool json:"verified" +ImportAssignmentSummary.Title string json:"title" +ImportAssignmentSummary.TitleSource string json:"title_source" +ImportAssignmentSummary.QuestionCount int json:"question_count" +ImportAssignmentSummary.PassThreshold int json:"pass_threshold" +ImportAssignmentSummary.QuestionIDs []string json:"question_ids" +assignmentInput.Title string json:"title" +assignmentInput.Description *string json:"description,omitempty" +assignmentInput.ImageURL *string json:"image_url,omitempty" +assignmentInput.VideoURL *string json:"video_url,omitempty" +assignmentInput.ContentJSON json.RawMessage json:"content_json" +assignmentUpdatePayload.CourseID string json:"course_id" +assignmentUpdatePayload.CourseModuleCode string json:"course_module_code" +assignmentUpdatePayload.Assignment assignmentInput json:"assignment" RegisterModuleEnvelope.Action string json:"action" RegisterModuleEnvelope.Status string json:"status" RegisterModuleEnvelope.SltHash string json:"slt_hash" diff --git a/docs/COURSE-LIFECYCLE.md b/docs/COURSE-LIFECYCLE.md index 645eee4..9e02b26 100644 --- a/docs/COURSE-LIFECYCLE.md +++ b/docs/COURSE-LIFECYCLE.md @@ -214,6 +214,30 @@ To import all modules in a course at once: andamio course import-all ./compiled/my-course --course-id <course-id> ``` +### Step 9: Quiz assignments (optional) + +An assignment can be a quiz instead of a Tiptap document: a `{"type": "quiz", "version": 1, ...}` envelope stored in the assignment's `content_json`, which the Andamio app grades client-side. The gateway and db-api treat `content_json` as opaque, so publishing a quiz is an ordinary module update — but `course import` converts Markdown, so quizzes have their own path. + +Publish a quiz file directly: + +```bash +# Preview: validates the envelope, prints question count / threshold / ids, sends nothing +andamio course import-assignment <course-id> 101 quiz.json --dry-run + +# Publish. The existing assignment's title, description, image_url and video_url +# are preserved; --title / --description override. A module with no assignment +# yet requires --title. +andamio course import-assignment <course-id> 101 quiz.json --title "Module Quiz" +``` + +Only the `assignment` key is sent — lessons, SLTs and the introduction are untouched. After the POST the module is re-fetched and `assignment.content_json` is deep-compared to the file; a mismatch, a degraded (206) read-back, or a read-back request that fails exits 1 with `kind: verify`, meaning the update was applied but could not be confirmed and should be inspected. + +**Published modules.** Assignments are editable in any module status; only SLTs lock after DRAFT (db-api's aggregate update soft-skips SLTs on non-DRAFT modules and states that lessons, assignments and introductions remain editable). `import-assignment` therefore works on DRAFT, APPROVED, PENDING_TX and ON_CHAIN modules alike. This statement rests on the gateway and db-api source; see the release notes for the live verification status. + +**In a module directory** the quiz lives at `assignment.quiz.json` in place of `assignment.md`. `course export` writes that file for a quiz module and no `assignment.md`; `course import` validates it and sends it back verbatim, so export followed by import of a quiz module is a server-side no-op. A directory holding both `assignment.md` and `assignment.quiz.json` is refused before any request. A file in that slot that is not a valid v1 quiz (for example an envelope from a newer app version) is preserved on disk but blocks re-import of the whole directory, lessons and introduction included, until it is valid. + +Validation enforces the union of the two apps' rules (`src/lib/quiz/quiz-envelope.ts` in fcb-fan-engagement-app and in andamio-app-v2; see `testdata/quiz/SOURCE.md`). Every violated rule is reported, and there is no bypass flag. + ## Assignment Commitment Lifecycle Assignment commitments track a student's progress through a module. They use different status names and transitions from project task commitments. diff --git a/docs/andamio-cli-context.md b/docs/andamio-cli-context.md index dbfa380..f309131 100644 --- a/docs/andamio-cli-context.md +++ b/docs/andamio-cli-context.md @@ -67,6 +67,7 @@ branch on whichever is more convenient. | 1 | `server` | 5xx response | | 1 | `backpressure` | 408 / 425 / 429 — retry later | | 1 | `canceled` | Interrupted, or a `--timeout` expired | +| 1 | `verify` | The update was accepted, but the read-back did not confirm the stored value — it differs from what was sent, or the read-back was degraded. Emitted by `course import-assignment`. The module WAS modified; inspect it rather than retrying blindly | | 2 | `not_found` | Resource doesn't exist (404) | | 3 | `auth` | No credentials, or 401 / 403 | | 4 | `removed_command` | Command was retired in 1.0 | @@ -173,6 +174,7 @@ andamio course modules "$COURSE_ID" --output json | `course export <course-id> <module>` | jwt | Export module to local files | | `course import <course-id> <module>` | jwt | Import local files to update module | | `course import-all <course-id>` | jwt | Import all modules from compiled directory | +| `course import-assignment <course-id> <module-code> <file.json>` | jwt | Publish a quiz envelope verbatim as the module's assignment `content_json`; validates first, sends only `assignment`, verifies by read-back (`kind: verify` on mismatch). `--dry-run`, `--show-payload`, `--title`, `--description` | ### teacher — Teacher operations @@ -382,7 +384,7 @@ Empty lists return `{"data": []}`. `kind` pairs with the exit code — both come from the same classification, so branch on whichever suits the caller. See [Exit Codes and Error Kinds](#exit-codes-and-error-kinds) for the full table: -`0` success, `1` generic/`server`/`backpressure`/`canceled`, `2` `not_found`, +`0` success, `1` generic/`server`/`backpressure`/`canceled`/`verify`, `2` `not_found`, `3` `auth`, `4` `removed_command`, `5` `unreachable`, `6` `conflict`, `7` `tier_limit`. diff --git a/docs/plans/2026-09-05-001-feat-course-import-assignment-quiz-plan.md b/docs/plans/2026-09-05-001-feat-course-import-assignment-quiz-plan.md new file mode 100644 index 0000000..269b260 --- /dev/null +++ b/docs/plans/2026-09-05-001-feat-course-import-assignment-quiz-plan.md @@ -0,0 +1,329 @@ +--- +title: course import-assignment for quiz content_json - Plan +type: feat +date: 2026-09-05 +artifact_contract: ce-unified-plan/v1 +artifact_readiness: implementation-ready +product_contract_source: ce-plan-bootstrap +execution: code +--- + +# course import-assignment for quiz content_json - Plan + +Realizes andamio-cli#165. Related: andamio-cli#59 (export round-trip), andamio-cli#61 (dry-run output convention), andamio-cli#62 (raw-curl avoidance), andamio-cli#134 (expired-JWT fail-fast), fcb-fan-engagement-app#13 (quiz envelope). + +## Goal Capsule + +- **Objective:** Teachers publish a quiz assignment (a `{"type":"quiz","version":1,…}` envelope in the assignment's `content_json`) from the CLI without raw `curl`, and `course export` followed by `course import` of a quiz module is a server-side no-op. +- **Authority:** This plan, then andamio-cli#165 (its §4 fixes the CLI's validation rule set), then the app's `src/lib/quiz/quiz-envelope.ts` for the wording and control flow of the rules the two share. The Product Contract wins on behavior; a KTD wins on mechanism. +- **Execution profile:** Go CLI, existing Cobra patterns, `go test ./...` is the contract. No new dependencies. +- **Stop conditions:** Stop and report if the gateway's `models.JSONMap` reshapes the envelope so read-back can never match, or if the app's validator has changed since 2026-09-05 in a way that contradicts §4 of the issue. A gateway refusal of an assignment-only update on a non-DRAFT module is not a stop: U6 step 2 owns that outcome. +- **Tail ownership:** The caller (LFG) owns simplify, review, commit, PR, and CI. + +--- + +## Product Contract + +### Summary + +Add `andamio course import-assignment <course-id> <module-code> <file.json>`, a quiz publisher that validates a quiz envelope, sends it verbatim as the module's `assignment.content_json`, touches no other module field, and proves the write by re-fetching and deep-comparing. Teach `course import <dir>` to send `assignment.quiz.json` verbatim and `course export` to write it for a non-`doc` assignment, so the export/import round trip stops destroying quizzes. Validation mirrors the app's `validateQuizDefinition` as read on 2026-09-05, pinned by golden fixtures that guard the CLI against regressing from that snapshot; a rule change in the app is re-mirrored by hand against the recorded source commit. A new error kind `verify` names the read-back mismatch. + +### Problem Frame + +FCB Fan Campus assignments can be quizzes. The app grades them client-side and treats `content_json` as opaque, so nothing backend-side changes. The CLI cannot publish one: `course import` reads only `assignment.md` and converts Markdown to Tiptap, and there is no raw request command. Today the publish step is a hand-built `curl` with the JWT and API key read out of `~/.andamio/config.json`, the same gap andamio-cli#62 closed for module creation. + +The round trip also loses data. `course export` runs `tiptapToMarkdown` on the envelope, matches no node type, and writes an empty `assignment.md`. A later `course import` of that directory replaces the quiz with an empty text assignment. + +### Requirements + +**New command** + +- R1. `course import-assignment <course-id> <module-code> <file.json>` accepts `--course <name>` as an alternative to the positional course id, plus `--title`, `--description`, `--dry-run`, `--show-payload`, and the global `--output`. +- R2. The file is parsed and validated as a quiz envelope before any request. A `type: "doc"` file is rejected with an error stating that Tiptap assignments are authored as `assignment.md` in a module directory and published with `course import <dir>`; any other `type` is rejected as not a quiz. +- R3. The update payload carries only `course_id`, `course_module_code`, and `assignment`. It never carries `lessons`, `slts`, `introduction`, or `title` at the module level. +- R4. `assignment.content_json` is the parsed file verbatim. No transformation. +- R5. `assignment.title` and `assignment.description` come from the existing assignment unless `--title` or `--description` override them. `image_url` and `video_url` are carried from the existing assignment unchanged. If the module has no assignment and `--title` is absent, the command fails with `title required for a module with no existing assignment` before the update request (the module is fetched first to learn whether an assignment exists). +- R6. Auth is the user JWT through `requireUserAuth`, including the andamio-cli#134 expired-token fail-fast. +- R7. After the POST the command re-fetches the module and deep-compares `assignment.content_json` to the file, and compares `title`, `description`, `image_url`, and `video_url` to the values it sent. A mismatch exits non-zero with `kind: verify` in JSON mode, and the text message states that the update was sent but did not read back identical. A degraded read-back (the list endpoint answers 206 with `meta.warning` and no assignment) also exits with `kind: verify`, with a message stating that the update was accepted but the read-back was degraded and names the warning. If the re-fetch fails outright or returns no matching module, the command exits with the underlying error's kind and a message stating that the update was accepted but verification could not run. In every branch `--output json` emits exactly one JSON document. +- R7a. A degraded pre-fetch (206 with `meta.warning`) is an error before the update request that names the warning; the command never infers "no existing assignment" from a degraded read. +- R8. `--dry-run` sends nothing and prints the summary: question count, pass threshold, question ids, and whether the title came from the flag or the existing assignment. `--show-payload` adds the full payload on stderr in text mode, following andamio-cli#61. Nothing reads stdin. +- R9. `--output json` emits `{"course_id","module_code","assignment":{"title","question_count","pass_threshold","question_ids"},"verified":true}`; dry runs emit the same shape with `"dry_run":true` and `"verified":false`. Text output is a one-line summary. + +**Directory import** + +- R10. `course import <dir>` sends `assignment.quiz.json` verbatim as `assignment.content_json` when present, validated per R14, with the existing title, description, image_url, and video_url preserved. +- R11. When both `assignment.md` and `assignment.quiz.json` exist, `course import` fails before any request. The ambiguity is never resolved by picking one. +- R12. The `course import --dry-run` text summary reports `Assignment: quiz (N questions, threshold M)` for a quiz, and the JSON result carries an additive `assignment_quiz` summary object. + +**Export** + +- R13. When an assignment's `content_json.type` is anything other than `"doc"`, `course export` writes it pretty-printed to `assignment.quiz.json`, writes no `assignment.md`, and lists `assignment.quiz.json` in `files`. Introduction and lessons keep their current Markdown behavior. + +**Validation** + +- R14. Validation enforces: `type == "quiz"`; `version` is the integer 1; `questions` is a non-empty array; each question has a string `id` unique across the quiz, a non-empty string `prompt`, an optional string `help`, `options` as an array of at least two `{value, label}` string pairs with unique values, and `correctValue` equal to exactly one option's `value`; `passThreshold` is an integer in `1..len(questions)`; `intro`, if present, is an object with `type: "doc"`. +- R15. Every violated rule is reported, one line each, naming the question id where one applies. Validation never stops at the first failure and never panics on malformed elements. +- R16. There is no bypass flag. A rule the app adds that the CLI lacks is fixed by updating the CLI. +- R17. Golden fixtures under `testdata/quiz/` cover every rule in R14, valid and invalid. The cases mirrored from the app's `quiz-envelope.test.ts` are labeled `app`, and on those the Go validator emits the same issue codes as the app; the three rules the app does not enforce (KTD4) are labeled `cli-additional`. A `testdata/quiz/SOURCE.md` records the app commits the fixtures were copied from, so a rule change in the app has a concrete re-sync point. + +**Failure contract and docs** + +- R18. `verify` is a new `kind` in the `--output json` error envelope. It shares exit 1 and is documented in `andamio help exit-codes`, the `main.go` comment block, README, `docs/andamio-cli-context.md`, and the CLAUDE.md Failure Contract. +- R19. README command list and examples, `docs/COURSE-LIFECYCLE.md` (a quiz assignment step), the CLAUDE.md command table and its mirror in `docs/andamio-cli-context.md`, and the CHANGELOG `[Unreleased]` section are updated in the same PR. +- R20. Command help and `docs/COURSE-LIFECYCLE.md` state the behavior on a published (non-DRAFT) module, and state that a directory holding a non-v1 or non-quiz `assignment.quiz.json` cannot be re-imported until that file is a valid quiz (KTD6). + +### Scope Boundaries + +- No quiz support for `introduction` or lessons. Only assignments carry quizzes today. +- No `--skip-validation` or any other bypass. +- No new exit code. `verify` shares exit 1 (see KTD2). +- No change to how `course assignment <course-id> <module-code>` renders a quiz; it already passes the gateway payload through. +- No gateway or db-api change. Both treat `content_json` as opaque `jsonb`. + +#### Deferred to Follow-Up Work + +- fcb-fan-engagement-app: point the publish section of `docs/quiz-content-format.md` at the new command (named in the issue as an app-repo follow-up). +- andamio-docs `content/docs/apps-tooling/cli/index.mdx`: tracked with the post-1.0 docs update, andamio-docs#64. +- Board placement for andamio-cli issues is a Dev Circle call, not part of this change. + +--- + +## Planning Contract + +### Key Technical Decisions + +- KTD1. **Hyphenated verb at the `course` level: `course import-assignment`.** Matches `create-module` and `import-all`. Not `course assignment import`: `course assignment` is a leaf GET with `ExactArgs(2)` in `cmd/andamio/course.go`, and turning it into a group would make a module code named `import` ambiguous. Governs R1. +- KTD2. **`verify` is a new kind on exit 1, not a new exit code.** `cmd/andamio/main.go` states the rule: kinds without a dedicated exit code share exit 1 because they are already distinguishable via `kind`. A read-back mismatch is one such kind. Adding a kind is permitted by the stability statement in `exitcodes_help.go`; renaming or re-mapping one is not. Governs R7, R18. +- KTD3. **Verbatim in value, structural in comparison.** The payload carries `content_json` as `json.RawMessage` so the CLI never rebuilds the envelope from a typed struct; `encoding/json` still compacts and escapes it on marshal, and the gateway re-decodes it into a map before db-api stores it, so equality is structural end to end, never byte-for-byte. Every comparison in tests and in the read-back unmarshals both sides into `interface{}` and uses `reflect.DeepEqual`. Governs R4, R7. +- KTD4. **One validator in `internal/quiz`, used by all three commands.** Recognition (`type`, `version`, `questions` array) is separate from validity, mirroring the app's `isQuizContentEnvelope` vs `validateQuizDefinition`. Issue codes reuse the app's names (`unsupported-version`, `empty-questions`, `invalid-threshold`, `threshold-exceeds-questions`, `missing-correct-value`, `dangling-correct-value`, `duplicate-option-values`, `duplicate-question-ids`, `too-few-options`, `malformed-question`). The three checks in R14 that the app's runtime validator does not enforce today (non-empty string `prompt`, optional string `help`, `intro` is a `doc` object) get CLI-side codes (`malformed-prompt`, `malformed-help`, `malformed-intro`) and are labeled `cli-additional` in the fixtures so the two rule sets stay distinguishable. Governs R14–R17. +- KTD5. **Carry every assignment metadata field from the existing record.** db-api's `processAssignmentUpdate` (`internal/handlers/course_v2_module.go` in andamio-db-api-go) assigns `Title`, `Description`, `ImageURL`, and `VideoURL` from the input unconditionally, so an omitted field is nulled, not preserved. `course import` already merges these in `updateModuleContent`; `import-assignment` does the same. Governs R5, R10. +- KTD6. **Detection key is `content_json.type != "doc"`, on both sides.** Export writes `assignment.quiz.json` for any non-`doc` assignment (per issue §3), so the stored value is preserved on disk instead of being flattened to an empty `assignment.md`. Import validates that file as a v1 quiz per R2 and R14 before sending, so a `version: 2` quiz or an unknown envelope type fails loudly at parse time, and that failure blocks lesson and introduction updates for the same directory until the file is a valid quiz. The trade-off is deliberate: preserved-and-blocked beats silently destroyed. R20 states it in help and docs. Governs R2, R13, R20. +- KTD7. **`assignment.quiz.json` and `assignment.md` are mutually exclusive inputs, enforced in `readCompiledModule`.** The check runs at parse time, before `fetchExistingModule`, so the conflict never costs a request. Governs R11. +- KTD8. **Verify runs against the same teacher list endpoint the fetch used, and both calls check `meta.warning`.** `fetchExistingModule` already returns the assignment with inline `content_json`; a second call after the POST is the read-back. The list endpoint is a merged read that answers 206 with `meta.warning` and chain-only modules (no `assignment`) when db-api is degraded, and `client.Post` accepts every 2xx, so a degraded response must be recognized via `metaWarning` in `helpers.go` rather than read as "no assignment". No new endpoint. Governs R7, R7a. + +### High-Level Technical Design + +The new command's lifecycle has seven steps with two exits before the network and one typed failure after it. + +```mermaid +flowchart TB + A[Read file.json] --> B{Recognize envelope} + B -->|type doc| X1[error: use course import] + B -->|other type| X2[error: not a quiz] + B -->|type quiz| C[Validate R14 - collect all issues] + C -->|issues| X3[error: one line per issue] + C -->|clean| D[fetchExistingModule] + D --> E[Build payload: assignment only, metadata merged per KTD5] + E --> F{--dry-run?} + F -->|yes| G[Print summary, exit 0] + F -->|no| H[POST course-module/update] + H --> I[fetchExistingModule again] + I --> J{DeepEqual content_json?} + J -->|yes| K[Print result, verified true] + J -->|no| X4[VerifyError - kind verify, exit 1] +``` + +File detection on the round trip: + +| Side | Condition | Writes / reads | Notes | +|------|-----------|----------------|-------| +| export | `content_json.type == "doc"` | `assignment.md` | unchanged | +| export | any other `type` | `assignment.quiz.json` | pretty-printed, no `assignment.md` | +| import | only `assignment.md` | Markdown to Tiptap | unchanged | +| import | only `assignment.quiz.json` | validate, send verbatim | title and metadata from existing | +| import | both | hard error before any request | KTD7 | + +### Assumptions + +This plan was authored without synchronous user confirmation. The items below fill gaps in the issue and should be reviewed before or during implementation. + +- `verify` maps to exit 1 (KTD2). The issue says only "non-zero". If a dedicated code is wanted, it is a one-line change in `main.go` plus the documentation surfaces in R18, but only before the release that ships `verify`; after that, the stability statement KTD2 cites bars re-mapping it. +- The three `cli-additional` validation checks in KTD4 are hard errors, as issue §4 specifies, even though the app's runtime validator does not enforce them. Consequence: a quiz the app would render (for example an `intro` whose `type` is not `doc`) is refused by `course import` and `import-assignment`. The plan follows the issue because it is the spec for the CLI; downgrading those three to non-blocking stderr warnings is the alternative if that consequence is unwanted, and it needs no bypass flag. This is the one decision in the plan flagged for the user. +- The `--output json` dry-run shape reuses the success envelope with `dry_run: true` and `verified: false` rather than a separate shape. +- The `assignment_quiz` field on `ImportResult` is additive with `omitempty`, so existing `course import --output json` consumers see no change for Markdown assignments. `testdata/golden/schema.golden` will change and needs `-update`. +- Published-module behavior (R20): db-api's `AggregateUpdateModule` states in source that lessons, assignments, and introductions "remain editable in any status" and soft-skips only SLTs. The plan expects the preprod check to confirm this. If no preprod JWT and test course are available to the implementer, U6 records the source-level evidence in help text and the lifecycle doc with wording that names the evidence, and the PR states that the live check was not run. +- Golden fixtures live at repo-root `testdata/quiz/` as the issue names, read by the `internal/quiz` tests via a relative path. + +### Sources and Research + +- `cmd/andamio/course_import.go`: `readCompiledModule` (file parsing, H1 extraction), `fetchExistingModule` (teacher list endpoint, returns `Assignment` map with inline `content_json`), `updateModuleContent` (metadata merge, dry-run and `--show-payload` convention), `ImportResult`. +- `cmd/andamio/course_export.go`: `fetchModuleData` (wraps assignment as `data.content.{content_json,title}`), `writeCompiledModule` (writes `assignment.md`, appends to `result.Files`), `convertContentToMarkdown`. +- `cmd/andamio/course_create_module.go`: the andamio-cli#62 precedent for a command that replaces a raw `curl`. +- `cmd/andamio/helpers.go`: `requireUserAuth`, `withTierLimitRemedy` (command-layer error decoration pattern). +- `internal/apierr/errors.go`: `Kind*` constants and the single `Kind(err)` mapper; `cmd/andamio/main.go` exit switch; `cmd/andamio/exitcodes_help.go`. +- `cmd/andamio/publish_module_test.go`: `stubPublishServer` is the httptest pattern for a route-checked stub with captured request body. +- `cmd/andamio/surface_test.go`: `TestCommandSurfaceGolden` and `TestSchemaSurfaceGolden` fail on a new command or a new JSON-tagged field until run with `-update`. +- fcb-fan-engagement-app `src/lib/quiz/quiz-envelope.ts` and `src/lib/quiz/quiz-envelope.test.ts` (read 2026-09-05): reference validator and its test cases; `docs/quiz-content-format.md`: the minimal publish body. +- andamio-db-api-go `internal/handlers/course_v2_module.go`: `AggregateUpdateModule` soft-skips SLTs only; `AggregateAssignmentInput` has `Title string`, `ContentJSON models.JSONMap`; `processAssignmentUpdate` overwrites every field (KTD5). +- `docs/solutions/logic-errors/export-import-round-trip-title-preservation.md` and `docs/solutions/integration-issues/cli-course-import-app-parity-and-payload-alignment.md`: the replace-all semantics of the aggregate update and why the CLI merges metadata. +- `docs/solutions/architecture/typed-output-envelope-with-gateway-state-fallbacks.md`: typed struct with JSON tags for every `--output json` envelope, never `map[string]interface{}`. +- `docs/solutions/architecture/cli-composability-audit-and-fix.md`: progress to stderr, data to stdout, typed errors drive exit codes. + +--- + +## Implementation Units + +### U1. Quiz envelope package and golden fixtures + +- **Goal:** One recognizer, validator, and summarizer for quiz envelopes, with fixtures that pin the rule set against the app. +- **Requirements:** R2, R14, R15, R16, R17. KTD4. +- **Dependencies:** none. +- **Files:** create `internal/quiz/quiz.go`, `internal/quiz/quiz_test.go`, `testdata/quiz/valid/*.json`, `testdata/quiz/invalid/*.json` with an expected-issues sidecar per invalid case (one issue code per line, plus a `source: app|cli-additional` marker), and `testdata/quiz/SOURCE.md` recording the app commits the fixtures mirror: `quiz-envelope.ts` at `3842a31f9b7a83bc8d7b4273dcb9dfa6b551ed8c` and `quiz-envelope.test.ts` at `77aa83366ef6b2df2e9c4624d567b890945c87f3` in fcb-fan-engagement-app. +- **Approach:** + 1. `Recognize(raw []byte)` returns one of: doc, quiz, other, not-an-object. It decodes into `map[string]interface{}` and inspects `type`; quiz recognition also requires numeric `version` and array `questions`, matching `isQuizContentEnvelope`. + 2. `Validate(env map[string]interface{}) []Issue` walks every rule in R14 and appends an `Issue{Code, Message, QuestionID}` per violation. It mirrors the app's control flow: empty `questions` returns early; a malformed question is reported and skipped, never dereferenced; well-formed questions alongside malformed ones still validate. Integer checks treat JSON numbers as integral only when the float64 has no fractional part. + 3. `Summary(env) Summary{QuestionCount, PassThreshold, QuestionIDs}` feeds the dry-run summary and the JSON envelopes of U3 and U4. + 4. Fixtures: one valid case per shape (minimal, with `intro`, with `help`), one invalid case per rule including every `quiz-envelope.test.ts` case (null question entry, options as a string, non-record option entries, non-string id, multi-issue definition). The test loads every fixture and asserts the exact issue-code set. +- **Patterns to follow:** `internal/cardano` and `internal/submit` for a small focused package with no `cmd` imports; the app's `validateQuizDefinition` for control flow and message wording. +- **Test scenarios:** + - Valid two-question envelope returns no issues and a summary of count 2, threshold 2, ids `q1`,`q2`. + - `version: 99` yields `unsupported-version` and still validates the rest. + - `version: 1.5` yields `unsupported-version` (non-integral). + - Empty `questions` yields exactly `empty-questions`. + - `passThreshold: 0` yields `invalid-threshold`; `passThreshold: 3` with two questions yields `threshold-exceeds-questions`; `passThreshold: 1.5` yields `invalid-threshold`. + - Missing `correctValue` yields `missing-correct-value` with `QuestionID` set; `correctValue: "zzz"` yields `dangling-correct-value`. + - Duplicate option values in one question yields `duplicate-option-values`; duplicate question ids yields `duplicate-question-ids`. + - One option yields `too-few-options`. + - `questions: [null]`, options as `"ab"`, an option entry `"a"`, and `id: 7` each yield `malformed-question` without panicking. + - `[null, q1, q2]` yields exactly one issue. + - Empty `prompt` yields `malformed-prompt`; `help: 3` yields `malformed-help`; `intro: {"type":"paragraph"}` and `intro: "x"` yield `malformed-intro`; `intro` absent or `null` yields nothing. + - Threshold 9 plus dangling `correctValue` yields at least two issues, each with a non-empty message. + - `Recognize` on `{"type":"doc","content":[]}` returns doc; on `{"type":"quiz-evidence"}` returns other; on `[…]` and `"quiz"` returns not-an-object; on `{"type":"quiz","version":1}` without `questions` returns other. +- **Verification:** `go test ./internal/quiz` passes; every R14 rule has at least one invalid fixture; every `quiz-envelope.test.ts` validity case has a fixture with the same expected code. + +### U2. `verify` error kind + +- **Goal:** A typed error for read-back mismatch that the exit switch and JSON envelope classify as `verify`. +- **Requirements:** R7, R18. KTD2. +- **Dependencies:** none. +- **Files:** modify `internal/apierr/errors.go`, `internal/apierr/errors_test.go`, `cmd/andamio/exitcodes_help.go`, `cmd/andamio/main.go` (comment block only), `README.md` (exit-code table), `docs/andamio-cli-context.md` (exit-code table and any second mention), `CLAUDE.md` (Failure Contract table). +- **Approach:** + 1. Add `KindVerify = "verify"` and `VerifyError{Path, Message string}` where `Path` names what was compared (`assignment.content_json`). `Error()` states that the update was sent and accepted but the stored value did not read back identical, so the caller knows the module was modified. + 2. Add the `errors.As` case to `Kind` after `removed` and before `notFound`. No `main.go` switch change: exit 1 is the default. + 3. Add the row to every table listed in Files. The help text names the mismatch as a distinct outcome from `server` and says the update was applied. +- **Patterns to follow:** `RemovedCommandError` for a typed error carrying more than a message; `TestKind_ClassifiesEachTypedError` for the mapper test. +- **Test scenarios:** + - `Kind(&VerifyError{})` returns `verify`; wrapped with `fmt.Errorf("%w")` and inside `ReportedError` it still returns `verify`. + - The exit-code table test in `cmd/andamio/exitcode_test.go` cannot drive this kind through `course list`; the end-to-end check lives in U3's command test instead, and this unit adds a comment in `exitcode_test.go` pointing there. + - `TestExitCodes_TextModeCarriesNoKind` posture holds for `VerifyError`: text mode prints the message on stderr with no `kind`. +- **Verification:** `go test ./internal/apierr ./cmd/andamio` passes; `andamio help exit-codes` lists `1 verify`; the five documentation surfaces agree. + +### U3. `course import-assignment` command + +- **Goal:** Publish a validated quiz envelope as `assignment.content_json`, touching nothing else, and prove it by read-back. +- **Requirements:** R1–R9, R7a, R20. KTD1, KTD3, KTD5, KTD8. +- **Dependencies:** U1, U2. +- **Files:** create `cmd/andamio/course_import_assignment.go`, `cmd/andamio/course_import_assignment_test.go`; modify `cmd/andamio/expired_jwt_test.go` (add the command to the hand-rolled PreRunE table in `TestExpiredJWT_FailFastOnJWTRequiredCommands`), `cmd/andamio/testdata/golden/commands.golden` and `cmd/andamio/testdata/golden/schema.golden` via `-update`. +- **Approach:** + 1. Register on `courseCmd` with `Use: "import-assignment <course-id> <module-code> <file.json>"`, `Args: cobra.RangeArgs(2, 3)` so `--course <name>` can replace the course-id positional the way `course export` does, and `PreRunE: requireUserAuth`. + 2. Read and recognize the file (U1). A doc yields an error naming `course import`; other types yield a not-a-quiz error; a quiz with issues yields one error whose message is the issue lines joined by newlines. + 3. `fetchExistingModule` for status and the existing assignment. This needs `fetchExistingModule` to also surface the envelope's `meta.warning` (a small signature extension, or a sibling helper that returns it); a non-empty warning on the pre-fetch is the R7a error. Resolve title: `--title`, else existing title, else the R5 error. Resolve description the same way without the error. Carry `image_url` and `video_url` from existing when non-empty. + 4. Build a typed payload struct with `CourseID`, `CourseModuleCode`, and `Assignment{Title, Description *string, ImageURL *string, VideoURL *string, ContentJSON json.RawMessage}`. Nothing else. + 5. Dry run: print the summary in text mode; print the payload on stderr when `--show-payload`; emit the JSON envelope with `dry_run: true`; return before any POST. + 6. POST to `/api/v2/course/teacher/course-module/update`, then `fetchExistingModule` again. Three failure branches, all printing no result envelope so `main.go` emits the single error document: the re-fetch errors or finds no module, so wrap the underlying error with a message that the update was accepted but verification could not run, keeping its kind; the re-fetch is degraded (`meta.warning` non-empty), so return `VerifyError` whose message names the warning and says the stored value could not be confirmed; the re-fetch succeeds but `reflect.DeepEqual` of the decoded file against the decoded `assignment.content_json` fails, or `title`/`description`/`image_url`/`video_url` differ from what was sent, so return `VerifyError` saying the update was sent but did not read back identical. + 7. Define `ImportAssignmentEnvelope` as a typed struct: `course_id`, `module_code`, `module_status`, `assignment{title, title_source, question_count, pass_threshold, question_ids}`, `dry_run` (omitempty), `verified`. + 8. Help text states published-module behavior per U6's finding and points at `course import` for Markdown assignments. +- **Patterns to follow:** `course_create_module.go` for a `course`-level command replacing raw `curl`; `updateModuleContent` for the `--dry-run` and `--show-payload` split (summary on stdout in text mode, payload on stderr); `RegisterModuleEnvelope` in `course_teacher_ops.go` for a typed envelope; `stubPublishServer` for the test server. +- **Test scenarios:** + - Valid quiz against a stub that serves the module list (existing assignment with title `Quiz` and a `doc` body) and accepts the update, then serves the list with the new `content_json`: exit 0, POST body contains exactly `course_id`, `course_module_code`, `assignment`; the captured `assignment.content_json`, decoded, is `reflect.DeepEqual` to the decoded file (the file is pretty-printed, so byte equality is not expected); `assignment.title` is `Quiz`; JSON output has `verified: true`, `question_count` 2, `pass_threshold` 2. + - Existing assignment has `description`, `image_url`, `video_url`: all three appear in the POST body unchanged. + - `--title "New"` overrides the existing title and `title_source` reports `flag`. + - Module with no assignment and no `--title`: the R5 error, and the stub records no POST. + - Stub returns the old `content_json` on re-fetch: exit 1, JSON stdout is exactly one document with `"kind":"verify"` and no result envelope, text stderr says the update was sent but did not read back identical. + - Stub returns the new `content_json` but a different `title` on re-fetch: exit 1, `kind: verify`. + - Stub answers the pre-fetch with 206, `meta.warning`, and a module carrying no `assignment`: no POST recorded, error names the warning, not the R5 title error. + - Stub accepts the update and answers the re-fetch with 206, `meta.warning`, and no `assignment`: exit 1, `kind: verify`, message names the warning and says the update was accepted, not "did not read back identical". + - Stub accepts the update and returns 503 on the re-fetch: exit 1, `kind: server`, stderr text says the update was accepted but verification could not run, JSON stdout is one document. + - The command appears in the `TestExpiredJWT_FailFastOnJWTRequiredCommands` table as the eighth hand-rolled PreRunE. + - `{"type":"doc",…}` file: error mentions `course import`, no request. + - `{"type":"quiz-evidence",…}` file: not-a-quiz error, no request. + - Invalid quiz with three violations: stderr lists three lines, no request. + - `--dry-run`: no POST recorded; text stdout holds the one-line summary; `--show-payload` adds the payload on stderr; JSON has `dry_run: true`, `verified: false`. + - Missing file and malformed JSON: exit 1 with a message naming the path, no request. + - Expired JWT fixture (reuse the `expired_jwt_test.go` helper): exit 3, `kind: auth`, no request. + - 404 from the update: exit 2, `kind: not_found`; 401: exit 3. + - Nothing is read from stdin: the test closes stdin and the command still completes. +- **Verification:** `go test ./cmd/andamio` passes including `TestCommandSurfaceGolden` and `TestSchemaSurfaceGolden` after `-update`; `andamio course import-assignment --help` shows all flags and the published-module statement. + +### U4. `course import <dir>` recognizes `assignment.quiz.json` + +- **Goal:** Directory import sends a quiz assignment verbatim and refuses an ambiguous directory. +- **Requirements:** R10, R11, R12. KTD4, KTD5, KTD7. +- **Dependencies:** U1. +- **Files:** modify `cmd/andamio/course_import.go`, `cmd/andamio/course_import_test.go`, `cmd/andamio/surface_test.go` (append `../../internal/quiz` to `schemaSrcDirs` so `quiz.Summary`'s JSON tags land in the golden); `cmd/andamio/testdata/golden/schema.golden` via `-update`. +- **Approach:** + 1. In `readCompiledModule`, stat both `assignment.md` and `assignment.quiz.json` first; both present is an error before anything else is parsed. + 2. For `assignment.quiz.json`: recognize and validate (U1); a `doc` or other type or any issue is an error naming the file. Store the raw bytes on `ContentSection` in a new `RawJSON json.RawMessage` field with `Title` empty so the existing "empty title keeps the existing title" merge applies. Store the `quiz.Summary` on `ImportData` for the result. + 3. In `updateModuleContent`, when `RawJSON` is set, use it for `assignment.content_json` instead of `TiptapJSON`. Metadata merge is unchanged. + 4. `ImportResult` gains `AssignmentQuiz *quiz.Summary` tagged `assignment_quiz,omitempty`. The text summary prints `Assignment: quiz (N questions, threshold M)` in place of `Assignment: yes` when set. `import-all` inherits this through `importModule`. +- **Patterns to follow:** the introduction and assignment blocks in `readCompiledModule` and `updateModuleContent`; `checkSilentSLTFailure` for a parse-time guard with a clear message. +- **Test scenarios:** + - Directory with `outline.md` and `assignment.quiz.json` (valid): `readCompiledModule` returns `Assignment.RawJSON` equal to the file bytes, empty `Title`, and a summary with the right counts. + - Directory with both assignment files: `readCompiledModule` returns an error naming both files; `fetchExistingModule` is never reached (unit test at the parse layer). + - Directory with an invalid quiz file: error lists every violated rule and names `assignment.quiz.json`. + - `updateModuleContent` with a quiz section and an existing assignment carrying title and description: payload `assignment.content_json` equals the raw bytes, `title` and `description` come from existing. + - `updateModuleContent` in dry-run: `payload.assignment.content_json` is the quiz, and no lessons or slts keys appear when the directory has none. + - Existing Markdown-only directories produce byte-identical payloads to before this change (regression guard on an existing fixture). + - Text summary shows `Assignment: quiz (2 questions, threshold 2)`; JSON result has `assignment_quiz` for a quiz and omits it for Markdown. +- **Verification:** `go test ./cmd/andamio` passes; an existing Markdown module import fixture is unchanged. + +### U5. `course export` writes `assignment.quiz.json` + +- **Goal:** A non-`doc` assignment survives export, and export followed by import is a no-op on the server. +- **Requirements:** R13. KTD6. +- **Dependencies:** U4 (for the round-trip test). +- **Files:** modify `cmd/andamio/course_export.go`, `cmd/andamio/course_export_test.go`. +- **Approach:** + 1. In `writeCompiledModule`, before converting the assignment to Markdown, read `content_json.type` from the wrapped assignment. If it is not `"doc"`, `json.MarshalIndent` the `content_json` map to `assignment.quiz.json`, append that name to `result.Files`, and skip `assignment.md` and image collection for the assignment. + 2. Whichever assignment file is written, remove the other one if it already exists in the output directory. Export only reaches an existing directory under `--force`, so the removal is inside the overwrite the user granted, and it keeps a re-export from producing the R11 conflict. + 3. Update the directory tree in the export command's `Long` help to list `assignment.quiz.json`, and add the R20 statement that a non-v1 or non-quiz file blocks re-import of the directory. +- **Patterns to follow:** the existing `assignment.md` block in `writeCompiledModule`; `writeFileAtomic`. +- **Test scenarios:** + - `ModuleData` with a quiz assignment: the output directory contains `assignment.quiz.json` and no `assignment.md`; `Files` lists `assignment.quiz.json`; the file decodes to a map deep-equal to the input `content_json`. + - `ModuleData` with a `doc` assignment: behavior unchanged, `assignment.md` written with the H1 title. + - `ModuleData` with no assignment: neither file written. + - Output directory already holding a stale `assignment.md`, exported again with a quiz assignment: afterwards only `assignment.quiz.json` exists, and `readCompiledModule` on the directory does not hit the R11 error. The reverse transition removes a stale `assignment.quiz.json`. + - Round trip: write a quiz module with `writeCompiledModule`, read it back with `readCompiledModule`, and build the payload with `updateModuleContent` against an `ExistingModuleData` carrying the same assignment; the payload's `assignment.content_json` is deep-equal to the exported `content_json`, and `title` equals the existing title. +- **Verification:** `go test ./cmd/andamio` passes; the round-trip scenario is a named test that andamio-cli#59 can cite. + +### U6. Docs, changelog, and published-module verification + +- **Goal:** Every documentation surface names the new command and the round-trip behavior, and the published-module statement rests on evidence. +- **Requirements:** R19, R20. +- **Dependencies:** U3, U4, U5. +- **Files:** modify `README.md`, `docs/COURSE-LIFECYCLE.md`, `CHANGELOG.md`, `CLAUDE.md` (course command table; Auth Flow sentence "the seven hand-rolled PreRunEs" becomes eight, naming `course import-assignment`), `docs/andamio-cli-context.md` (mirrored command table), `cmd/andamio/course_import_assignment.go` (help text). +- **Approach:** + 1. Preprod check: with a preprod JWT and a test course, run `course import-assignment --dry-run` and then the real command against a module in each of DRAFT and ON_CHAIN status. Record the outcome. If credentials are unavailable, state in the help text and lifecycle doc that db-api's aggregate update edits assignments in any status and soft-skips only SLTs, and say in the PR body that the live check did not run. + 2. Help text and `docs/COURSE-LIFECYCLE.md`: add a "Quiz assignments" step after Step 8 with the `import-assignment` example, the `assignment.quiz.json` directory convention, the both-files error, the R20 re-import statement, and the published-module statement from step 1. If the gateway refused, the command must surface that in one line rather than a raw 4xx; add that handling to U3 and re-run its tests. + 3. README: add `course import-assignment` to the Author list and an example under Course Import/Export, list `assignment.quiz.json` in the directory format, and add the `verify` row (U2 owns the table text). + 4. CHANGELOG `[Unreleased]`: an Added entry for the command, a Fixed entry for the export data-loss hazard naming andamio-cli#59, and an Added note for the `verify` kind under the same envelope-stability paragraph style as 1.0.0. + 5. CLAUDE.md and `docs/andamio-cli-context.md`: add the command row to both `course` tables and a sentence on the quiz file convention next to the Export/Import pattern. +- **Test expectation:** none for docs. The preprod check is a manual verification recorded in the PR. +- **Verification:** `go build` passes; `andamio course import-assignment --help`, README, and the lifecycle doc make the same published-module statement; `scripts/changelog-section.sh` is not affected because the entry stays under `[Unreleased]`. + +--- + +## Verification Contract + +| Gate | Command | Applies to | +|------|---------|------------| +| Build | `go build -o andamio ./cmd/andamio` | all units | +| Full suite | `go test ./...` | all units | +| Contract guards | `go test ./cmd/andamio -run 'TestExitCodes|TestNoSourceFileCallsARetiredRoute|TestCommandSurfaceGolden|TestSchemaSurfaceGolden'` | U2, U3, U4 | +| Golden refresh (intentional changes only) | `go test ./cmd/andamio -run 'TestCommandSurfaceGolden|TestSchemaSurfaceGolden' -update` | U3, U4 | +| Quiz fixtures | `go test ./internal/quiz` | U1 | +| Help surface | `./andamio course import-assignment --help` and `./andamio help exit-codes` | U2, U3, U6 | +| Preprod (manual, when credentials exist) | `./andamio course import-assignment <course> <module> testdata/quiz/valid/minimal.json --dry-run` then without `--dry-run` on a DRAFT and an ON_CHAIN module | U6 | + +Composability rules hold throughout: no stdin reads, progress on stderr, data on stdout, `--output json` emits exactly one JSON document. + +--- + +## Definition of Done + +- All six units complete; `go test ./...` green; golden files refreshed only for the intended additions (one new command, its flags, the `ImportAssignmentEnvelope` fields, `assignment_quiz`, and the `quiz.Summary` fields now scanned from `internal/quiz`). +- Every acceptance criterion in andamio-cli#165 maps to a passing test or, for the preprod check, to a statement in the PR body. +- `export` then `import` of a quiz module produces a payload whose `assignment.content_json` deep-equals the exported value (U5 round-trip test). +- The `verify` kind is documented in all five surfaces named in R18. +- README, `docs/COURSE-LIFECYCLE.md`, CLAUDE.md, and CHANGELOG `[Unreleased]` are updated in the same PR. +- No dead code from abandoned approaches remains in the diff. diff --git a/internal/apierr/errors.go b/internal/apierr/errors.go index fa35f4e..3e22a59 100644 --- a/internal/apierr/errors.go +++ b/internal/apierr/errors.go @@ -31,6 +31,7 @@ const ( KindUnreachable = "unreachable" KindCanceled = "canceled" KindTierLimit = "tier_limit" + KindVerify = "verify" KindError = "error" ) @@ -65,11 +66,14 @@ func Kind(err error) string { backpressure *BackpressureError removed *RemovedCommandError network *NetworkError + verify *VerifyError ) switch { case errors.As(err, &removed): return KindRemovedCommand + case errors.As(err, &verify): + return KindVerify case errors.As(err, ¬Found): return KindNotFound // Before auth and backpressure: a tier cap arrives on 429 today and 403 @@ -210,6 +214,37 @@ func (e *RemovedCommandError) Error() string { return "'andamio " + e.Command + "' was removed in Andamio CLI 1.0.\n" + e.Guidance } +// VerifyError is returned when a write was accepted by the gateway but the +// CLI's read-back could not confirm the stored value: the re-fetched value +// differs from what was sent, or the re-fetch came back degraded (206 with +// meta.warning and no content to compare). main.go maps this to exit 1 — it +// shares the generic code because "kind" already distinguishes it, per the +// rule in main.go's exit-code comment. +// +// This exists because the alternative outcomes all mislead. Reporting success +// would hide that the stored value is not what the caller asked for; +// reporting `server` or `error` would hide that the module WAS modified. A +// caller seeing `verify` knows two things at once: the update was applied, +// and it must be inspected. Path names what was compared +// ("assignment.content_json"); Message carries the specific mismatch or the +// gateway's degradation warning. +type VerifyError struct { + Path string + Message string + // Err is the underlying cause when the read-back request itself failed + // (a transport error, a 5xx, an expired token). Kind classifies the + // whole as verify — the write WAS applied, so "unreachable" or "auth" + // would send a script down the wrong branch — while Unwrap keeps the + // cause inspectable with errors.Is / errors.As. + Err error +} + +func (e *VerifyError) Error() string { + return fmt.Sprintf("update was accepted, but %s could not be verified: %s", e.Path, e.Message) +} + +func (e *VerifyError) Unwrap() error { return e.Err } + // ReportedError wraps an error whose output has already been printed to stdout // (e.g., a structured JSON result). main.go should set the exit code from the // wrapped error but skip printing a second error message. diff --git a/internal/apierr/errors_test.go b/internal/apierr/errors_test.go index 304fc26..7581b9a 100644 --- a/internal/apierr/errors_test.go +++ b/internal/apierr/errors_test.go @@ -21,6 +21,7 @@ func TestKind_ClassifiesEachTypedError(t *testing.T) { {"tier limit", &TierLimitError{Status: 429, Code: "tier_limit_exceeded", Message: "cap"}, KindTierLimit}, {"removed command", &RemovedCommandError{Command: "course student", Guidance: "use the app"}, KindRemovedCommand}, {"network", &NetworkError{Message: "unreachable"}, KindUnreachable}, + {"verify", &VerifyError{Path: "assignment.content_json", Message: "did not read back identical"}, KindVerify}, {"canceled", context.Canceled, KindCanceled}, {"deadline exceeded", context.DeadlineExceeded, KindCanceled}, {"plain error", errors.New("something"), KindError}, @@ -76,6 +77,19 @@ func TestKind_UnwrapsThroughErrorfWrapping(t *testing.T) { fmt.Errorf("aborted: %w", context.Canceled), KindCanceled, }, + { + "verify inside ReportedError", + &ReportedError{Err: fmt.Errorf("import-assignment: %w", &VerifyError{Path: "assignment.content_json", Message: "mismatch"})}, + KindVerify, + }, + { + // A failed read-back after an accepted write is verify, not the + // cause's kind: the module was modified, and "unreachable" would + // tell a script the request never happened. + "verify wrapping a transport failure stays verify", + &VerifyError{Path: "assignment.content_json", Message: "read-back failed", Err: &NetworkError{Message: "connection reset"}}, + KindVerify, + }, } for _, tc := range cases { diff --git a/internal/quiz/quiz.go b/internal/quiz/quiz.go new file mode 100644 index 0000000..15a9bd3 --- /dev/null +++ b/internal/quiz/quiz.go @@ -0,0 +1,459 @@ +// Package quiz recognizes, validates and summarizes quiz envelopes — the +// `type: "quiz"` shape that rides an assignment's opaque `content_json` field +// alongside ordinary Tiptap documents (`type: "doc"`). +// +// The Andamio apps are the authority for the shared rules. Two apps render +// quizzes — fcb-fan-engagement-app and andamio-app-v2 (app.andamio.io) — and +// any course on the gateway is viewable in the latter, so Validate enforces +// the UNION of their validateQuizDefinition rules: a quiz the CLI accepts +// renders in both. Recognize mirrors isQuizContentEnvelope; Validate keeps +// the apps' control flow, issue codes and wording where practical. The exact +// upstream revisions, and the two checks the CLI adds on top (malformed-help, +// malformed-intro), are recorded in testdata/quiz/SOURCE.md; a rule change in +// either app is re-mirrored by hand and pinned by the fixtures under +// testdata/quiz. +// +// Recognition and validity are deliberately separate, as in the app: an +// envelope with an unsupported version is still quiz-shaped (it must not be +// treated as a Tiptap doc), it is just not a valid quiz. +// +// The package has no dependency on cmd or on any other internal package. +package quiz + +import ( + "encoding/json" + "fmt" + "math" + "strings" +) + +// Kind is the recognized shape of a content_json value. +type Kind int + +const ( + // NotObject is any JSON value that is not an object: array, string, + // number, boolean or null. + NotObject Kind = iota + // Doc is a Tiptap document (`type: "doc"`). + Doc + // Quiz is a quiz-shaped envelope: `type: "quiz"`, numeric `version`, + // array `questions`. Validity is a separate question — see Validate. + Quiz + // Other is an object that is neither a doc nor quiz-shaped (including a + // quiz-evidence envelope and a `type: "quiz"` object missing `questions`). + Other +) + +func (k Kind) String() string { + switch k { + case NotObject: + return "not an object" + case Doc: + return "doc" + case Quiz: + return "quiz" + case Other: + return "other" + } + return fmt.Sprintf("Kind(%d)", int(k)) +} + +// SupportedVersion is the only quiz envelope version the CLI accepts. +const SupportedVersion = 1 + +// Issue codes. The first ten are the codes both apps share; the next three +// come from andamio-app-v2 only; the two after that are CLI-additional +// checks; CodeNotAQuiz is a guard for callers that hand Validate something +// Recognize would not call a quiz. +const ( + CodeUnsupportedVersion = "unsupported-version" + CodeEmptyQuestions = "empty-questions" + CodeInvalidThreshold = "invalid-threshold" + CodeThresholdExceedsQuestion = "threshold-exceeds-questions" + CodeMissingCorrectValue = "missing-correct-value" + CodeDanglingCorrectValue = "dangling-correct-value" + CodeDuplicateOptionValues = "duplicate-option-values" + CodeDuplicateQuestionIDs = "duplicate-question-ids" + CodeTooFewOptions = "too-few-options" + CodeMalformedQuestion = "malformed-question" + + CodeEmptyPrompt = "empty-prompt" + CodeEmptyOptionLabel = "empty-option-label" + CodeEmptyOptionValue = "empty-option-value" + + CodeMalformedHelp = "malformed-help" + CodeMalformedIntro = "malformed-intro" + + CodeNotAQuiz = "not-a-quiz" +) + +// AllCodes lists every issue code Validate can emit. +var AllCodes = []string{ + CodeUnsupportedVersion, + CodeEmptyQuestions, + CodeInvalidThreshold, + CodeThresholdExceedsQuestion, + CodeMissingCorrectValue, + CodeDanglingCorrectValue, + CodeDuplicateOptionValues, + CodeDuplicateQuestionIDs, + CodeTooFewOptions, + CodeMalformedQuestion, + CodeEmptyPrompt, + CodeEmptyOptionLabel, + CodeEmptyOptionValue, + CodeMalformedHelp, + CodeMalformedIntro, + CodeNotAQuiz, +} + +// Issue is one violated rule. QuestionID is set when the rule applies to a +// specific question. +type Issue struct { + Code string `json:"code"` + Message string `json:"message"` + QuestionID string `json:"question_id,omitempty"` +} + +// String renders the issue as one line suitable for stderr. +func (i Issue) String() string { + if i.QuestionID != "" { + return fmt.Sprintf("%s [question %q]: %s", i.Code, i.QuestionID, i.Message) + } + return fmt.Sprintf("%s: %s", i.Code, i.Message) +} + +// Summary is the scriptable digest of a quiz envelope. +type Summary struct { + QuestionCount int `json:"question_count"` + PassThreshold int `json:"pass_threshold"` + QuestionIDs []string `json:"question_ids"` +} + +// Recognize decodes raw JSON and classifies its shape. The error is non-nil +// only when raw is not valid JSON. env is the decoded object for Doc, Quiz and +// Other, and nil for NotObject. +func Recognize(raw []byte) (Kind, map[string]interface{}, error) { + var value interface{} + if err := json.Unmarshal(raw, &value); err != nil { + return NotObject, nil, fmt.Errorf("invalid JSON: %w", err) + } + env, ok := value.(map[string]interface{}) + if !ok { + return NotObject, nil, nil + } + return recognizeObject(env), env, nil +} + +// recognizeObject mirrors isQuizContentEnvelope, with a doc branch first. +// Classify is Recognize for an already-decoded value: the one shape +// classifier for callers holding a map (course export decides between +// assignment.md and assignment.quiz.json with it). nil is NotObject. +func Classify(env map[string]interface{}) Kind { + if env == nil { + return NotObject + } + return recognizeObject(env) +} + +func recognizeObject(env map[string]interface{}) Kind { + switch env["type"] { + case "doc": + return Doc + case "quiz": + if isQuizShaped(env) { + return Quiz + } + } + return Other +} + +func isQuizShaped(env map[string]interface{}) bool { + if env["type"] != "quiz" { + return false + } + if _, ok := env["version"].(float64); !ok { + return false + } + _, ok := env["questions"].([]interface{}) + return ok +} + +// Parse is Recognize followed by Validate when the value is quiz-shaped. It is +// the one-call path for commands: kind tells the caller how to route, issues +// is empty for a valid quiz (and always empty for non-quiz kinds). +func Parse(raw []byte) (env map[string]interface{}, kind Kind, issues []Issue, err error) { + kind, env, err = Recognize(raw) + if err != nil { + return nil, kind, nil, err + } + if kind == Quiz { + issues = Validate(env) + } + return env, kind, issues, nil +} + +// Validate mirrors the app's validateQuizDefinition and adds the three +// CLI-additional checks. It collects every issue it can find — it never stops +// at the first and never panics on malformed elements. env is expected to +// have been recognized as Quiz; anything else yields a single not-a-quiz +// issue rather than a false pass. +func Validate(env map[string]interface{}) []Issue { + issues := []Issue{} + + if env == nil || !isQuizShaped(env) { + return append(issues, Issue{ + Code: CodeNotAQuiz, + Message: `Not a quiz envelope — expected an object with type "quiz", a numeric version and a questions array.`, + }) + } + + version, ok := integral(env["version"]) + if !ok || version != SupportedVersion { + issues = append(issues, Issue{ + Code: CodeUnsupportedVersion, + Message: fmt.Sprintf("Envelope version %s is not supported (expected %d).", formatNumber(env["version"]), SupportedVersion), + }) + } + + // CLI-additional: intro, when present and non-null, must be a Tiptap doc. + // Checked before the empty-questions early return so it is always reported. + if intro, present := env["intro"]; present && intro != nil { + introObj, isObj := intro.(map[string]interface{}) + if !isObj || introObj["type"] != "doc" { + issues = append(issues, Issue{ + Code: CodeMalformedIntro, + Message: `intro must be a Tiptap document (an object with type "doc"), or null/absent.`, + }) + } + } + + rawQuestions := env["questions"].([]interface{}) + if len(rawQuestions) == 0 { + return append(issues, Issue{ + Code: CodeEmptyQuestions, + Message: "The quiz has no questions.", + }) + } + + threshold, ok := integral(env["passThreshold"]) + if !ok || threshold < 1 { + issues = append(issues, Issue{ + Code: CodeInvalidThreshold, + Message: "passThreshold must be a positive whole number.", + }) + } else if threshold > len(rawQuestions) { + issues = append(issues, Issue{ + Code: CodeThresholdExceedsQuestion, + Message: fmt.Sprintf("passThreshold (%d) exceeds the question count (%d) — the quiz can never be passed.", threshold, len(rawQuestions)), + }) + } + + seenIDs := map[string]bool{} + for _, rawQuestion := range rawQuestions { + question, isObj := rawQuestion.(map[string]interface{}) + if !isObj { + issues = append(issues, Issue{ + Code: CodeMalformedQuestion, + Message: "A questions entry is not an object.", + }) + continue + } + + questionID, hasID := question["id"].(string) + if !hasID { + issues = append(issues, Issue{ + Code: CodeMalformedQuestion, + Message: "A question is missing a string id.", + }) + continue + } + + // andamio-app-v2: a blank prompt renders as a working quiz with an + // empty question. Non-string prompts render blank too. CLI-additional: + // help, when present and non-null, must be a string. Neither depends + // on the options shape, so they are reported even when options are + // broken. + if prompt, isStr := question["prompt"].(string); !isStr || strings.TrimSpace(prompt) == "" { + issues = append(issues, Issue{ + Code: CodeEmptyPrompt, + Message: fmt.Sprintf("Question %q has an empty prompt.", questionID), + QuestionID: questionID, + }) + } + if help, present := question["help"]; present && help != nil { + if _, isStr := help.(string); !isStr { + issues = append(issues, Issue{ + Code: CodeMalformedHelp, + Message: fmt.Sprintf("Question %q has a help field that is not a string.", questionID), + QuestionID: questionID, + }) + } + } + + options, optionsOK := optionRecords(question["options"]) + if !optionsOK { + issues = append(issues, Issue{ + Code: CodeMalformedQuestion, + Message: fmt.Sprintf("Question %q has malformed options — expected an array of { value, label } string pairs.", questionID), + QuestionID: questionID, + }) + continue + } + + if seenIDs[questionID] { + issues = append(issues, Issue{ + Code: CodeDuplicateQuestionIDs, + Message: fmt.Sprintf("Question id %q is used more than once — answers persist keyed by id.", questionID), + QuestionID: questionID, + }) + } + seenIDs[questionID] = true + + optionValues := make([]string, 0, len(options)) + distinct := map[string]bool{} + for _, opt := range options { + optionValues = append(optionValues, opt.value) + distinct[opt.value] = true + } + if len(optionValues) < 2 { + issues = append(issues, Issue{ + Code: CodeTooFewOptions, + Message: fmt.Sprintf("Question %q has fewer than two options.", questionID), + QuestionID: questionID, + }) + } + if len(distinct) != len(optionValues) { + issues = append(issues, Issue{ + Code: CodeDuplicateOptionValues, + Message: fmt.Sprintf("Question %q has duplicate option values — the correct answer would be ambiguous.", questionID), + QuestionID: questionID, + }) + } + + // andamio-app-v2: blank labels render as empty answer buttons (one + // issue per question, with the count); an empty-string value is + // "no answer" to the taker's all-answered gate, so selecting it + // dead-locks the quiz. A whitespace value is not empty to that gate. + blankLabels := 0 + emptyValue := false + for _, opt := range options { + if strings.TrimSpace(opt.label) == "" { + blankLabels++ + } + if opt.value == "" { + emptyValue = true + } + } + if blankLabels == 1 { + issues = append(issues, Issue{ + Code: CodeEmptyOptionLabel, + Message: fmt.Sprintf("Question %q has an option with an empty label.", questionID), + QuestionID: questionID, + }) + } else if blankLabels > 1 { + issues = append(issues, Issue{ + Code: CodeEmptyOptionLabel, + Message: fmt.Sprintf("Question %q has %d options with empty labels.", questionID, blankLabels), + QuestionID: questionID, + }) + } + if emptyValue { + issues = append(issues, Issue{ + Code: CodeEmptyOptionValue, + Message: fmt.Sprintf("Question %q has an option with an empty value — selecting it would never register as an answer.", questionID), + QuestionID: questionID, + }) + } + + correctValue, isStr := question["correctValue"].(string) + if !isStr || correctValue == "" { + issues = append(issues, Issue{ + Code: CodeMissingCorrectValue, + Message: fmt.Sprintf("Question %q has no correctValue designated.", questionID), + QuestionID: questionID, + }) + } else if !distinct[correctValue] { + issues = append(issues, Issue{ + Code: CodeDanglingCorrectValue, + Message: fmt.Sprintf("Question %q has correctValue %q matching none of its options — it can never be answered correctly.", questionID, correctValue), + QuestionID: questionID, + }) + } + } + + return issues +} + +// Summarize digests an envelope. It is exact on a validated quiz and tolerant +// on anything else: non-array questions count as zero, non-object questions +// and non-string ids are skipped, and a non-integral or missing threshold +// reports as zero. QuestionIDs is never nil so JSON output emits []. +func Summarize(env map[string]interface{}) Summary { + s := Summary{QuestionIDs: []string{}} + if env == nil { + return s + } + if threshold, ok := integral(env["passThreshold"]); ok { + s.PassThreshold = threshold + } + questions, _ := env["questions"].([]interface{}) + s.QuestionCount = len(questions) + for _, raw := range questions { + if q, ok := raw.(map[string]interface{}); ok { + if id, ok := q["id"].(string); ok { + s.QuestionIDs = append(s.QuestionIDs, id) + } + } + } + return s +} + +type optionRecord struct { + value string + label string +} + +// optionRecords mirrors Array.isArray(options) && options.every(isQuizOptionRecord). +func optionRecords(raw interface{}) ([]optionRecord, bool) { + list, ok := raw.([]interface{}) + if !ok { + return nil, false + } + out := make([]optionRecord, 0, len(list)) + for _, item := range list { + rec, ok := item.(map[string]interface{}) + if !ok { + return nil, false + } + value, vOK := rec["value"].(string) + label, lOK := rec["label"].(string) + if !vOK || !lOK { + return nil, false + } + out = append(out, optionRecord{value: value, label: label}) + } + return out, true +} + +// integral reports whether v is a JSON number with no fractional part and +// returns it as an int. JSON numbers decode as float64; 1.5 is not integral. +func integral(v interface{}) (int, bool) { + f, ok := v.(float64) + if !ok || math.IsNaN(f) || math.IsInf(f, 0) || f != math.Trunc(f) { + return 0, false + } + if f > math.MaxInt32 || f < math.MinInt32 { + return 0, false + } + return int(f), true +} + +// formatNumber renders a decoded JSON value for a message without the +// float64 artifacts ("99" not "99.000000", "1.5" as written). +func formatNumber(v interface{}) string { + if f, ok := v.(float64); ok { + return fmt.Sprintf("%v", f) + } + return fmt.Sprintf("%v", v) +} diff --git a/internal/quiz/quiz_test.go b/internal/quiz/quiz_test.go new file mode 100644 index 0000000..f412285 --- /dev/null +++ b/internal/quiz/quiz_test.go @@ -0,0 +1,297 @@ +package quiz + +import ( + "bufio" + "encoding/json" + "os" + "path/filepath" + "reflect" + "sort" + "strings" + "testing" +) + +const fixtureRoot = "../../testdata/quiz" + +// minimalQuiz mirrors validQuiz in the app's quiz-envelope.test.ts. +const minimalQuiz = `{ + "type": "quiz", "version": 1, "passThreshold": 2, + "questions": [ + {"id": "q1", "prompt": "What is a wallet?", + "options": [{"value": "a", "label": "A key manager"}, {"value": "b", "label": "A bank account"}], + "correctValue": "a"}, + {"id": "q2", "prompt": "What is a credential?", + "options": [{"value": "a", "label": "A sticker"}, {"value": "b", "label": "An on-chain record"}], + "correctValue": "b"} + ] +}` + +func codes(issues []Issue) []string { + out := make([]string, 0, len(issues)) + for _, is := range issues { + out = append(out, is.Code) + } + sort.Strings(out) + return out +} + +func mustEnv(t *testing.T, raw string) map[string]interface{} { + t.Helper() + var env map[string]interface{} + if err := json.Unmarshal([]byte(raw), &env); err != nil { + t.Fatalf("fixture JSON: %v", err) + } + return env +} + +// readSidecar parses a <case>.issues file: first line "source: app" (both +// apps), "source: app-v2" (andamio-app-v2 only) or "source: cli-additional", +// then one expected issue code per line. +func readSidecar(t *testing.T, path string) (source string, expected []string) { + t.Helper() + f, err := os.Open(path) + if err != nil { + t.Fatalf("missing sidecar %s: %v", path, err) + } + defer f.Close() + sc := bufio.NewScanner(f) + first := true + for sc.Scan() { + line := strings.TrimSpace(sc.Text()) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + if first { + first = false + if !strings.HasPrefix(line, "source: ") { + t.Fatalf("%s: first line must be 'source: <app|app-v2|cli-additional>', got %q", path, line) + } + source = strings.TrimPrefix(line, "source: ") + if source != "app" && source != "app-v2" && source != "cli-additional" { + t.Fatalf("%s: unknown source %q", path, source) + } + continue + } + expected = append(expected, line) + } + if err := sc.Err(); err != nil { + t.Fatal(err) + } + if source == "" { + t.Fatalf("%s: empty sidecar", path) + } + if len(expected) == 0 { + t.Fatalf("%s: sidecar lists no expected codes", path) + } + sort.Strings(expected) + return source, expected +} + +func TestValidFixtures(t *testing.T) { + paths, err := filepath.Glob(filepath.Join(fixtureRoot, "valid", "*.json")) + if err != nil { + t.Fatal(err) + } + if len(paths) < 3 { + t.Fatalf("expected at least 3 valid fixtures, found %d", len(paths)) + } + for _, p := range paths { + name := strings.TrimSuffix(filepath.Base(p), ".json") + t.Run(name, func(t *testing.T) { + raw, err := os.ReadFile(p) + if err != nil { + t.Fatal(err) + } + env, kind, issues, err := Parse(raw) + if err != nil { + t.Fatalf("Parse: %v", err) + } + if kind != Quiz { + t.Fatalf("kind = %v, want Quiz", kind) + } + if len(issues) != 0 { + t.Fatalf("valid fixture produced issues: %v", issues) + } + if name == "minimal" { + got := Summarize(env) + want := Summary{QuestionCount: 2, PassThreshold: 2, QuestionIDs: []string{"q1", "q2"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("Summarize = %+v, want %+v", got, want) + } + } + }) + } +} + +func TestInvalidFixtures(t *testing.T) { + paths, err := filepath.Glob(filepath.Join(fixtureRoot, "invalid", "*.json")) + if err != nil { + t.Fatal(err) + } + if len(paths) == 0 { + t.Fatal("no invalid fixtures found") + } + // Every R14 rule must be covered by at least one fixture. + covered := map[string]bool{} + for _, p := range paths { + name := strings.TrimSuffix(filepath.Base(p), ".json") + t.Run(name, func(t *testing.T) { + raw, err := os.ReadFile(p) + if err != nil { + t.Fatal(err) + } + source, expected := readSidecar(t, strings.TrimSuffix(p, ".json")+".issues") + env, kind, issues, err := Parse(raw) + if err != nil { + t.Fatalf("Parse: %v", err) + } + if kind != Quiz { + t.Fatalf("kind = %v, want Quiz (invalid fixtures must still be recognized)", kind) + } + got := codes(issues) + if !reflect.DeepEqual(got, expected) { + t.Fatalf("source=%s issue codes = %v, want %v\nissues: %v", source, got, expected, issues) + } + for _, is := range issues { + if is.Message == "" { + t.Errorf("issue %s has empty message", is.Code) + } + if is.String() == "" { + t.Errorf("issue %s has empty String()", is.Code) + } + covered[is.Code] = true + isAppV2 := is.Code == CodeEmptyPrompt || is.Code == CodeEmptyOptionLabel || is.Code == CodeEmptyOptionValue + isAdditional := is.Code == CodeMalformedHelp || is.Code == CodeMalformedIntro + if source == "app" && (isAppV2 || isAdditional) { + t.Errorf("source: app fixture yields a code the FCB app does not emit: %s", is.Code) + } + if source == "app-v2" && isAdditional { + t.Errorf("source: app-v2 fixture yields CLI-additional code %s", is.Code) + } + } + // Never let Summarize panic on malformed input. + _ = Summarize(env) + }) + } + for _, code := range AllCodes { + if code == CodeNotAQuiz { + continue // guarded directly in TestValidateRefusesNonQuiz; fixtures are all recognized quizzes + } + if !covered[code] { + t.Errorf("no invalid fixture exercises issue code %q", code) + } + } +} + +func TestRecognize(t *testing.T) { + cases := []struct { + name string + raw string + want Kind + }{ + {"doc", `{"type":"doc","content":[]}`, Doc}, + {"quiz", minimalQuiz, Quiz}, + {"quiz unsupported version still recognized", `{"type":"quiz","version":99,"questions":[]}`, Quiz}, + {"quiz evidence is other", `{"type":"quiz-evidence","version":1}`, Other}, + {"quiz without questions is other", `{"type":"quiz","version":1}`, Other}, + {"quiz with string version is other", `{"type":"quiz","version":"1","questions":[]}`, Other}, + {"array", `[{"type":"quiz"}]`, NotObject}, + {"string", `"quiz"`, NotObject}, + {"null", `null`, NotObject}, + {"number", `7`, NotObject}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + kind, env, err := Recognize([]byte(tc.raw)) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if kind != tc.want { + t.Fatalf("kind = %v, want %v", kind, tc.want) + } + if tc.want == NotObject && env != nil { + t.Fatalf("env should be nil for non-objects, got %v", env) + } + if tc.want != NotObject && env == nil { + t.Fatal("env should be non-nil for objects") + } + }) + } +} + +func TestRecognizeInvalidJSON(t *testing.T) { + kind, env, err := Recognize([]byte(`{"type": "quiz",`)) + if err == nil { + t.Fatal("expected error for invalid JSON") + } + if kind != NotObject || env != nil { + t.Fatalf("kind=%v env=%v on invalid JSON", kind, env) + } +} + +func TestIntroNullOrAbsentIsFine(t *testing.T) { + absent := mustEnv(t, minimalQuiz) + if issues := Validate(absent); len(issues) != 0 { + t.Fatalf("absent intro: %v", issues) + } + withNull := mustEnv(t, minimalQuiz) + withNull["intro"] = nil + if issues := Validate(withNull); len(issues) != 0 { + t.Fatalf("intro: null: %v", issues) + } + // A JSON literal null decodes to a nil interface value under the key. + raw := strings.Replace(minimalQuiz, `"type": "quiz",`, `"type": "quiz", "intro": null,`, 1) + _, kind, issues, err := Parse([]byte(raw)) + if err != nil || kind != Quiz { + t.Fatalf("Parse: kind=%v err=%v", kind, err) + } + if len(issues) != 0 { + t.Fatalf("intro: null via JSON: %v", issues) + } +} + +func TestValidateNamesQuestionIDs(t *testing.T) { + env := mustEnv(t, minimalQuiz) + qs := env["questions"].([]interface{}) + q2 := qs[1].(map[string]interface{}) + q2["correctValue"] = "zzz" + issues := Validate(env) + if len(issues) != 1 { + t.Fatalf("want 1 issue, got %v", issues) + } + if issues[0].Code != CodeDanglingCorrectValue || issues[0].QuestionID != "q2" { + t.Fatalf("got %+v", issues[0]) + } + if !strings.Contains(issues[0].String(), "q2") { + t.Fatalf("String() should name the question: %q", issues[0].String()) + } +} + +func TestValidateRefusesNonQuiz(t *testing.T) { + // Validate is meant to run after Recognize, but it must not panic or + // report success when handed something that is not a quiz envelope. + for _, raw := range []string{ + `{"type":"doc","content":[]}`, + `{"type":"quiz","version":1}`, + `{"type":"quiz","version":1,"questions":"nope"}`, + } { + issues := Validate(mustEnv(t, raw)) + if len(issues) != 1 || issues[0].Code != CodeNotAQuiz { + t.Fatalf("%s: got %v, want single %s", raw, issues, CodeNotAQuiz) + } + } + if issues := Validate(nil); len(issues) != 1 || issues[0].Code != CodeNotAQuiz { + t.Fatalf("nil env: got %v", issues) + } +} + +func TestSummarizeTolerant(t *testing.T) { + got := Summarize(mustEnv(t, `{"type":"quiz","version":1,"passThreshold":1.5,"questions":[null,{"id":7},{"id":"ok"}]}`)) + want := Summary{QuestionCount: 3, PassThreshold: 0, QuestionIDs: []string{"ok"}} + if !reflect.DeepEqual(got, want) { + t.Fatalf("Summarize = %+v, want %+v", got, want) + } + if got := Summarize(nil); got.QuestionIDs == nil { + t.Fatal("QuestionIDs must be a non-nil empty slice so JSON emits []") + } +} diff --git a/testdata/quiz/SOURCE.md b/testdata/quiz/SOURCE.md new file mode 100644 index 0000000..5f6edc9 --- /dev/null +++ b/testdata/quiz/SOURCE.md @@ -0,0 +1,65 @@ +# Quiz envelope fixtures — source of truth + +These fixtures pin `internal/quiz` against the two Andamio apps that render +quizzes. Any course on the gateway is viewable in app.andamio.io, so the CLI +enforces the **union** of both validators: a quiz the CLI accepts renders in +both. The apps are the authority for every rule they define; the CLI mirrors +them by hand and this directory is what catches drift. + +## Mirrored from + +Repository `Andamio-Platform/fcb-fan-engagement-app`, read 2026-09-05: + +| File | Commit | Mirrored as | +|------|--------|-------------| +| `src/lib/quiz/quiz-envelope.ts` | `3842a31f9b7a83bc8d7b4273dcb9dfa6b551ed8c` | `Recognize` ← `isQuizContentEnvelope`, `Validate` ← `validateQuizDefinition` | +| `src/lib/quiz/quiz-envelope.test.ts` | `77aa83366ef6b2df2e9c4624d567b890945c87f3` | `valid/minimal.json` ← `validQuiz`; every `validateQuizDefinition` case ← one `invalid/*.json` | + +Repository `Andamio-Platform/andamio-app-v2`, read 2026-09-05 (the file's +last commit, not the repo HEAD): + +| File | Commit | Mirrored as | +|------|--------|-------------| +| `src/lib/quiz/quiz-envelope.ts` | `ca30c2a` | the three rules the FCB app lacks: `empty-prompt` (prompt missing, non-string, or whitespace-only), `empty-option-label` (one issue per question, with the count when more than one label is blank), `empty-option-value` (an empty-string value; a whitespace value is not empty) | +| `src/lib/quiz/quiz-envelope.test.ts` | `ca30c2a` | `invalid/empty-prompt-*.json`, `invalid/empty-option-label*.json`, `invalid/empty-option-value.json`, `valid/option-value-whitespace.json` | + +Fetch either file with: + +``` +gh api repos/Andamio-Platform/fcb-fan-engagement-app/contents/src/lib/quiz/quiz-envelope.ts --jq .content | base64 -d +gh api repos/Andamio-Platform/fcb-fan-engagement-app/contents/src/lib/quiz/quiz-envelope.test.ts --jq .content | base64 -d +``` + +**A rule change in either app is re-mirrored by hand.** Nothing here fetches +the apps at test time. When a validator changes, update `internal/quiz/quiz.go`, +update or add fixtures, and bump the commit in the matching table above. + +## Layout + +- `valid/<case>.json` — must recognize as a quiz and produce zero issues. +- `invalid/<case>.json` + `invalid/<case>.issues` — must recognize as a quiz + (invalid is not the same as unrecognized) and produce exactly the sidecar's + code set. Sidecar format: first line `source: app`, `source: app-v2` or + `source: cli-additional`, then one expected issue code per line; order is + irrelevant. + +## Source labels + +- `source: app` — the app's `validateQuizDefinition` emits the same code set + for this input. Most of these are the app's own test cases verbatim + (`null-question` keeps the app's `passThreshold: 2`, so the app also + reports `threshold-exceeds-questions` for it); `non-integral-version` and + `threshold-non-integral` are not in the app's test file but exercise its + `version !== 1` and `Number.isInteger` rules. +- `source: app-v2` — rules only andamio-app-v2 enforces: `empty-prompt`, + `empty-option-label`, `empty-option-value`. A `source: app` fixture must + never yield one of these codes; the test enforces that. +- `source: cli-additional` — rules neither app enforces. Codes: + `malformed-help` (help present, non-null, not a string), + `malformed-intro` (intro present, non-null, not an object with `type: "doc"`). + Neither a `source: app` nor a `source: app-v2` fixture may yield one of + these codes; the test enforces that. + +`not-a-quiz` is a guard code for callers that skip `Recognize`; it has no +fixture because every fixture here is quiz-shaped by construction, and it is +tested directly in `internal/quiz/quiz_test.go`. diff --git a/testdata/quiz/invalid/dangling-correct-value.issues b/testdata/quiz/invalid/dangling-correct-value.issues new file mode 100644 index 0000000..9b86d57 --- /dev/null +++ b/testdata/quiz/invalid/dangling-correct-value.issues @@ -0,0 +1,2 @@ +source: app +dangling-correct-value diff --git a/testdata/quiz/invalid/dangling-correct-value.json b/testdata/quiz/invalid/dangling-correct-value.json new file mode 100644 index 0000000..c0aa3c3 --- /dev/null +++ b/testdata/quiz/invalid/dangling-correct-value.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "zzz" + } + ] +} diff --git a/testdata/quiz/invalid/duplicate-option-values.issues b/testdata/quiz/invalid/duplicate-option-values.issues new file mode 100644 index 0000000..c465e1c --- /dev/null +++ b/testdata/quiz/invalid/duplicate-option-values.issues @@ -0,0 +1,2 @@ +source: app +duplicate-option-values diff --git a/testdata/quiz/invalid/duplicate-option-values.json b/testdata/quiz/invalid/duplicate-option-values.json new file mode 100644 index 0000000..c789460 --- /dev/null +++ b/testdata/quiz/invalid/duplicate-option-values.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "One" + }, + { + "value": "a", + "label": "Two" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/duplicate-question-ids.issues b/testdata/quiz/invalid/duplicate-question-ids.issues new file mode 100644 index 0000000..bc6b52c --- /dev/null +++ b/testdata/quiz/invalid/duplicate-question-ids.issues @@ -0,0 +1,2 @@ +source: app +duplicate-question-ids diff --git a/testdata/quiz/invalid/duplicate-question-ids.json b/testdata/quiz/invalid/duplicate-question-ids.json new file mode 100644 index 0000000..8ffa81d --- /dev/null +++ b/testdata/quiz/invalid/duplicate-question-ids.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q1", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-option-label.issues b/testdata/quiz/invalid/empty-option-label.issues new file mode 100644 index 0000000..ae332b7 --- /dev/null +++ b/testdata/quiz/invalid/empty-option-label.issues @@ -0,0 +1,2 @@ +source: app-v2 +empty-option-label diff --git a/testdata/quiz/invalid/empty-option-label.json b/testdata/quiz/invalid/empty-option-label.json new file mode 100644 index 0000000..239de1d --- /dev/null +++ b/testdata/quiz/invalid/empty-option-label.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-option-labels-two.issues b/testdata/quiz/invalid/empty-option-labels-two.issues new file mode 100644 index 0000000..ae332b7 --- /dev/null +++ b/testdata/quiz/invalid/empty-option-labels-two.issues @@ -0,0 +1,2 @@ +source: app-v2 +empty-option-label diff --git a/testdata/quiz/invalid/empty-option-labels-two.json b/testdata/quiz/invalid/empty-option-labels-two.json new file mode 100644 index 0000000..c3ad9ab --- /dev/null +++ b/testdata/quiz/invalid/empty-option-labels-two.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": " " + }, + { + "value": "b", + "label": "" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-option-value.issues b/testdata/quiz/invalid/empty-option-value.issues new file mode 100644 index 0000000..675634f --- /dev/null +++ b/testdata/quiz/invalid/empty-option-value.issues @@ -0,0 +1,2 @@ +source: app-v2 +empty-option-value diff --git a/testdata/quiz/invalid/empty-option-value.json b/testdata/quiz/invalid/empty-option-value.json new file mode 100644 index 0000000..5013d13 --- /dev/null +++ b/testdata/quiz/invalid/empty-option-value.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "b" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-prompt-empty.issues b/testdata/quiz/invalid/empty-prompt-empty.issues new file mode 100644 index 0000000..9edffe2 --- /dev/null +++ b/testdata/quiz/invalid/empty-prompt-empty.issues @@ -0,0 +1,2 @@ +source: app-v2 +empty-prompt diff --git a/testdata/quiz/invalid/empty-prompt-empty.json b/testdata/quiz/invalid/empty-prompt-empty.json new file mode 100644 index 0000000..057785d --- /dev/null +++ b/testdata/quiz/invalid/empty-prompt-empty.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-prompt-missing.issues b/testdata/quiz/invalid/empty-prompt-missing.issues new file mode 100644 index 0000000..9edffe2 --- /dev/null +++ b/testdata/quiz/invalid/empty-prompt-missing.issues @@ -0,0 +1,2 @@ +source: app-v2 +empty-prompt diff --git a/testdata/quiz/invalid/empty-prompt-missing.json b/testdata/quiz/invalid/empty-prompt-missing.json new file mode 100644 index 0000000..ff2c3cf --- /dev/null +++ b/testdata/quiz/invalid/empty-prompt-missing.json @@ -0,0 +1,36 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-prompt-whitespace.issues b/testdata/quiz/invalid/empty-prompt-whitespace.issues new file mode 100644 index 0000000..9edffe2 --- /dev/null +++ b/testdata/quiz/invalid/empty-prompt-whitespace.issues @@ -0,0 +1,2 @@ +source: app-v2 +empty-prompt diff --git a/testdata/quiz/invalid/empty-prompt-whitespace.json b/testdata/quiz/invalid/empty-prompt-whitespace.json new file mode 100644 index 0000000..07bee20 --- /dev/null +++ b/testdata/quiz/invalid/empty-prompt-whitespace.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": " ", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/empty-questions.issues b/testdata/quiz/invalid/empty-questions.issues new file mode 100644 index 0000000..4f408c0 --- /dev/null +++ b/testdata/quiz/invalid/empty-questions.issues @@ -0,0 +1,2 @@ +source: app +empty-questions diff --git a/testdata/quiz/invalid/empty-questions.json b/testdata/quiz/invalid/empty-questions.json new file mode 100644 index 0000000..776ba4b --- /dev/null +++ b/testdata/quiz/invalid/empty-questions.json @@ -0,0 +1,6 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [] +} diff --git a/testdata/quiz/invalid/malformed-help.issues b/testdata/quiz/invalid/malformed-help.issues new file mode 100644 index 0000000..df31342 --- /dev/null +++ b/testdata/quiz/invalid/malformed-help.issues @@ -0,0 +1,2 @@ +source: cli-additional +malformed-help diff --git a/testdata/quiz/invalid/malformed-help.json b/testdata/quiz/invalid/malformed-help.json new file mode 100644 index 0000000..025eb17 --- /dev/null +++ b/testdata/quiz/invalid/malformed-help.json @@ -0,0 +1,42 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a", + "help": [ + "not", + "a", + "string" + ] + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/malformed-intro-not-doc.issues b/testdata/quiz/invalid/malformed-intro-not-doc.issues new file mode 100644 index 0000000..8720f46 --- /dev/null +++ b/testdata/quiz/invalid/malformed-intro-not-doc.issues @@ -0,0 +1,2 @@ +source: cli-additional +malformed-intro diff --git a/testdata/quiz/invalid/malformed-intro-not-doc.json b/testdata/quiz/invalid/malformed-intro-not-doc.json new file mode 100644 index 0000000..827c6ab --- /dev/null +++ b/testdata/quiz/invalid/malformed-intro-not-doc.json @@ -0,0 +1,41 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ], + "intro": { + "type": "paragraph", + "content": [] + } +} diff --git a/testdata/quiz/invalid/malformed-intro-string.issues b/testdata/quiz/invalid/malformed-intro-string.issues new file mode 100644 index 0000000..8720f46 --- /dev/null +++ b/testdata/quiz/invalid/malformed-intro-string.issues @@ -0,0 +1,2 @@ +source: cli-additional +malformed-intro diff --git a/testdata/quiz/invalid/malformed-intro-string.json b/testdata/quiz/invalid/malformed-intro-string.json new file mode 100644 index 0000000..eab84ac --- /dev/null +++ b/testdata/quiz/invalid/malformed-intro-string.json @@ -0,0 +1,38 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ], + "intro": "Welcome to the quiz" +} diff --git a/testdata/quiz/invalid/missing-correct-value.issues b/testdata/quiz/invalid/missing-correct-value.issues new file mode 100644 index 0000000..1d0f1bc --- /dev/null +++ b/testdata/quiz/invalid/missing-correct-value.issues @@ -0,0 +1,2 @@ +source: app +missing-correct-value diff --git a/testdata/quiz/invalid/missing-correct-value.json b/testdata/quiz/invalid/missing-correct-value.json new file mode 100644 index 0000000..2107aac --- /dev/null +++ b/testdata/quiz/invalid/missing-correct-value.json @@ -0,0 +1,36 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ] + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/missing-options.issues b/testdata/quiz/invalid/missing-options.issues new file mode 100644 index 0000000..7e9640f --- /dev/null +++ b/testdata/quiz/invalid/missing-options.issues @@ -0,0 +1,2 @@ +source: app +malformed-question diff --git a/testdata/quiz/invalid/missing-options.json b/testdata/quiz/invalid/missing-options.json new file mode 100644 index 0000000..dc27756 --- /dev/null +++ b/testdata/quiz/invalid/missing-options.json @@ -0,0 +1,27 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/multi-issue.issues b/testdata/quiz/invalid/multi-issue.issues new file mode 100644 index 0000000..3413298 --- /dev/null +++ b/testdata/quiz/invalid/multi-issue.issues @@ -0,0 +1,3 @@ +source: app +threshold-exceeds-questions +dangling-correct-value diff --git a/testdata/quiz/invalid/multi-issue.json b/testdata/quiz/invalid/multi-issue.json new file mode 100644 index 0000000..d24e39b --- /dev/null +++ b/testdata/quiz/invalid/multi-issue.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 9, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "zzz" + } + ] +} diff --git a/testdata/quiz/invalid/non-integral-version.issues b/testdata/quiz/invalid/non-integral-version.issues new file mode 100644 index 0000000..ed7bbbe --- /dev/null +++ b/testdata/quiz/invalid/non-integral-version.issues @@ -0,0 +1,2 @@ +source: app +unsupported-version diff --git a/testdata/quiz/invalid/non-integral-version.json b/testdata/quiz/invalid/non-integral-version.json new file mode 100644 index 0000000..a5884d2 --- /dev/null +++ b/testdata/quiz/invalid/non-integral-version.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1.5, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/non-record-option.issues b/testdata/quiz/invalid/non-record-option.issues new file mode 100644 index 0000000..7e9640f --- /dev/null +++ b/testdata/quiz/invalid/non-record-option.issues @@ -0,0 +1,2 @@ +source: app +malformed-question diff --git a/testdata/quiz/invalid/non-record-option.json b/testdata/quiz/invalid/non-record-option.json new file mode 100644 index 0000000..7919d9b --- /dev/null +++ b/testdata/quiz/invalid/non-record-option.json @@ -0,0 +1,34 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + "a", + { + "value": "b", + "label": "B" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/non-string-id.issues b/testdata/quiz/invalid/non-string-id.issues new file mode 100644 index 0000000..7e9640f --- /dev/null +++ b/testdata/quiz/invalid/non-string-id.issues @@ -0,0 +1,2 @@ +source: app +malformed-question diff --git a/testdata/quiz/invalid/non-string-id.json b/testdata/quiz/invalid/non-string-id.json new file mode 100644 index 0000000..954dd00 --- /dev/null +++ b/testdata/quiz/invalid/non-string-id.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": 7, + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/null-question.issues b/testdata/quiz/invalid/null-question.issues new file mode 100644 index 0000000..7969f3c --- /dev/null +++ b/testdata/quiz/invalid/null-question.issues @@ -0,0 +1,3 @@ +source: app +malformed-question +threshold-exceeds-questions diff --git a/testdata/quiz/invalid/null-question.json b/testdata/quiz/invalid/null-question.json new file mode 100644 index 0000000..88dffc7 --- /dev/null +++ b/testdata/quiz/invalid/null-question.json @@ -0,0 +1,8 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + null + ] +} diff --git a/testdata/quiz/invalid/null-then-valid.issues b/testdata/quiz/invalid/null-then-valid.issues new file mode 100644 index 0000000..7e9640f --- /dev/null +++ b/testdata/quiz/invalid/null-then-valid.issues @@ -0,0 +1,2 @@ +source: app +malformed-question diff --git a/testdata/quiz/invalid/null-then-valid.json b/testdata/quiz/invalid/null-then-valid.json new file mode 100644 index 0000000..4b15694 --- /dev/null +++ b/testdata/quiz/invalid/null-then-valid.json @@ -0,0 +1,38 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + null, + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/options-string.issues b/testdata/quiz/invalid/options-string.issues new file mode 100644 index 0000000..7e9640f --- /dev/null +++ b/testdata/quiz/invalid/options-string.issues @@ -0,0 +1,2 @@ +source: app +malformed-question diff --git a/testdata/quiz/invalid/options-string.json b/testdata/quiz/invalid/options-string.json new file mode 100644 index 0000000..d689941 --- /dev/null +++ b/testdata/quiz/invalid/options-string.json @@ -0,0 +1,28 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": "ab", + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/threshold-exceeds.issues b/testdata/quiz/invalid/threshold-exceeds.issues new file mode 100644 index 0000000..2b6f709 --- /dev/null +++ b/testdata/quiz/invalid/threshold-exceeds.issues @@ -0,0 +1,2 @@ +source: app +threshold-exceeds-questions diff --git a/testdata/quiz/invalid/threshold-exceeds.json b/testdata/quiz/invalid/threshold-exceeds.json new file mode 100644 index 0000000..2a1ab05 --- /dev/null +++ b/testdata/quiz/invalid/threshold-exceeds.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 3, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/threshold-non-integral.issues b/testdata/quiz/invalid/threshold-non-integral.issues new file mode 100644 index 0000000..5a9ade0 --- /dev/null +++ b/testdata/quiz/invalid/threshold-non-integral.issues @@ -0,0 +1,2 @@ +source: app +invalid-threshold diff --git a/testdata/quiz/invalid/threshold-non-integral.json b/testdata/quiz/invalid/threshold-non-integral.json new file mode 100644 index 0000000..95187db --- /dev/null +++ b/testdata/quiz/invalid/threshold-non-integral.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 1.5, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/threshold-zero.issues b/testdata/quiz/invalid/threshold-zero.issues new file mode 100644 index 0000000..5a9ade0 --- /dev/null +++ b/testdata/quiz/invalid/threshold-zero.issues @@ -0,0 +1,2 @@ +source: app +invalid-threshold diff --git a/testdata/quiz/invalid/threshold-zero.json b/testdata/quiz/invalid/threshold-zero.json new file mode 100644 index 0000000..197bf47 --- /dev/null +++ b/testdata/quiz/invalid/threshold-zero.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 0, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/too-few-options.issues b/testdata/quiz/invalid/too-few-options.issues new file mode 100644 index 0000000..3433292 --- /dev/null +++ b/testdata/quiz/invalid/too-few-options.issues @@ -0,0 +1,2 @@ +source: app +too-few-options diff --git a/testdata/quiz/invalid/too-few-options.json b/testdata/quiz/invalid/too-few-options.json new file mode 100644 index 0000000..887b4e7 --- /dev/null +++ b/testdata/quiz/invalid/too-few-options.json @@ -0,0 +1,33 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "Only" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/invalid/unsupported-version.issues b/testdata/quiz/invalid/unsupported-version.issues new file mode 100644 index 0000000..ed7bbbe --- /dev/null +++ b/testdata/quiz/invalid/unsupported-version.issues @@ -0,0 +1,2 @@ +source: app +unsupported-version diff --git a/testdata/quiz/invalid/unsupported-version.json b/testdata/quiz/invalid/unsupported-version.json new file mode 100644 index 0000000..2561488 --- /dev/null +++ b/testdata/quiz/invalid/unsupported-version.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 99, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": "a", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/valid/minimal.json b/testdata/quiz/valid/minimal.json new file mode 100644 index 0000000..ceac65f --- /dev/null +++ b/testdata/quiz/valid/minimal.json @@ -0,0 +1,25 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { "value": "a", "label": "A key manager" }, + { "value": "b", "label": "A bank account" } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { "value": "a", "label": "A sticker" }, + { "value": "b", "label": "An on-chain record" } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/valid/option-value-whitespace.json b/testdata/quiz/valid/option-value-whitespace.json new file mode 100644 index 0000000..f35d0b1 --- /dev/null +++ b/testdata/quiz/valid/option-value-whitespace.json @@ -0,0 +1,37 @@ +{ + "type": "quiz", + "version": 1, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { + "value": " ", + "label": "A key manager" + }, + { + "value": "b", + "label": "A bank account" + } + ], + "correctValue": "b" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { + "value": "a", + "label": "A sticker" + }, + { + "value": "b", + "label": "An on-chain record" + } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/valid/with-help.json b/testdata/quiz/valid/with-help.json new file mode 100644 index 0000000..c7df4c8 --- /dev/null +++ b/testdata/quiz/valid/with-help.json @@ -0,0 +1,29 @@ +{ + "type": "quiz", + "version": 1, + "intro": null, + "passThreshold": 2, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "help": "Think about what it holds.", + "options": [ + { "value": "a", "label": "A key manager" }, + { "value": "b", "label": "A bank account" } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "Which of these is a credential?", + "help": "Only one is recorded on-chain.", + "options": [ + { "value": "a", "label": "A sticker" }, + { "value": "b", "label": "An on-chain record" }, + { "value": "c", "label": "A receipt" } + ], + "correctValue": "b" + } + ] +} diff --git a/testdata/quiz/valid/with-intro.json b/testdata/quiz/valid/with-intro.json new file mode 100644 index 0000000..a3a28bb --- /dev/null +++ b/testdata/quiz/valid/with-intro.json @@ -0,0 +1,34 @@ +{ + "type": "quiz", + "version": 1, + "intro": { + "type": "doc", + "content": [ + { + "type": "paragraph", + "content": [{ "type": "text", "text": "Answer both questions to unlock the credential." }] + } + ] + }, + "passThreshold": 1, + "questions": [ + { + "id": "q1", + "prompt": "What is a wallet?", + "options": [ + { "value": "a", "label": "A key manager" }, + { "value": "b", "label": "A bank account" } + ], + "correctValue": "a" + }, + { + "id": "q2", + "prompt": "What is a credential?", + "options": [ + { "value": "a", "label": "A sticker" }, + { "value": "b", "label": "An on-chain record" } + ], + "correctValue": "b" + } + ] +} diff --git a/todos/039-pending-p2-teacher-module-list-206-blind-spot.md b/todos/039-pending-p2-teacher-module-list-206-blind-spot.md new file mode 100644 index 0000000..b111a39 --- /dev/null +++ b/todos/039-pending-p2-teacher-module-list-206-blind-spot.md @@ -0,0 +1,42 @@ +--- +status: pending +priority: p2 +issue_id: "039" +tags: [degraded-read, 206, course, teacher-endpoints, reliability] +dependencies: [] +--- + +# Teacher module-list scans outside import share the 206 blind spot + +## Problem + +The merged `POST /v2/course/teacher/course-modules/list` answers 206 with +`meta.warning` when one backend is down, and it carries `content` only from +db-api. On a db-api outage every module comes back chain-only with no +`content`, so a client-side scan for one module code finds nothing and reports +"not found" — for a module that exists. + +PR #166 (#165) fixed this for the import paths: `fetchExistingModule` +(`cmd/andamio/course_import.go`) returns `errDegradedRead` when the module is +missing and the warning names the db side (`warningHidesDBContent`), and +`course import` / `course import-assignment` refuse to write on it. The other +scans of the same list still treat "not in the list" as "not found": + +- `cmd/andamio/course.go` (`fetchTeacherModuleContent`) +- `cmd/andamio/course_export.go` (`fetchModuleData`) +- `cmd/andamio/course_teacher_ops.go` + +Export is the worst case: a db-api outage exports nothing where a module has +content. Raised in the review of #166 as a follow-up candidate. + +## Proposed fix + +One shared `findTeacherModule(ctx, c, courseID, moduleCode)` helper in +`cmd/andamio/helpers.go` that owns the list call, the module match, and the +`errDegradedRead` / `errModuleNotFound` distinction, used by all four call +sites. Map `errModuleNotFound` to `apierr.NotFoundError` at the call sites +that expose it (`course.go` already does) so the exit code stays 2. + +Related: `todos/031-pending-p1-no-typed-api-contract-coupling.md` (a typed +`MergedCourseModuleItem` with `Source` would make the chain-only case +explicit instead of inferred from a missing key). diff --git a/todos/040-pending-p3-quiz-assignment-residuals.md b/todos/040-pending-p3-quiz-assignment-residuals.md new file mode 100644 index 0000000..851cf1f --- /dev/null +++ b/todos/040-pending-p3-quiz-assignment-residuals.md @@ -0,0 +1,61 @@ +--- +status: pending +priority: p3 +issue_id: "040" +tags: [quiz, course-import, import-assignment, verify, maintainability] +dependencies: ["039-pending-p2-teacher-module-list-206-blind-spot"] +--- + +# Residuals from the quiz-assignment work (#165, PR #166) + +Real but not fixed in #166. Each is small; none blocks the feature. + +## Read-back cannot see a clobbered concurrent edit + +`course import-assignment` fetches, POSTs, then re-fetches. A teacher editing +description / image / video in the app between the fetch and the POST is +silently reverted (db-api overwrites every assignment field it is given) and +`verified: true` then certifies the revert. A compare-and-set would need +gateway support (an `updated_at` or version on the assignment); until then +the window is one round trip. + +## Read-back has no retry + +The post-write re-fetch uses `Post`, not `PostWithRetry` +(`course_teacher_ops.go` retries the same endpoint). A transient blip after +an accepted write is reported as `kind: verify` with the cause wrapped — +correct, but a retry would turn most of those into a confirmed publish. + +## No committed fixture for the teacher module-list wire shape + +`verifyStoredAssignment` and `updateModuleContent` assume the list nests the +assignment's raw JSON under `content.assignment.content_json`. Unlike the #90 +qualified-contributors fixture (`internal/client/testdata/`), nothing pins +that shape; field drift would surface only as a spurious `kind: verify` in +production. Capture one from preprod. + +## Validator drift is undetectable by CI + +`internal/quiz` mirrors two apps' validators by hand (`testdata/quiz/SOURCE.md` +records the commits). A rule added in either app produces zero CLI failures +until someone re-mirrors. A weekly job that fetches both `quiz-envelope.ts` +files and diffs their issue-code lists against `quiz.AllCodes` would catch it. + +## Duplicated metadata merge and hand-maintained verify field table + +The preserve-existing-metadata rule lives in three shapes (two map loops in +`course_import.go`, a typed construction in `course_import_assignment.go`); +`existingString` duplicates `getStr` (`user.go`); the `fields` table in +`verifyStoredAssignment` will silently miss any new `assignmentInput` field. +Reflect over `assignmentInput`'s json tags, or generate the table from it. + +## Export reports the quiz only by filename + +`ExportResult.Files` lists `assignment.quiz.json`; there is no structured +`assignment_quiz` field to mirror `course import`'s. Additive when wanted. + +## Live preprod check on an ON_CHAIN module + +The "assignments are editable in any module status" statement rests on +gateway and db-api source. Run `course import-assignment` against an +`ON_CHAIN` module on preprod once and record the result in the CHANGELOG.