Repository navigation
feat(api): add the run-result timeline schema skeleton (issue #2452) #3385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,165 @@ | ||||||||||||||||||||||||
| { | ||||||||||||||||||||||||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||||||||||||||||||||||||
| "$id": "https://github.com/VMAFx/vmafx/api/run-result-timeline.schema.json", | ||||||||||||||||||||||||
| "title": "VMAFx run-result timeline, schema version 1", | ||||||||||||||||||||||||
| "description": "One versioned report format for a scoring run's provenance, per-frame scores, per-window statistics and events (VMAFx/vmafx#2452, RC5). It evolves additively under HISS-14: after a release, members are added, never removed, renamed or retyped; schema_version is the integer the document was written against, and a reader refuses a version it does not know by name. docs/api/run-result-timeline.md is the human reference. The skeleton pins the four sections' identities, not yet their full field contracts: provenance is the record of vmafx_context_provenance_json() (RC4 WP5, #2288; the authoritative field contract is the OpenAPI Provenance schema in api/openapi/vmafx-server-v1.yaml and the VmafxProvenance definition in core/api/vmafx.toml), and scoring_configuration is the option surface of #2258 (the scoring component group of the same OpenAPI document). The C ABI writer/reader and the byte-equal write-read-write identity check land in the follow-up PRs of the same issue.", | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "additionalProperties": false, | ||||||||||||||||||||||||
| "required": [ | ||||||||||||||||||||||||
| "schema_version", | ||||||||||||||||||||||||
| "provenance", | ||||||||||||||||||||||||
| "frames", | ||||||||||||||||||||||||
| "windows", | ||||||||||||||||||||||||
| "events" | ||||||||||||||||||||||||
| ], | ||||||||||||||||||||||||
|
Comment on lines
+8
to
+14
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 MEDIUM RISK If scoring configuration is required, add it to the schema's required list and include it in the minimal fixture; otherwise make its optional status explicit in the documentation and acceptance tests.
|
||||||||||||||||||||||||
| "properties": { | ||||||||||||||||||||||||
| "schema_version": { | ||||||||||||||||||||||||
| "const": 1, | ||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needed: record the versioning model in an ADR before this lands.
With
|
||||||||||||||||||||||||
| "description": "The schema version the document was written against. 1 is the shape this file defines; a reader refuses an unknown version by name (#2452 acceptance)." | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "provenance": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/provenance" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "scoring_configuration": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/scoring_configuration" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "frames": { | ||||||||||||||||||||||||
| "type": "array", | ||||||||||||||||||||||||
| "description": "The per-frame series, in ascending time-key order: one entry per scored frame.", | ||||||||||||||||||||||||
| "items": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/frame_score" | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "windows": { | ||||||||||||||||||||||||
| "type": "array", | ||||||||||||||||||||||||
| "description": "Per-window statistics over ranges of the timeline: pooling and aggregation results.", | ||||||||||||||||||||||||
| "items": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/window_statistic" | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "events": { | ||||||||||||||||||||||||
| "type": "array", | ||||||||||||||||||||||||
| "description": "The run's events, in ascending time-key order: alignment decisions, drops, repeats, freezes, switches and masked areas.", | ||||||||||||||||||||||||
| "items": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/event" | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "$defs": { | ||||||||||||||||||||||||
| "time_key": { | ||||||||||||||||||||||||
| "description": "Where an entry sits on the timeline: at least one of the 0-based frame index, the presentation timestamp in nanoseconds from the run's zero, or the SMPTE-style timecode. A writer carries what the run knows - a frame score carries the frame index, a wall-clock event carries the timestamp - and an empty time key is refused.", | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "additionalProperties": false, | ||||||||||||||||||||||||
| "anyOf": [ | ||||||||||||||||||||||||
| { "required": ["frame_index"] }, | ||||||||||||||||||||||||
| { "required": ["timestamp_ns"] }, | ||||||||||||||||||||||||
| { "required": ["timecode"] } | ||||||||||||||||||||||||
| ], | ||||||||||||||||||||||||
| "properties": { | ||||||||||||||||||||||||
| "frame_index": { | ||||||||||||||||||||||||
| "type": "integer", | ||||||||||||||||||||||||
| "minimum": 0, | ||||||||||||||||||||||||
| "description": "0-based display order, the stream-metadata display_index of api/stream-metadata-fields.json." | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "timestamp_ns": { | ||||||||||||||||||||||||
| "type": "integer", | ||||||||||||||||||||||||
| "minimum": 0, | ||||||||||||||||||||||||
| "description": "Presentation timestamp in nanoseconds from the run's zero." | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "timecode": { | ||||||||||||||||||||||||
| "type": "string", | ||||||||||||||||||||||||
| "pattern": "^[0-9]{2}:[0-9]{2}:[0-9]{2}(:[0-9]{2})?$", | ||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: the pattern accepts
Suggested change
|
||||||||||||||||||||||||
| "description": "SMPTE-style HH:MM:SS or HH:MM:SS:FF timecode." | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "provenance": { | ||||||||||||||||||||||||
| "description": "The provenance record of the context that produced the run: vmafx_context_provenance_json() (RC4 WP5, #2288) carries one key per field of VmafxProvenance and its models, features and annotations. The authoritative field contract is the OpenAPI Provenance schema (api/openapi/vmafx-server-v1.yaml) and the VmafxProvenance definition in core/api/vmafx.toml; this schema pins the identity anchor so a timeline stays attributable and leaves the field contract to that single source.", | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "required": ["digest"], | ||||||||||||||||||||||||
| "properties": { | ||||||||||||||||||||||||
| "digest": { | ||||||||||||||||||||||||
| "type": "string", | ||||||||||||||||||||||||
| "pattern": "^sha256:[0-9a-f]{64}$", | ||||||||||||||||||||||||
| "description": "The record's own digest: the digest member of VmafxProvenance, the SHA-256 of the record's canonical JSON (RFC 8785) without digest and elapsed_ns." | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "scoring_configuration": { | ||||||||||||||||||||||||
| "description": "The scoring-option surface in effect for the run (#2258, WP8): the scoring component group of api/openapi/vmafx-server-v1.yaml. The skeleton leaves the members to that single source.", | ||||||||||||||||||||||||
| "type": "object" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "frame_score": { | ||||||||||||||||||||||||
| "description": "One scored frame: its time key and the scores written for it.", | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "additionalProperties": false, | ||||||||||||||||||||||||
| "required": ["time"], | ||||||||||||||||||||||||
| "properties": { | ||||||||||||||||||||||||
| "time": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/time_key" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "scores": { | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "description": "Keys are feature names as the mounted models name them (for example vmaf, integer_adm2); values are that frame's numbers for the feature. A frame without scores is an empty object.", | ||||||||||||||||||||||||
| "additionalProperties": { | ||||||||||||||||||||||||
| "type": "number" | ||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needed: same as line 133: a frame score that is NaN or ±Inf has no valid spelling here.
Suggested change
|
||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
|
Comment on lines
+104
to
+106
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "window_statistic": { | ||||||||||||||||||||||||
| "description": "One statistic computed over a window of the timeline; the range is the half-open interval the writer knows (start inclusive, end exclusive when both ends are frame indices).", | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "additionalProperties": false, | ||||||||||||||||||||||||
| "required": ["start", "end", "statistic", "value"], | ||||||||||||||||||||||||
| "properties": { | ||||||||||||||||||||||||
| "start": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/time_key" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "end": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/time_key" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "statistic": { | ||||||||||||||||||||||||
| "type": "string", | ||||||||||||||||||||||||
| "minLength": 1, | ||||||||||||||||||||||||
| "description": "The statistic's name as the writer computes it (for example pooled, mean, worst_frame)." | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "feature": { | ||||||||||||||||||||||||
| "type": "string", | ||||||||||||||||||||||||
| "minLength": 1, | ||||||||||||||||||||||||
| "description": "The feature the statistic belongs to; omitted for a run-level statistic." | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "value": { | ||||||||||||||||||||||||
| "type": ["number", "string"], | ||||||||||||||||||||||||
| "description": "The computed value as a JSON number; a writer spells a non-finite result as the string NaN, +Inf or -Inf, and a reader matches that spelling." | ||||||||||||||||||||||||
|
Comment on lines
+133
to
+134
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needed: use the spelling the CLI's JSON output already uses for non-finite values. This schema spells them as strings here and forbids them in frame scores (line 105). A converter from today's JSON output, which the issue asks for, would have to translate, and a frame with a non-finite score could not be written at all. One spelling across outputs is the HISS-19 rule. If telling NaN apart from ±Inf matters, that belongs in the ADR, and the CLI writer would change with it.
Suggested change
|
||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
|
Comment on lines
+133
to
+135
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 MEDIUM RISK Restrict string values to the documented non-finite spellings, such as
Comment on lines
+132
to
+135
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Resolve the ambiguous The Restrict the string branch to the three spellings. This closes the gap now, before the contract is frozen under HISS-14. After release, members cannot be retyped. Proposed fix
Committable suggestion
|
||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "event": { | ||||||||||||||||||||||||
| "description": "One run event. A planted event without a time key is refused by name (#2452 acceptance).", | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "additionalProperties": false, | ||||||||||||||||||||||||
| "required": ["type", "time"], | ||||||||||||||||||||||||
| "properties": { | ||||||||||||||||||||||||
| "type": { | ||||||||||||||||||||||||
| "enum": [ | ||||||||||||||||||||||||
| "alignment_decision", | ||||||||||||||||||||||||
| "drop", | ||||||||||||||||||||||||
| "repeat", | ||||||||||||||||||||||||
| "freeze", | ||||||||||||||||||||||||
| "switch", | ||||||||||||||||||||||||
| "masked_area" | ||||||||||||||||||||||||
| ], | ||||||||||||||||||||||||
| "description": "alignment_decision records how streams were aligned; drop, repeat and freeze record frame-handling decisions; switch records a source or configuration switch mid-run; masked_area records a region excluded from scoring. The vocabulary grows additively under HISS-14." | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "time": { | ||||||||||||||||||||||||
| "$ref": "#/$defs/time_key" | ||||||||||||||||||||||||
| }, | ||||||||||||||||||||||||
| "detail": { | ||||||||||||||||||||||||
| "type": "object", | ||||||||||||||||||||||||
| "description": "The event's facts as the writer reports them (for example the alignment offsets of an alignment_decision); the skeleton constrains nothing further." | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| { | ||
| "schema_version": 1, | ||
| "provenance": { | ||
| "digest": "sha256:1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f809" | ||
| }, | ||
| "frames": [ | ||
| { | ||
| "time": { "frame_index": 0 }, | ||
| "scores": { "vif": 0.95102 } | ||
| } | ||
| ], | ||
| "windows": [], | ||
| "events": [ | ||
| { | ||
| "type": "drop", | ||
| "detail": { "reason": "duplicate pts" } | ||
| } | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "schema_version": 2, | ||
| "provenance": { | ||
| "digest": "sha256:1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f809" | ||
| }, | ||
| "frames": [], | ||
| "windows": [], | ||
| "events": [] | ||
| } |
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,69 @@ | ||||||||||
| { | ||||||||||
| "schema_version": 1, | ||||||||||
| "provenance": { | ||||||||||
| "abi_major": 0, | ||||||||||
| "abi_minor": 2, | ||||||||||
| "abi_patch": 0, | ||||||||||
| "active_backend": 1, | ||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fix The OpenAPI schema types Proposed fix
Committable suggestion
|
||||||||||
| "active_backend": 1, | |
| "active_backend": "cpu", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: this hash has 41 hex digits; a Git commit id has 40. The commit the version string on line 8 names is 5c941077b175b6d72827e952e2067c9e800eb1ad.
| "commit": "5c941077bd0f8f68d2c1430e548c902a5245a8ab8", | |
| "commit": "5c941077b175b6d72827e952e2067c9e800eb1ad", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needed: model is not a member of ScoreOptions, the options object of the scoring group (api/openapi/vmafx-server-v1.yaml:168). In a request it sits next to options in ScoreRequest, and the run's models are already recorded in the provenance models array. Dropping the line keeps the fixture inside the contract; naming ScoreOptions in the schema description at line 89 would make the pointer exact.
| "model": "vmaf_v1.0.16", |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "schema_version": 1, | ||
| "provenance": { | ||
| "digest": "sha256:1a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f809" | ||
| }, | ||
| "frames": [], | ||
| "windows": [], | ||
| "events": [] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| - **The RC5 run-result timeline has a schema skeleton.** `api/run-result-timeline.schema.json` defines the versioned document that will carry one run's provenance, per-frame scores, per-window statistics and events across the CLI, the C API and the server (#2452); it evolves additively under HISS-14, and the acceptance fixtures in `api/testdata/run-result-timeline/` pin the shape in `scripts/ci/tests/test_run_result_timeline_schema.py`. The writer/reader C ABI and the byte-equal write-read-write identity check land in the follow-up PRs of the same issue. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| <!-- markdownlint-disable MD013 --> | ||
| # Run-result timeline — `api/run-result-timeline.schema.json` | ||
|
|
||
| VMAFx plans to write one report format for a scoring run instead of letting | ||
| alignment, fields, masks and bits-per-pixel each invent their own | ||
| ([VMAFx/vmafx#2452](https://github.com/VMAFx/vmafx/issues/2452), RC5): a | ||
| versioned JSON document that carries the run's provenance, its per-frame | ||
| scores, its per-window statistics and its events - alignment decisions, drops, | ||
| repeats, freezes, switches, masked areas - each located by one time key that | ||
| can carry a frame index, a timestamp or a timecode. | ||
|
|
||
| [`api/run-result-timeline.schema.json`](https://github.com/VMAFx/vmafx/blob/master/api/run-result-timeline.schema.json) | ||
| is the contract of that document, and this page is its human reference. It is | ||
| the shape only, not the reporting code: no CLI flag, C call or server endpoint | ||
| writes the document yet. The writer/reader C ABI and the identity checks land | ||
| in the follow-up PRs of the same issue; the converters from the existing | ||
| XML/JSON/CSV outputs follow them. | ||
|
|
||
| ## What the document holds | ||
|
|
||
| | Section | Carries | Contract | | ||
| | --- | --- | --- | | ||
| | `schema_version` | The schema version the document was written against (`1` today) | A reader refuses an unknown version by name | | ||
| | `provenance` | The provenance record of the context that produced the scores (`vmafx_context_provenance_json()`, RC4 WP5, #2288) | The authoritative field contract is the OpenAPI [`Provenance` schema](../../api/openapi/vmafx-server-v1.yaml) and the `VmafxProvenance` definition in [`core/api/vmafx.toml`](../../core/api/vmafx.toml); the timeline schema pins the `digest` identity anchor | | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium · Decision record is missing When the provenance and scoring contracts are referenced without an accompanying ADR, the schema's non-trivial choice to reference authoritative contracts rather than copy them has no ADR, and the alternatives matrix remains only in the PR description. The decision and its alternatives are not recorded in the required location. Add an ADR documenting the decision and put the alternatives matrix in its Prompt for AI agentsIn `docs/api/run-result-timeline.md` at line 24:
**Decision record is missing**
When the provenance and scoring contracts are referenced without an accompanying ADR, the schema's non-trivial choice to reference authoritative contracts rather than copy them has no ADR, and the alternatives matrix remains only in the PR description. The decision and its alternatives are not recorded in the required location.
Add an ADR documenting the decision and put the alternatives matrix in its `Alternatives considered` section.From 8. Every non-trivial architectural, policy, or scope decision ships as its own
ADR file `docs/adr/NNNN-kebab-case.md` following
[docs/adr/0000-template.md](../adr/0000-template.md) **before** the
commit that implements it lands. The ADR's index row lives inFrom 9. Every fork-local PR ships the **six deep-dive deliverables** in the same
PR (per [ADR-0108](../adr/0108-deep-dive-deliverables-rule.md)):
(a) research digest under [`docs/research/`](../research/) (or
"no digest needed: trivial"); (b) decision matrix in the accompanying
ADR's `## Alternatives considered` (or "no alternatives: only-one-way
fix"); (c) `AGENTS.md` invariant note in the relevant package (orThere was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium · Research digest is missing The PR keeps its analysis in the issue thread and adds no digest under Add a research digest under Prompt for AI agentsIn `docs/api/run-result-timeline.md` at line 24:
**Research digest is missing**
The PR keeps its analysis in the issue thread and adds no digest under `docs/research/`, so the fork-local research-digest requirement is unmet. Add a research digest under `docs/research/` or, if the change is trivial, the required explicit trivial opt-out.
Add a research digest under `docs/research/`, or explicitly state `no digest needed: trivial` if justified.From 9. Every fork-local PR ships the **six deep-dive deliverables** in the same
PR (per [ADR-0108](../adr/0108-deep-dive-deliverables-rule.md)):
(a) research digest under [`docs/research/`](../research/) (or
"no digest needed: trivial"); (b) decision matrix in the accompanying |
||
| | `scoring_configuration` | The scoring-option surface in effect (#2258, WP8) | The `scoring` component group of the same OpenAPI document is the single source | | ||
| | `frames` | The per-frame series: a time key plus the frame's per-feature scores | The time key carries at least one of frame index, timestamp, timecode | | ||
| | `windows` | Per-window statistics: a start, an end, the statistic's name and its value | Ranges are the half-open interval the writer knows | | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium · Invalid window ranges pass validation When a window has an end before its start, or its endpoints use different time-key coordinate types, the schema validates window endpoints independently and does not compare them. Reversed or incomparable endpoints therefore pass validation, leaving consumers with invalid or ambiguous half-open ranges. Validate that window endpoints use comparable coordinates and that the end is not before the start. Prompt for AI agentsIn `docs/api/run-result-timeline.md` at line 27:
**Invalid window ranges pass validation**
When a window has an end before its start, or its endpoints use different time-key coordinate types, the schema validates window endpoints independently and does not compare them. Reversed or incomparable endpoints therefore pass validation, leaving consumers with invalid or ambiguous half-open ranges.
Validate that window endpoints use comparable coordinates and that the end is not before the start. |
||
| | `events` | The run's events: a type from the additive vocabulary, a required time key, optional facts | An event without a time key is refused by name | | ||
|
|
||
| The time key mirrors what a run actually knows: a frame score carries the | ||
| 0-based display-order frame index (the stream-metadata `display_index` of | ||
| [`api/stream-metadata-fields.json`](https://github.com/VMAFx/vmafx/blob/master/api/stream-metadata-fields.json)), | ||
| a wall-clock event carries the nanosecond timestamp, and a writer may add the | ||
| SMPTE-style timecode when the run has one. | ||
|
|
||
| ## Versioning | ||
|
|
||
| The document evolves additively under the project's | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium · Additive changes break older readers When a later release adds a member or event type while retaining schema version 1, the schema closes objects to unknown properties and fixes the event vocabulary, so older validators reject documents containing additive changes. Readers cannot rely on the stated backward-compatible evolution contract. Make schema validation tolerate additive members and event types for the same schema version, or revise the versioning contract so older readers can handle later documents. Prompt for AI agentsIn `docs/api/run-result-timeline.md` at line 38:
**Additive changes break older readers**
When a later release adds a member or event type while retaining schema version 1, the schema closes objects to unknown properties and fixes the event vocabulary, so older validators reject documents containing additive changes. Readers cannot rely on the stated backward-compatible evolution contract.
Make schema validation tolerate additive members and event types for the same schema version, or revise the versioning contract so older readers can handle later documents. |
||
| [Append-Only ABI](../principles.md) invariant (HISS-14): after a release, the | ||
| schema gains members, and never removes, renames or retypes one. `schema_version` | ||
| is the integer a document was written against; a reader that meets a version it | ||
| does not know refuses the document by name rather than guessing. The acceptance | ||
| fixtures that pin today's shape live in | ||
| [`api/testdata/run-result-timeline/`](https://github.com/VMAFx/vmafx/tree/master/api/testdata/run-result-timeline) | ||
| and run in | ||
| [`scripts/ci/tests/test_run_result_timeline_schema.py`](https://github.com/VMAFx/vmafx/blob/master/scripts/ci/tests/test_run_result_timeline_schema.py) | ||
| (the `tooling` suite): | ||
|
|
||
| ```bash | ||
| python -m pytest scripts/ci/tests/test_run_result_timeline_schema.py -q | ||
| ``` | ||
|
|
||
| ## Not in the skeleton | ||
|
|
||
| The skeleton pins identities, not every field contract. The provenance record's | ||
| fields and the scoring-option surface stay with their single sources (the | ||
| OpenAPI document and `core/api/vmafx.toml`) instead of being copied here, so | ||
| the two contracts cannot drift; the timeline schema constrains what makes a | ||
| timeline attributable and addressable. The richer verdict and facts machinery | ||
| of #2311 extends this schema in 1.1. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| ## Run-result timeline schema skeleton (2026-11-10) | ||
|
|
||
| New files only; nothing upstream corresponds to them, so a rebase cannot | ||
| collide on their content. Two rebase-sensitive surfaces: | ||
|
|
||
| - `REUSE.toml` — the contributor annotation block sits directly after the | ||
| first `[[annotations]]` catch-all. Other contributor pull requests (#3317 | ||
| and later) append their blocks at the same position; a conflict resolves by | ||
| keeping both blocks, catch-all first and contributor blocks after it. Never | ||
| fold a contributor block into the catch-all: the catch-all credits | ||
| `2026 Lusoris`, and folding would misattribute contributor files to the | ||
| maintainer (the credits gate reads the file headers against `REUSE.toml`). | ||
| - `api/run-result-timeline.schema.json` — the document evolves additively | ||
| under HISS-14. A rebase that removes, renames or retypes a schema member, | ||
| or reorders the `schema_version` contract, is a break of the additive | ||
| contract, not a conflict to clean up; restore the schema side and reopen | ||
| the discussion instead. | ||
|
Comment on lines
+13
to
+17
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium · Invariant note is missing The rebase note declares the schema's additive-evolution rule rebase-sensitive, but the PR adds no invariant note under the relevant package's Add the invariant note in the relevant package's Prompt for AI agentsIn `docs/rebase-notes.d/run-result-timeline-schema.md` at lines 13-17:
**Invariant note is missing**
The rebase note declares the schema's additive-evolution rule rebase-sensitive, but the PR adds no invariant note under the relevant package's `AGENTS.md` or `AGENTS.d/`. The required package guidance is absent.
Add the invariant note in the relevant package's `AGENTS.md`, or in its `AGENTS.d/` page where that directory exists.From 9. Every fork-local PR ships the **six deep-dive deliverables** in the same
PR (per [ADR-0108](../adr/0108-deep-dive-deliverables-rule.md)):
(a) research digest under [`docs/research/`](../research/) (or
"no digest needed: trivial"); (b) decision matrix in the accompanying
ADR's `## Alternatives considered` (or "no alternatives: only-one-way
fix"); (c) `AGENTS.md` invariant note in the relevant package (or
"no rebase-sensitive invariants"; where the package has an `AGENTS.d/`,
the note is a page there, see [agents index](agents-index.md)); |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: 2452 is the issue; this PR is 3385.