Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,20 @@ correction to the pipeline framing this file used to carry.
> correction below, had zero internal callers, and keeping it around
> would have let the tree imply a scope the contract now denies; see
> "What does not belong here" for why removal, not just naming the gap,
> was the right close. Nothing here is a stable API.
> was the right close.
>
> **The conformance/drift verdict artifact exists in both languages:**
> `engine/src/verdict.rs` and `go/verdict` assemble the five required fields
> of `docs/VERDICT-SCHEMA.md` (`verdict_schema_version: 1`) from `evaluate`'s
> result and the existing digest projections, refuse an inconsistent result
> (`E_INCONSISTENT_VERDICT`), and are cross-checked byte-for-byte through
> `conformance/differential/verdict/`, including a test that the corpus
> detects deliberate divergence.
> `evaluate_verdict`/`verdict.EvaluateAndBuild` is the primary entry point (plan and
> result are bound by construction); `build_verdict`/`verdict.Build` is the
> low-level assembler and checks structure only -- see `docs/VERDICT-SCHEMA.md`.
>
> Nothing here is a stable API.

---

Expand Down
18 changes: 18 additions & 0 deletions conformance/differential/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ differential/
plan/ -- F8: plan.Evaluate
digest/ -- F9: digestprojection.*
materialized/ -- MaterializedObject::try_new / NewMaterializedObject
verdict/ -- the assembled five-field verdict artifact (build_verdict / verdict.Build)
```

One group per primitive, landed incrementally — matching how F6
Expand Down Expand Up @@ -203,3 +204,20 @@ is, only whether both sides agree on being numeric or not) — so no
vector here depends on it — but a future vector author relying on
exact-comparator behavior that distinguishes `16` from `16.0` by
*spelling* would hit a real asymmetry this layer does not paper over.

### `verdict/`

```text
input.json: the `plan/` group's input (intent, observed, observation, plan), plus an
optional `tamper` list of edits to the evaluate result before the artifact
is built: {op: drop_consumed|add_consumed|drop_field|set_object, path|value}
expected.json: an artifact vector: {verdict: <the five-key tree>, canonical: <its canonical
bytes as a string>}
a refusal vector: {error: {code, path}}
```

Both languages must produce the identical canonical bytes, and a success
vector's `object`/`fields` must equal the same-named `plan/` vector's pin. A
second test per language applies eight deliberate divergences to a correct
artifact and requires the comparison to catch each.

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"verdict": {
"conformance_plan_digest": "sha256:deb83850cc0f689d6f0c643694567125bae9b2268bdaa6cf5ac20dddf1094683",
"fields": {
"/nics/{name=nic-0,zone=eu}/subnet": "DRIFT"
},
"object": "DRIFT",
"observation_digest": "sha256:66fe8f76ccc06b0722d74a23fcd893e4167dcd3e6942218e1e6e08a60f333a72",
"verdict_schema_version": 1
},
"canonical": "{\"conformance_plan_digest\":\"sha256:deb83850cc0f689d6f0c643694567125bae9b2268bdaa6cf5ac20dddf1094683\",\"fields\":{\"/nics/{name=nic-0,zone=eu}/subnet\":\"DRIFT\"},\"object\":\"DRIFT\",\"observation_digest\":\"sha256:66fe8f76ccc06b0722d74a23fcd893e4167dcd3e6942218e1e6e08a60f333a72\",\"verdict_schema_version\":1}",
"why": "a multi-key element identity is part of the evaluated path in fields"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"intent": {
"nics": [
{"name": "nic-0", "zone": "eu", "subnet": "prod-a"}
]
},
"observed": {
"nics": [
{"name": "nic-0", "zone": "eu", "subnet": "prod-WRONG"}
]
},
"observation": [
{"path": "/nics/{name=nic-0,zone=eu}/subnet", "coverage": "observed"}
],
"plan": {
"scalars": [],
"collections": [
{
"path": "/nics",
"topology": "map",
"keys": ["name", "zone"],
"elements": [{"path": "subnet", "compare": "exact"}]
}
]
}
}
16 changes: 16 additions & 0 deletions conformance/differential/verdict/oci_conformant/expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"verdict": {
"conformance_plan_digest": "sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850",
"fields": {
"/memory_gb": "CONFORMANT",
"/network_interfaces/{name=nic-0}/subnet": "CONFORMANT",
"/network_interfaces/{name=nic-1}/subnet": "CONFORMANT",
"/shape": "CONFORMANT"
},
"object": "CONFORMANT",
"observation_digest": "sha256:5e24b8bb061a32c1ee09f8ede05374d7b13cfd89798985eb1a426781c93c2fe5",
"verdict_schema_version": 1
},
"canonical": "{\"conformance_plan_digest\":\"sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850\",\"fields\":{\"/memory_gb\":\"CONFORMANT\",\"/network_interfaces/{name=nic-0}/subnet\":\"CONFORMANT\",\"/network_interfaces/{name=nic-1}/subnet\":\"CONFORMANT\",\"/shape\":\"CONFORMANT\"},\"object\":\"CONFORMANT\",\"observation_digest\":\"sha256:5e24b8bb061a32c1ee09f8ede05374d7b13cfd89798985eb1a426781c93c2fe5\",\"verdict_schema_version\":1}",
"why": "every planned field matches: the artifact carries the CONFORMANT object verdict, the four evaluated paths, and the two digests"
}
40 changes: 40 additions & 0 deletions conformance/differential/verdict/oci_conformant/input.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"intent": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observed": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16.0,
"provider_id": "ocid1.instance.oc1..aaaa",
"lifecycle_state": "RUNNING",
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}, "private_ip": "10.0.1.17"},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}, "private_ip": "10.0.2.9"}
]
},
"observation": [
{"path": "/shape", "coverage": "observed"},
{"path": "/memory_gb", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-0}/subnet", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-1}/subnet", "coverage": "observed"}
],
"plan": {
"scalars": [
{"path": "/shape", "compare": "exact"},
{"path": "/memory_gb", "compare": "numeric"}
],
"collections": [
{
"path": "/network_interfaces",
"topology": "map",
"keys": ["name"],
"elements": [{"path": "subnet", "compare": "exact"}]
}
]
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"verdict": {
"conformance_plan_digest": "sha256:928988250ac70e1fc043022a372c2d72b5aa3a3018f98ac84ad3faa36e086fc4",
"fields": {
"/boot_volume": "OBSERVED_ABSENT",
"/memory_gb": "CONFORMANT",
"/network_interfaces/{name=nic-0}/subnet": "CONFORMANT",
"/network_interfaces/{name=nic-1}/subnet": "CONFORMANT",
"/shape": "CONFORMANT"
},
"object": "CONFORMANT",
"observation_digest": "sha256:449a85439d54f9c4497e80ec85070a927a8519f9e619728f8d83760d241885ab",
"verdict_schema_version": 1
},
"canonical": "{\"conformance_plan_digest\":\"sha256:928988250ac70e1fc043022a372c2d72b5aa3a3018f98ac84ad3faa36e086fc4\",\"fields\":{\"/boot_volume\":\"OBSERVED_ABSENT\",\"/memory_gb\":\"CONFORMANT\",\"/network_interfaces/{name=nic-0}/subnet\":\"CONFORMANT\",\"/network_interfaces/{name=nic-1}/subnet\":\"CONFORMANT\",\"/shape\":\"CONFORMANT\"},\"object\":\"CONFORMANT\",\"observation_digest\":\"sha256:449a85439d54f9c4497e80ec85070a927a8519f9e619728f8d83760d241885ab\",\"verdict_schema_version\":1}",
"why": "a field the intent does not declare and the observation finds absent is OBSERVED_ABSENT and does not drift the object"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"intent": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observed": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16.0,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observation": [
{"path": "/shape", "coverage": "observed"},
{"path": "/memory_gb", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-0}/subnet", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-1}/subnet", "coverage": "observed"},
{"path": "/boot_volume", "coverage": "absent"}
],
"plan": {
"scalars": [
{"path": "/shape", "compare": "exact"},
{"path": "/memory_gb", "compare": "numeric"},
{"path": "/boot_volume", "compare": "exact"}
],
"collections": [
{
"path": "/network_interfaces",
"topology": "map",
"keys": ["name"],
"elements": [{"path": "subnet", "compare": "exact"}]
}
]
}
}
16 changes: 16 additions & 0 deletions conformance/differential/verdict/oci_drift/expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"verdict": {
"conformance_plan_digest": "sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850",
"fields": {
"/memory_gb": "CONFORMANT",
"/network_interfaces/{name=nic-0}/subnet": "CONFORMANT",
"/network_interfaces/{name=nic-1}/subnet": "DRIFT",
"/shape": "CONFORMANT"
},
"object": "DRIFT",
"observation_digest": "sha256:69ec2c422d32b89c8bd0d060296982e8172e8fd6084984aee65c8325dd47a39f",
"verdict_schema_version": 1
},
"canonical": "{\"conformance_plan_digest\":\"sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850\",\"fields\":{\"/memory_gb\":\"CONFORMANT\",\"/network_interfaces/{name=nic-0}/subnet\":\"CONFORMANT\",\"/network_interfaces/{name=nic-1}/subnet\":\"DRIFT\",\"/shape\":\"CONFORMANT\"},\"object\":\"DRIFT\",\"observation_digest\":\"sha256:69ec2c422d32b89c8bd0d060296982e8172e8fd6084984aee65c8325dd47a39f\",\"verdict_schema_version\":1}",
"why": "a single element subnet mismatch drifts only that field; the artifact aggregates to DRIFT and still lists every evaluated path"
}
38 changes: 38 additions & 0 deletions conformance/differential/verdict/oci_drift/input.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"intent": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observed": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16.0,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-WRONG", "kind": "Subnet"}}
]
},
"observation": [
{"path": "/shape", "coverage": "observed"},
{"path": "/memory_gb", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-0}/subnet", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-1}/subnet", "coverage": "observed"}
],
"plan": {
"scalars": [
{"path": "/shape", "compare": "exact"},
{"path": "/memory_gb", "compare": "numeric"}
],
"collections": [
{
"path": "/network_interfaces",
"topology": "map",
"keys": ["name"],
"elements": [{"path": "subnet", "compare": "exact"}]
}
]
}
}
16 changes: 16 additions & 0 deletions conformance/differential/verdict/oci_not_comparable/expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"verdict": {
"conformance_plan_digest": "sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850",
"fields": {
"/memory_gb": "NOT_COMPARABLE",
"/network_interfaces/{name=nic-0}/subnet": "CONFORMANT",
"/network_interfaces/{name=nic-1}/subnet": "CONFORMANT",
"/shape": "CONFORMANT"
},
"object": "DRIFT",
"observation_digest": "sha256:10e27324b2fd39a6ed030ed9861e54e06e00634a159e16cceb346b510daf8e2b",
"verdict_schema_version": 1
},
"canonical": "{\"conformance_plan_digest\":\"sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850\",\"fields\":{\"/memory_gb\":\"NOT_COMPARABLE\",\"/network_interfaces/{name=nic-0}/subnet\":\"CONFORMANT\",\"/network_interfaces/{name=nic-1}/subnet\":\"CONFORMANT\",\"/shape\":\"CONFORMANT\"},\"object\":\"DRIFT\",\"observation_digest\":\"sha256:10e27324b2fd39a6ed030ed9861e54e06e00634a159e16cceb346b510daf8e2b\",\"verdict_schema_version\":1}",
"why": "an incomparable field makes the object DRIFT; the artifact keeps NOT_COMPARABLE as the field verdict"
}
38 changes: 38 additions & 0 deletions conformance/differential/verdict/oci_not_comparable/input.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"intent": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observed": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": "large",
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observation": [
{"path": "/shape", "coverage": "observed"},
{"path": "/memory_gb", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-0}/subnet", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-1}/subnet", "coverage": "observed"}
],
"plan": {
"scalars": [
{"path": "/shape", "compare": "exact"},
{"path": "/memory_gb", "compare": "numeric"}
],
"collections": [
{
"path": "/network_interfaces",
"topology": "map",
"keys": ["name"],
"elements": [{"path": "subnet", "compare": "exact"}]
}
]
}
}
16 changes: 16 additions & 0 deletions conformance/differential/verdict/oci_unobserved/expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"verdict": {
"conformance_plan_digest": "sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850",
"fields": {
"/memory_gb": "UNOBSERVED",
"/network_interfaces/{name=nic-0}/subnet": "CONFORMANT",
"/network_interfaces/{name=nic-1}/subnet": "CONFORMANT",
"/shape": "CONFORMANT"
},
"object": "INCOMPLETE",
"observation_digest": "sha256:7afeba3ff68ab2ce4f789c4dae85727371a75120e9f458a4c3b8c9c39ee692e3",
"verdict_schema_version": 1
},
"canonical": "{\"conformance_plan_digest\":\"sha256:d973c7d2e9cf02f49ad1f59889a0dd2aaabd98b41d30c07377c63fdafd387850\",\"fields\":{\"/memory_gb\":\"UNOBSERVED\",\"/network_interfaces/{name=nic-0}/subnet\":\"CONFORMANT\",\"/network_interfaces/{name=nic-1}/subnet\":\"CONFORMANT\",\"/shape\":\"CONFORMANT\"},\"object\":\"INCOMPLETE\",\"observation_digest\":\"sha256:7afeba3ff68ab2ce4f789c4dae85727371a75120e9f458a4c3b8c9c39ee692e3\",\"verdict_schema_version\":1}",
"why": "a planned field that was not observed makes the object INCOMPLETE; the observation digest changes with the coverage"
}
37 changes: 37 additions & 0 deletions conformance/differential/verdict/oci_unobserved/input.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{
"intent": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observed": {
"shape": "VM.Standard.E4.Flex",
"memory_gb": 16.0,
"network_interfaces": [
{"name": "nic-0", "subnet": {"ref": "prod-subnet-a", "kind": "Subnet"}},
{"name": "nic-1", "subnet": {"ref": "prod-subnet-b", "kind": "Subnet"}}
]
},
"observation": [
{"path": "/shape", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-0}/subnet", "coverage": "observed"},
{"path": "/network_interfaces/{name=nic-1}/subnet", "coverage": "observed"}
],
"plan": {
"scalars": [
{"path": "/shape", "compare": "exact"},
{"path": "/memory_gb", "compare": "numeric"}
],
"collections": [
{
"path": "/network_interfaces",
"topology": "map",
"keys": ["name"],
"elements": [{"path": "subnet", "compare": "exact"}]
}
]
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"error": {
"code": "E_INCONSISTENT_VERDICT",
"path": "/ghost"
},
"why": "a result that cannot be an evaluate result for the plan is refused, never digested into an artifact"
}
Loading
Loading