Switch conformance tests to common runner and align scenario coverage

.github/instructions/build-config.instructions.md

@@ -21,8 +21,8 @@ these, not just the first:
 - `[testenv] deps`: the factor-conditional `-r …/tests/requirements.<factor>.txt` lines plus
   `{[testenv]test_deps}` / `{[testenv]pytest_deps}`. Requirements install here, **not** in
   `commands_pre`.
-- `[testenv] commands`: the pytest line (which must `--ignore` `tests/test_conformance.py`) and the
-  separate `-conformance` pytest line.
+- `[testenv] commands`: the pytest line (which must `--ignore` `tests/conformance`) and the
+  separate `-conformance` pytest line (targeting `conformance.yaml`).
 - `[testenv:typecheck] deps`: `{toxinidir}/instrumentation/<pkg>[instruments]`.
 
 The uv workspace picks up new packages via the `instrumentation/*` glob in root `pyproject.toml`,

.github/instructions/instrumentation.instructions.md

@@ -119,12 +119,13 @@ Flag, with a link to the rule:
   responses) before committing. Examples: `authorization`, `openai-organization`,
   `openai-project`, `Set-Cookie`, and any response-body field tied to a real
   account.
-- Conformance: packages ship `tests/conformance/<scenario>.py` modules (each
-  defining a subclass of
-  `opentelemetry.test_util_genai.conformance.Scenario` that sets
-  `expected_spans`, `expected_metrics`, and implements `run(...)`) and a
-  `tests/test_conformance.py` that runs them via
-  `opentelemetry.test_util_genai.conformance.run_conformance`.
+- Conformance: packages ship `tests/conformance/conformance.yaml` (declaring
+  runner, mock server, environment variables, the `weaver.registry` override,
+  expected spans/metrics, and `expected_violations`), standalone scenario scripts
+  (`tests/conformance/<scenario>.py`) executed under `opentelemetry-instrument`, and a
+  committed `tests/conformance/data.json`. They run via
+  `pytest …/tests/conformance/conformance.yaml`. Flag a `conformance.yaml` with no
+  `weaver.registry` — it would validate against the runner's pin, not `versions.env`'s.
 
 ## 7. Examples
 

.github/renovate.json5

@@ -31,16 +31,14 @@
       ]
     },
     {
-      // Keep .weaver.toml schema URL pinned to the latest Weaver tag.
       "customType": "regex",
-      "managerFilePatterns": ["/^\\.weaver\\.toml$/"],
+      "managerFilePatterns": ["/^dev-requirements-conformance\\.txt$/"],
       "matchStrings": [
-        "#:schema https://raw.githubusercontent.com/open-telemetry/weaver/v(?<currentValue>[^/]+)/schemas/weaver-config.json"
+        "git\\+https://github\\.com/(?<packageName>open-telemetry/semantic-conventions-conformance)\\.git@(?<currentValue>[0-9a-f]{40})"
       ],
-      "datasourceTemplate": "github-tags",
-      "depNameTemplate": "open-telemetry/weaver",
-      "versioningTemplate": "semver-coerced",
-      "extractVersionTemplate": "^v(?<version>.*)$"
+      "datasourceTemplate": "git-refs",
+      "depNameTemplate": "open-telemetry/semantic-conventions-conformance",
+      "versioningTemplate": "git"
     }
   ],
   "packageRules": [

.github/skills/migrate-from-openinference/SKILL.md

@@ -165,8 +165,6 @@ violate:
    add an `expected_violation` in the scenario that covers the missing piece.
    All these must be documented in `MIGRATION_REPORT.md` as well, with links to the skipped scenario and the expected violation.
 
-8. **Do not modify weaver policies.**
-
 ## Augment mode: the package already exists
 
 The package already has a working, conformant implementation; OpenInference
@@ -527,23 +525,24 @@ Add current `util/opentelemetry-util-genai` and `instrumentation/opentelemetry-i
 
 ### 8. Conformance scenarios
 
-Author conformance scenarios using the **`write-conformance-tests`** skill —
-it's the generic procedure (scenario modules, the `test_conformance.py`
-runner, declared gaps, lib-specific assertions, weaver policies) and applies
-to any instrumentation. Migration-specific notes on top of that skill:
+Author conformance scenarios using the **`write-conformance-tests`** skill -
+it's the generic procedure (`conformance.yaml`, standalone scenario scripts,
+declared gaps, and `genai-mock-server`) and applies to any instrumentation.
+Migration-specific notes on top of that skill:
 
-- Drop OpenInference's `examples/` tree — its end-to-end demos are replaced
+- Drop OpenInference's `examples/` tree - its end-to-end demos are replaced
   by conformance scenarios, not migrated.
 - For an operation blocked by a util-genai/semconv gap, point the
-  `expected_violations` / `xfail` `reason=` at the gap row in
+  `expected_violations` `reason=` in `conformance.yaml` at the gap row in
   `MIGRATION_REPORT.md`.
 
-### 9. Cassettes (or a transport proxy)
+### 9. Cassettes (for unit tests)
 
 - Copy cassettes from OpenInference's `tests/cassettes/` (or wherever the OpenInference package
   parks them) into the migrated package's `tests/cassettes/`. Reuse names so existing
   unit tests keep loading them.
-- Reuse existing cassettes for conformance scenarios when they are applicable.
+- Conformance scenarios run against `genai-mock-server` (or mock transport / model), not cassettes.
+  Cassettes are used only for unit tests.
 - **AI-generated cassettes.** For a cassette OpenInference lacks and you
   can't record (no provider access), you may synthesize one from the
   provider's API reference via AI. Start it with a

.github/skills/review-migration/SKILL.md

@@ -258,15 +258,15 @@ async `def`, `Stream` / `AsyncStream` wrappers).
 
 For each distinct GenAI semconv operation the migrated package emits (`chat`,
 `embeddings`, `execute_tool`, `invoke_agent`, `invoke_workflow`,
-`create_agent`, …) there should be at least one happy-path scenario file under
-`tests/conformance/<op>.py` driven by `run_conformance(...)`. More
+`create_agent`, …) there should be at least one happy-path scenario script under
+`tests/conformance/<scenario>.py` configured in `tests/conformance/conformance.yaml`. More
 scenarios per operation are fine but never required.
 
 | Operation | Scenario file | Status |
 |---|---|---|
 
-Mention if conformance scenario is skipped, there are expected_violations,
-or `uv run tox -e py312-test-instrumentation-genai-<lib>` fails.
+Mention if conformance scenario is skipped, there are expected_violations in
+`conformance.yaml`, or `uv run tox -e py314-test-instrumentation-genai-<lib>-conformance` fails.
 
 #### 4c. Docstring / README coverage
 
@@ -309,8 +309,8 @@ are none, render `_No follow-up issues recommended._`
 
 - `.github/skills/migrate-from-openinference/SKILL.md` — the migration skill; it
   runs this review at its final step to produce `MIGRATION_REPORT.md`.
-- `.github/skills/write-conformance-tests/SKILL.md` — authoring the
-  conformance scenarios this report checks in §4b.
+- `.github/skills/write-conformance-tests/SKILL.md` - authoring the
+  conformance scenarios and conformance.yaml this report checks in §4b.
 - `.github/instructions/instrumentation.instructions.md` — the copilot
   PR-review rules for `instrumentation/**`; generic instrumentation
   violations are flagged there and not repeated in this report.