OpenTelemetry Semantic Conventions Conformance

Does an instrumentation actually emit what the semantic conventions say it should? This repo answers that the same way for every library, every implementation and every language: run a small program that exercises the library, collect what it emits through Weaver live-check, and check it against expectations declared in YAML.

tools/runner/	the runner. Generic — it carries no semantic conventions of its own
tools/gen-ai/	what makes a run a GenAI run: the registry pin, the advice policies, and a mock LLM server so scenarios are deterministic without cassettes
tools/database/	what makes a run a database run: the upstream registry pin, database span classification, and ephemeral database backends
tools/http/	the same for HTTP: the upstream registry pin, and the test client that drives both sides of the domain
tools/python/	the launcher a Python scenario runs under when programmatic configuration is needed for a scenario
tools/java/	what every JVM scenario shares, in any domain: the launcher that builds and runs one, the Gradle convention plugins, and the SDK bootstrap
tools/js/	what every Node scenario shares: the launcher that installs a build, and the SDK bootstrap
tools/ruby/	what every Ruby scenario shares: the launcher that installs and runs its locked bundle, plus scenario lifecycle and SDK helpers
tools/dotnet/	what every .NET scenario shares: the launcher that builds and runs one, the shutdown protocol every scenario needs, and the SDK bootstrap the ones measuring library instrumentation own
tools/php/	what every PHP scenario shares: the portable Composer and built-in-server launcher, plus environment support
tools/go/	what every Go scenario shares: the launcher that builds and runs a scenario, the shutdown protocol, and the SDK bootstrap
scenarios/gen-ai/	the GenAI scenarios and the coverage they produce
scenarios/database/	the database scenarios and the coverage they produce
scenarios/http/	the HTTP scenarios and the coverage they produce

A conformance directory names the wrapper it wants under runner:, so one command runs any of them:

pip install -e tools/runner -e tools/database/runner \
  -e tools/gen-ai/mock-server -e tools/gen-ai/runner
otel-conformance path/to/directory --report-only

See the runner's README for what a scenario and its conformance.yaml look like.

Maintainers

• Christophe Kamphaus, Independent
• Jay DeLuca, Grafana Labs
• Josh Suereth, Google
• Liudmila Molkova, Google
• Trask Stalnaker, Microsoft

For more information about the maintainer role, see the community repository.

Approvers

• None

For more information about the approver role, see the community repository.