Drupal's AI ecosystem adopted EEE 0.2.2 as its result format

[georgekastanis]

I maintain ai_eval, an open-source evaluation module for Drupal (the CMS). It is part of a broader effort to standardize evaluation contracts across the Drupal AI ecosystem, the Drupal Eval Commons umbrella: datasets, rubrics, judges, and result records.

For the result record we adopted eval.schema.json, pinned at v0.2.2, as our export format (cross-project tracking issue). Details:

• Envelopes stay strictly within the 0.2.2 schema, no extension fields. Drupal-specific context (Drupal and module versions, judge metadata) is carried in the native additional_details string maps.
• Each run exports evaluation_results items for avg_score and pass_rate, the latter carrying a Wilson confidence interval. LLM-judge results include judge-validation metadata (TPR/TNR against human labels, trust status), which felt aligned with this project's meta-evaluation mission.
• Our test suite validates every emitted envelope against the vendored auto-generated 0.2.2 JSON Schema using opis/json-schema. We noticed the Pydantic move in 0.2.2; consuming the generated artifact works fine for us.
• Envelopes are produced from stored eval runs by a Drush command (ai-eval:export-envelope).

We do not push anything to the datastore and do not need CMS frameworks on your roadmap; we pin versions on our side to insulate against pre-1.0 churn. If a CMS track ever becomes interesting to you, or you would like Drupal-context feedback on the schema, happy to help.

Thanks for MIT-licensing this. It saved us designing a result format from scratch.