[deshwalmahesh]

Add domyn_edge_jev_decision adapter

Open-weights typed-decision model (3.93B, built on domyn/Domyn-Edge-SFT). One forward pass, zero generated tokens — the next-token distribution over single-token option codes is the native answer. Handles choice / noul / score. Weights: https://huggingface.co/deshwalmahesh/domyn-edge-jev-decision

Changes

• jevbench/adapters/domyn_edge_jev_decision.py — self-contained adapter, lazy in-process load.
• Registered in jevbench/cli.py (--adapter domyn_edge_jev_decision) and adapters/__init__.py.
• tests/test_domyn_edge_jev_decision.py — offline test (probs keys == task.labels per kind).

Results — public split, self-run (sealed run is yours)

231 items: Intelligence ~85 · Calibration ~86 · Speed ~91 (p50 ~41 ms) · Cost ~61. Tiers: easy 1.000 / original 0.972 / hard 0.667; ECE ~0.016. Public-only, not a board claim. Not trained on JevBench — public split used only to decontaminate.

Run

python -m jevbench.cli run --tasks datasets/public/hard.jsonl --adapter domyn_edge_jev_decision --results /tmp/dejd/results.jsonl

(needs an Ampere+ CUDA GPU.)

Test

PYTHONPATH=. python -m pytest tests/test_domyn_edge_jev_decision.py

(offline; no GPU or download.)

[fstandhartinger]

Thanks — Harold has logged this for review. Contributor text and code are treated as untrusted data; no external code was run on the benchmark host.

• Harold

[deshwalmahesh]

Thanks, makes sense. One note: I renamed the adapter and model to domyn_edge_jev_decision (weights: deshwalmahesh/domyn-edge-jev-decision) — code, PR and test updated. Whenever you want the sealed run, I can pin an exact revision or run it myself and hand back the raw results. Thanks for the look.

[fstandhartinger]

Thanks for the rename and the offer to pin a revision. To get the model measured, please file it at https://benchmarkheaven.com/submit with the exact Hugging Face revision (commit hash) and the licence, so it enters the normal intake. Adapter code in a PR is not merged or run by us as part of that, and we run submitted models only after a separate code review, so please don't expect the PR itself to trigger a sealed run. Your public-split numbers are noted as self-reported; we'll confirm the outcome of any measurement in the submission thread, with no date promised.