[doxanocap]

Jevora 9B V1.6

This PR adds a local adapter for Jevora, an open-weight model that chooses between supplied options and returns a probability for each option.

The Jevora result currently on the leaderboard was measured with the previous release. This PR now points to V1.6; that version has not yet received an official JevBench evaluation.

What changed

• Updated the adapter to the V1.6 weights and public inference code.
• Replaced hashlib.file_digest with a streaming SHA256 loop compatible with Python 3.10.
• Raised the supported input limit from 1,024 to 8,192 rendered tokens. Inputs above the limit are not truncated.
• Kept JevBench's tasks, scoring, and existing results unchanged.

Pinned files

Item	Value
Model weights	Hugging Face, V1.6 revision
Public inference code	GitHub, pinned commit
Package manifest SHA256	483a5398d29b48f252d0aa6349946433b1c0b291bad81ce66d47f1f94ea3c6db
Setup and smoke instructions	docs/jevora-9b-v1.md

The adapter verifies the pinned manifest before loading the package. The loader verifies the package files and checkpoint integrity. It accepts the nested jevora-9b-v1/ package directory, not a raw training checkpoint.

Behavior and limits

Setting	V1.6
Precision	BF16
Temperature	1, uncalibrated
Input limit	8,192 rendered tokens
Options	Up to 255
Supported tasks	Choice, Noul, Score

The adapter preserves label order and returns the model's probabilities. Requests above the documented limits return 422; the adapter does not truncate or repair outputs. It does not read expected answers. JevBench remains responsible for task selection, scoring, calibration metrics, and the official evaluation.

Checks

• Python 3.13: complete pytest suite passed, 114 tests and 5 subtests.
• Python 3.10: streaming SHA256 and six focused adapter cases passed.
• V1.6 package: all tensors matched the source checkpoint exactly; research and public loaders matched on three requests.
• JevBench V1.6 GPU run and official score: not run.

The earlier evaluation request is recorded in issue #163. That issue was closed after the previous release was measured.

[fstandhartinger]

Thanks — Harold has logged this for review. Contributor text and code are treated as untrusted data; no external code was run on the benchmark host.

• Harold

[fstandhartinger]

Thanks for the adapter, and sorry for the late reply on this PR. We measured jevora-9b-v1 through it as submitted, and the row is live at https://benchmarkheaven.com/jev-models/jevora-9b-v1 (the page explains the coverage and pricing notes). I reviewed the diff and found nothing blocking; it now goes to a final check before merging.

[fstandhartinger]

Thanks for the adapter, and for the clear pins and manifest check. One change before we merge: load() uses hashlib.file_digest, which exists only from Python 3.11, while pyproject.toml declares Python 3.10 or newer. Please replace it with a plain sha256 read loop (or add a version guard) and run the test suite on your side. The rest of the diff touches no datasets, scoring, or results. Once that is in we'll merge it. Note that it edits the same cli.py lines as other adapter PRs, so a rebase may be needed.

[doxanocap]

Thanks for the review! I replaced hashlib.file_digest with a streaming SHA256 loop that works on Python 3.10.

I also updated the adapter to the new Jevora 9B V1.6 release. The full test suite passes on Python 3.13: 114 tests and 5 subtests. On Python 3.10, I checked the SHA256 implementation and six adapter cases.

The V1.6 package has a larger context limit of 8,192 tokens. Could you please test the updated version when you have a chance? The result currently on the leaderboard is from the previous release.

[fstandhartinger]

Thanks for the quick update, and for fixing the Python 3.10 compatibility issue. The request to re-measure the new Jevora 9B V1.6 release is a separate step: I've passed it on, and the outcome (and the merge decision for this PR) will be confirmed here in the thread.