Skip to content

v0.2.0 is tagged and unreleased, so PyPI still serves the build with the grounding bug #75

Open

Description

@ChelseaKR

Measured

v0.2.0 is a signed annotated tag on origin/main, pyproject.toml at that
tag says version = "0.2.0", and CHANGELOG.md at that tag has its
## [0.2.0] - 2026-09-07 section. Everything the release gate checks is
already true.

What is missing is the trigger. There is no GitHub Release for v0.2.0, and
release.yml has not run since 2026-08-19:

$ gh release list -R ChelseaKR/gauntlet
v0.1.0  Latest  v0.1.0  2026-08-16T04:49:48Z

$ gh run list -R ChelseaKR/gauntlet --workflow release.yml --limit 5
2026-08-19  workflow_dispatch  success  release
2026-08-16  release            failure  v0.1.0

So PyPI still serves 0.1.0, uploaded 2026-08-19:

$ curl -s https://pypi.org/pypi/gauntlet-evals/json | jq -r '.info.version, .urls[0].upload_time_iso_8601'
0.1.0
2026-08-19T04:38:32.402744Z

Why this one is worth doing rather than noting

pip install gauntlet-evals — the install line the README gives — hands a
reader a build that predates four things 0.2.0's changelog records:

  • run --record / --replay, the merge gate that contacts nothing;
  • verify and sign, so an edited pack is detectable;
  • lint DIR, checking a case directory before a run spends a request;
  • and the correctness entry "An unverifiable citation was counted as
    grounded"
    — a published build that scores a gate wrong in the permissive
    direction.

This is also the one package in the portfolio with a live external reader.
#27 is an interoperability enquiry from the maintainer of another eval schema
who read cases.py, results.py and the five gate modules before writing. If
that reader installs from PyPI rather than cloning, they get the pre-0.2.0
shapes, which are not the shapes discussed in that thread.

What it needs

Publishing a GitHub Release for the existing v0.2.0 tag, or dispatching
release.yml against it. That is an owner action (the workflow verifies the
tag signature and then uploads under trusted publishing), and I have not
touched the tag or the workflow.

One thing to look at while doing it: pyproject.toml carries both
license = "Apache-2.0" and a License :: OSI Approved :: Apache Software License classifier. Recent build backends reject that combination, and the
0.2.0 build is the first one that would meet whichever hatchling the runner
picks up.

Prepared with AI assistance; reviewed before submission.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions