release: kern-llm on PyPI, the runtime and both engine plugins in one wheel

.github/workflows/release.yml

@@ -1,12 +1,14 @@
 name: Release
 
 # A tag `vX.Y.Z` on master builds `kern` and `kern-serve` for Linux x86_64
-# and aarch64 on Ubuntu 24.04, proves each binary starts there without a
-# CUDA toolkit, and publishes a GitHub release with the archives, one
-# SHA256SUMS, the installer, and build provenance. The target is a Linux
-# CUDA box, nothing older than the runner's glibc. A pull request that
-# touches this pipeline runs the build and packaging, never the publish.
-# The GPU gate is manual and comes after: docs/release.md.
+# and aarch64 on Ubuntu 24.04 and the `kern-llm` wheel (abi3, manylinux
+# 2.28) for both, proves each starts there without a CUDA toolkit, and
+# publishes a GitHub release with the archives, the wheels, one SHA256SUMS,
+# the installer and build provenance, then the wheels to PyPI by trusted
+# publishing: no token, PyPI trusts this workflow in the `pypi`
+# environment. A pull request that touches this pipeline or kern-py runs
+# the builds and checks, never the publish. The GPU gate is manual and
+# comes after: docs/release.md.
 
 on:
   push:
@@ -15,6 +17,7 @@ on:
     paths:
       - ".github/workflows/release.yml"
       - "scripts/**"
+      - "crates/kern-py/**"
   workflow_dispatch:
 
 permissions:
@@ -70,10 +73,52 @@ jobs:
           if-no-files-found: error
           retention-days: 3
 
+  wheel:
+    name: Wheel ${{ matrix.arch }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - arch: x86_64
+            runner: ubuntu-24.04
+          - arch: aarch64
+            runner: ubuntu-24.04-arm
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 60
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          persist-credentials: false
+
+      - name: Build
+        uses: PyO3/maturin-action@v1
+        with:
+          working-directory: crates/kern-py
+          manylinux: "2_28"
+          maturin-version: v1.14.1
+          args: --release --locked --out ../../wheels
+
+      - name: Check
+        run: |
+          set -euo pipefail
+          version=$(grep -m1 '^version = ' Cargo.toml | cut -d'"' -f2)
+          if [ "$GITHUB_REF_TYPE" = tag ] && [ "$GITHUB_REF_NAME" != "v$version" ]; then
+            echo "::error::tag $GITHUB_REF_NAME does not match workspace version $version"
+            exit 1
+          fi
+          scripts/check_wheel.sh "$version" wheels/*.whl
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: wheel-${{ matrix.arch }}
+          path: wheels/*.whl
+          if-no-files-found: error
+          retention-days: 3
+
   publish:
     name: Publish
     if: github.ref_type == 'tag'
-    needs: build
+    needs: [build, wheel]
     runs-on: ubuntu-24.04
     timeout-minutes: 15
     permissions:
@@ -87,7 +132,7 @@ jobs:
 
       - uses: actions/download-artifact@v4
         with:
-          pattern: kern-*
+          pattern: "*"
           path: dist
           merge-multiple: true
 
@@ -97,13 +142,16 @@ jobs:
           cd dist
           cat ./*.sha256 >SHA256SUMS
           rm ./*.sha256
+          sha256sum *.whl >>SHA256SUMS
           cp ../scripts/install.sh install.sh
           sha256sum -c SHA256SUMS
           ls -la
 
       - uses: actions/attest-build-provenance@v3
         with:
-          subject-path: dist/*.tar.gz
+          subject-path: |
+            dist/*.tar.gz
+            dist/*.whl
 
       - name: Create release
         env:
@@ -123,10 +171,34 @@ jobs:
           Install the latest release, or this one with \`KERN_VERSION=$GITHUB_REF_NAME\`:
 
               curl -fsSL https://kern-baa.pages.dev/install.sh | sh
+
+          The Python runtime and the vLLM and SGLang plugins, for Python 3.10 or newer:
+
+              pip install kern-llm==$VERSION
           EOF
           gh release create "$GITHUB_REF_NAME" dist/* \
             --repo "$GITHUB_REPOSITORY" \
             --verify-tag \
             --title "kern $VERSION" \
             --notes-file notes.md \
             --generate-notes
+
+  pypi:
+    name: PyPI
+    if: github.ref_type == 'tag'
+    needs: publish
+    runs-on: ubuntu-24.04
+    timeout-minutes: 15
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          pattern: wheel-*
+          path: wheels
+          merge-multiple: true
+
+      - uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          packages-dir: wheels/

README.md

@@ -263,14 +263,13 @@ cargo build --release
 Inside vLLM, the manifest is the model and vLLM keeps everything else:
 
 ```bash
-(cd crates/kern-py && maturin build --release) && pip install target/wheels/kern-*.whl
-pip install python/kern_vllm
+pip install kern-llm
 hf download Pegainfer/kern-qwen38-sm103 --local-dir kern-qwen38
 KERN_MANIFEST=kern-qwen38/manifests/qwen3.8-27b-vllm.json KERN_KERNELS=kern-qwen38/cubins \
 vllm serve Qwen/Qwen3.8-27B --hf-overrides '{"architectures": ["KernForCausalLM"]}'
 ```
 
-SGLang works the same way through `python/kern_sglang`: [docs/sglang.md](docs/sglang.md).
+SGLang works the same way through `kern_sglang`, in the same package: [docs/sglang.md](docs/sglang.md).
 `kern <cmd> --help` lists the flags; `crates/kern-run/src/config.rs`
 documents `kern.toml`.
 
@@ -298,7 +297,7 @@ golden-checked in CI:
 | `crates/kern-test` | The `kern test` harness: diff, tap, noise floor, fuzz, perf, verdict (pure, no CUDA) |
 | `crates/kern-run` | The `kern` binary: `run`, `test`, `bench`, `server`, `verify`, `cut`, `kernels` |
 | `crates/kern-serve` | Continuous batching and the OpenAI-compatible endpoint |
-| `crates/kern-py`, `python/` | `import kern`, and the vLLM and SGLang plugins |
+| `crates/kern-py` | `pip install kern-llm`: `import kern`, and the vLLM and SGLang plugins |
 | `tools/` | Capture, kernel mining, manifest generators: all model knowledge lives here |
 | `examples/` | Generated manifests, the artifact a provider ships |
 

crates/kern-py/README.md

@@ -0,0 +1,23 @@
+# kern-llm
+
+[kern](https://github.com/pegainfer-project/kern) runs a model as a manifest: a typed declaration of
+buffers, states, ops and programs, compiled kernels, and weights. This package is kern for Python:
+
+- `kern`: the runtime binding (`kern.Runtime`), the readers of a manifest's host contract
+  (`kern.host`) and the rank group over `torch.distributed` (`kern.group`);
+- `kern_vllm`: a vLLM plugin that runs a manifest as the model;
+- `kern_sglang`: the same for SGLang.
+
+```bash
+pip install kern-llm
+hf download Pegainfer/kern-qwen38-sm103 --local-dir kern-qwen38
+KERN_MANIFEST=kern-qwen38/manifests/qwen3.8-27b-vllm.json KERN_KERNELS=kern-qwen38/cubins \
+vllm serve Qwen/Qwen3.8-27B --hf-overrides '{"architectures": ["KernForCausalLM"]}' \
+  --max-num-seqs 128 --max-num-batched-tokens 8192 --enable-prefix-caching \
+  -cc '{"mode": 0, "cudagraph_mode": "FULL_DECODE_ONLY"}'
+```
+
+Linux x86_64 and aarch64, Python 3.10 or newer. The runtime loads the NVIDIA driver (CUDA 13, r580 or
+newer) and cuBLAS 13 when it first runs; torch comes from the engine. The plugins do nothing unless
+`KERN_MANIFEST` is set. Guides: [vLLM](https://github.com/pegainfer-project/kern/blob/master/docs/vllm.md),
+[SGLang](https://github.com/pegainfer-project/kern/blob/master/docs/sglang.md).

crates/kern-py/pyproject.toml

@@ -3,10 +3,30 @@ requires = ["maturin>=1.8,<2"]
 build-backend = "maturin"
 
 [project]
-name = "kern"
+name = "kern-llm"
+description = "The kern runtime for Python, and the plugins that run a kern manifest as a vLLM or SGLang model"
+readme = "README.md"
 requires-python = ">=3.10"
+license = "Apache-2.0"
 dynamic = ["version"]
+classifiers = [
+    "Programming Language :: Rust",
+    "Programming Language :: Python :: Implementation :: CPython",
+    "Operating System :: POSIX :: Linux",
+    "Environment :: GPU :: NVIDIA CUDA :: 13",
+]
+
+[project.urls]
+Source = "https://github.com/pegainfer-project/kern"
+Documentation = "https://kern-baa.pages.dev"
+
+[project.entry-points."vllm.general_plugins"]
+kern = "kern_vllm:register"
+
+[project.entry-points."sglang.srt.plugins"]
+kern = "kern_sglang:register"
 
 [tool.maturin]
 module-name = "kern._kern"
 python-source = "python"
+python-packages = ["kern_vllm", "kern_sglang"]

python/kern_sglang/kern_sglang/__init__.py → crates/kern-py/python/kern_sglang/__init__.py

@@ -5,7 +5,7 @@
 its forward is one or more programs of a manifest whose states are host
 states, bound to the tensors SGLang allocated for each layer.
 
-    pip install kern kern-sglang
+    pip install kern-llm
     KERN_MANIFEST=<manifest.json> KERN_KERNELS=<cubin dir> \\
       sglang serve --model-path <checkpoint> --page-size 64 --attention-backend trtllm_mha \\
         --mamba-radix-cache-strategy extra_buffer --disable-prefill-cuda-graph ...

python/kern_vllm/kern_vllm/__init__.py → crates/kern-py/python/kern_vllm/__init__.py

@@ -5,7 +5,7 @@
 the model: its forward is one or more programs of a manifest whose states
 are host states, bound to the pages vLLM allocated for each layer.
 
-    pip install kern kern-vllm
+    pip install kern-llm
     KERN_MANIFEST=<manifest.json> KERN_KERNELS=<cubin dir> \\
       vllm serve <checkpoint> --hf-overrides '{"architectures": ["KernForCausalLM"]}' ...
 

crates/kern-py/src/lib.rs

@@ -4,7 +4,7 @@
 //! buffers, host states, programs, vars — so it is the same for every host.
 //! What a particular engine calls its KV cache, its block table or its
 //! stream lives in that engine's adapter, in Python, next to the engine it
-//! tracks (`python/kern_vllm` for vLLM): an engine that changes its
+//! tracks (`kern_vllm` for vLLM): an engine that changes its
 //! internals changes the adapter, never this crate.
 //!
 //! The host reads the verified manifest as JSON ([`Runtime::manifest`]) to

docs/release.md

@@ -1,17 +1,25 @@
 # Release：怎么发一个 kern binary，什么时候算发完
 
-kern 的 release 是两个 binary：`kern` 与 `kern-serve`，Linux x86_64 与 aarch64
-各一份，不含 CUDA 库（runtime 在首次使用时 dlopen driver 与 cuBLAS），不含 Python，
-不含模型。目标机器就是一台 Linux CUDA 机：Ubuntu 24.04 或更新。装完能跑什么由 manifest 决定，不在本页范围。
+kern 的 release 是两样东西，Linux x86_64 与 aarch64 各一份，都不含 CUDA 库（runtime
+在首次使用时 dlopen driver 与 cuBLAS），不含模型：
 
-流水线在 `.github/workflows/release.yml`，打包与自检在
-`scripts/package_release.sh`，安装脚本在 `scripts/install.sh`。
+- 两个 binary：`kern` 与 `kern-serve`，不含 Python；目标机器是 Ubuntu 24.04 或更新。
+- PyPI 包 `kern-llm`：一个 abi3 wheel（Python 3.10+，manylinux 2.28），装的是
+  `import kern` 与 vLLM、SGLang 两个插件（`kern_vllm`、`kern_sglang`，entry point 注册，
+  不设 `KERN_MANIFEST` 时不做事）。torch 由引擎带，wheel 不依赖它。
+
+装完能跑什么由 manifest 决定，不在本页范围。
+
+流水线在 `.github/workflows/release.yml`，binary 的打包与自检在
+`scripts/package_release.sh`，wheel 的自检在 `scripts/check_wheel.sh`，安装脚本在
+`scripts/install.sh`。
 
 ## 用户看到的
 
 ```bash
 curl -fsSL https://kern-baa.pages.dev/install.sh | sh
-kern --version          # kern 0.2.0 (<commit>, cuda 13.0)
+kern --version          # kern 0.3.0 (<commit>, cuda 13.0)
+pip install kern-llm    # import kern; vllm / sglang serve 时插件自动加载
 ```
 
 - 网站的 `/install.sh` 是一条 302，指向最新 release 里的 `install.sh`；
@@ -34,8 +42,8 @@ kern --version          # kern 0.2.0 (<commit>, cuda 13.0)
 
 ## 流水线做什么
 
-1. **触发**：push 一个 `v*` tag。改到 workflow 或 `scripts/` 的 PR 也跑 build 与
-   打包（不 publish），所以流水线本身的改动在合并前就被 CI 验过。
+1. **触发**：push 一个 `v*` tag。改到 workflow、`scripts/` 或 `crates/kern-py` 的 PR
+   也跑 build、打包与 wheel（不 publish），所以流水线本身的改动在合并前就被 CI 验过。
 2. **构建**：两个 target 各自在 Ubuntu 24.04 runner 上原生编（aarch64 用
    GitHub 的 arm runner，不交叉），apt 装 protoc 与 libssl-dev，`--locked` 一次编出
    kern-run 与 kern-serve。cudarc 绑定的 CUDA 版本由 Cargo.toml 的
@@ -45,19 +53,27 @@ kern --version          # kern 0.2.0 (<commit>, cuda 13.0)
    tag；`NEEDED` 里没有任何 CUDA 库；`kern verify examples/qwen3-4b.json` 与
    `kern-serve --help` 跑通（无 GPU，证明 loader 满足）。
    任一条不过，release 不出。
-4. **发布**：合并两份 `.sha256` 成一份 `SHA256SUMS`，附上 `install.sh`，
-   对两个 tar.gz 做 GitHub build provenance attestation，`gh release create
-   --verify-tag`，notes 是固定的一段要求说明 + 自动生成的 commit 列表。
+4. **wheel**：两个 arch 各在原生 runner 上用 maturin-action 的 manylinux 2.28 容器编
+   `crates/kern-py`；`check_wheel.sh` 装进干净 venv，查 `kern.__version__` = tag、
+   两个插件包在、两个 entry point 在、扩展模块 `NEEDED` 里没有 CUDA 库。
+5. **发布**：合并两份 `.sha256` 与 wheel 的 sha256 成一份 `SHA256SUMS`，附上
+   `install.sh`，对 tar.gz 与 wheel 做 GitHub build provenance attestation，
+   `gh release create --verify-tag`，notes 是固定的一段要求说明 + 自动生成的 commit 列表。
    同名 release 已存在则拒绝，不覆盖资产。
+6. **PyPI**：GitHub release 出了之后，同一批 wheel 用 trusted publishing 上传
+   `kern-llm`：没有 token，PyPI 认的是 `pegainfer-project/kern` 的 `release.yml`
+   在 `pypi` environment 里跑出的 OIDC 身份（pypi.org 项目页 Publishing 里登记）。
+   单独一个 job：这步失败只红它自己，修好后 `gh run rerun <run-id> --failed`，
+   release 不用重发。PyPI 不允许覆盖同版本。
 
 ## 怎么发
 
 ```bash
 # 1. bump：workspace version 在根 Cargo.toml 的 [workspace.package]
-git commit -am "release: v0.1.0"
+git commit -am "release: v0.3.0"
 git push origin master
 # 2. tag
-git tag -a v0.1.0 -m "kern v0.1.0" && git push origin v0.1.0
+git tag -a v0.3.0 -m "kern v0.3.0" && git push origin v0.3.0
 # 3. 看 Actions 的 Release 跑绿
 gh run watch
 ```
@@ -70,16 +86,21 @@ GitHub runner 没有 GPU，流水线只能证明 binary 起得来。release 发
 ```bash
 curl -fsSL https://kern-baa.pages.dev/install.sh | sh
 kern --version                  # commit 与 tag 一致，cuda 13.0
-cd ~/kern-1 && kern test qwen3-4b   # 最后一行 PASS
+cd <kern checkout> && kern test qwen3-4b   # 最后一行 PASS
+pip install kern-llm==0.3.0 && python -c 'import kern; print(kern.__version__)'   # 引擎的 venv 里
 ```
 
-三行都过，release 才算发完。不过就删掉 release 与 tag 重发，0.x 不做 hotfix 版本：
+再在引擎里过一遍 conc1：[vllm.md](vllm.md) 与 [sglang.md](sglang.md) 开头那条命令起
+Qwen3.8-27B，greedy 输出与 `kern run` 逐 token 相同。都过，release 才算发完。
+
+不过时：PyPI 上的版本号用过就不能再传，所以不是删了重发，而是 yank 掉 PyPI 上那一版、
+删掉 GitHub release 与 tag，修好后 bump 一个 patch 版本再发：
 
 ```bash
-gh release delete v0.1.0 --yes --cleanup-tag
+gh release delete v0.3.0 --yes --cleanup-tag   # PyPI 那版在 pypi.org 的项目页 yank
 ```
 
 ## 不做的
 
-macOS、Windows、musl（静态 musl 的 dlopen 不能用）、老 glibc、Homebrew、PyPI、
+macOS、Windows、musl（静态 musl 的 dlopen 不能用）、老 glibc、Homebrew、sdist、
 docker 镜像。