diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c2bdb3d..4be3c37 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,7 +26,7 @@ jobs: - name: Import inside the declared env run: | nix develop --command python -c \ - "import python_gate, rich; print(python_gate.greet())" + "import pyra, python_gate, rich; print(pyra.greet()); assert python_gate.square(12) == pyra.square(12) == 144" uv: name: uv sync --locked @@ -37,4 +37,6 @@ jobs: - name: Sync from uv.lock (packages, not a closure) run: uv sync --locked - name: Run via the uv-managed env - run: uv run python-gate + run: | + uv run pyra + uv run python-gate diff --git a/README.md b/README.md index c2c96b7..d517abb 100644 --- a/README.md +++ b/README.md @@ -1,40 +1,82 @@ -# python-gate +![Pyra: a gold faceted pyramid with a flame, beside the word Pyra](docs/brand/pyra-social-preview.jpg) + +# Pyra + +**Formerly [python-gate](https://github.com/Hedronite/python-gate).** The repository moved to [Hedronite/pyra](https://github.com/Hedronite/pyra); GitHub still redirects the old URL. + +Pyra is a closed-world hybrid runtime for Python, aimed at ML work. Nix is the gate. uv owns the expression. `nix run` is the product. **This is a GitHub template.** Use it as *Use this template*, or: ```bash -nix flake new -t github:Hedronite/python-gate ./my-app +nix flake new -t github:Hedronite/pyra ./my-app ``` -Nix is the gate. uv owns the expression. `nix run` is the product. +What this tree builds today is Level 3: pure Python, one virtualenv, a wrapped application. CUDA, PyO3, and Bend are the Level 4 direction, described below. They are not in this tree. -`mkVirtualEnv` / `withPackages` builds a world; `mkApplication` / `wrapProgram` locks the door; nix-ld is the FHS lie (unsafe, Level 2). +`mkVirtualEnv` / `withPackages` builds a world; `mkApplication` / `wrapProgram` locks the door; nix-ld is the FHS lie (Level 2). A Python shop that is not on NixOS still `uv sync`s from `uv.lock`. A Nix user or CI `nix develop`s and `nix run .#`s the same closure. Two proofs: `uv.lock` (packages) and `flake.lock` (builders). **A lockfile is not a closure.** -This repository is the working example *and* the cloneable template. v1 is Level 3: no CUDA, no PyO3, no native extensions. +This repository is the working example *and* the cloneable template. ## The ladder -| Level | What you have | What you do not have | +| Level | What you have | Where it stands | | --- | --- | --- | -| 0 | Host Python, `pip install` | A lock, a world, a product | -| 1 | `uv.lock` / `poetry.lock` | A closure. Builders are the host | -| 2 | nix-ld / steam-run / `LD_LIBRARY_PATH` | Bit-identity. This is the FHS lie | -| **3** | **uv2nix `mkVirtualEnv` + `mkApplication`. This template** | CUDA, PyO3, native extensions | -| 4 | Native/FFI bits patched into the store | (out of scope for v1) | +| 0 | Host Python, `pip install` | No lock, no world, no product | +| 1 | `uv.lock` / `poetry.lock` | Packages are pinned. Builders are still the host. Not a closure | +| 2 | nix-ld / steam-run / `LD_LIBRARY_PATH` | A wheel can run. This is the FHS lie: not bit-identical | +| **3** | **uv2nix `mkVirtualEnv` + `mkApplication`. This template, today** | Pure Python. The closure is the product | +| 4 | The same closure, plus native code that Nix evaluated in | Direction only. PyO3/maturin kernels, Bend 2 components with machine-checked laws, optional CUDA. Not in this tree | + +Level 2 can run a wheel. It is not a closed world. It is not bit-identical. + +## Direction + +Level 4 is the evolution being explored: one Nix store path in which ordinary Python, PyO3/Rust extensions, and Bend 2 components can coexist. The product would still be `nix run`. The pure-Python inner loop would stay available, and would stay the default for packages that need nothing else. Proof failures, where a Bend component is present, would fail the build. The same closure would serve a person and an agent. + +Nothing in this section is implemented in this repository. Read it as a direction and a set of open questions. + +The shape under discussion is three layers, still one project: + +- **Control plane.** Orchestration, data, tracking, experiments, serving, tool interfaces. Ordinary Python, managed by uv, realized by uv2nix. This is what Level 3 already is. +- **Kernel plane.** Two extension mechanisms, both built inside the Nix evaluation and installed into the same virtualenv. PyO3/maturin for memory-safe or ecosystem-shaped kernels (sampling, tokenizers, loaders, rust-numpy, small CUDA kernels). Bend 2 for components whose correctness properties are stated as laws and checked by the Bend compiler before the extension exists — shape-safe transforms, proved round-trips, tree-parallel reductions, policy components. +- **The closed world as the runtime.** People and agents work inside `nix develop` or against the wrapped product, and call those kernels as ordinary Python functions. + +Packaging would stay on the current base: a flake using pyproject-nix, uv2nix, and pyproject-build-systems. Pure Python would keep the Level 3 path. A PyO3 package would declare the maturin build backend, bring a `Cargo.lock`, and be injected into the uv2nix set. A Bend component would ship `.bend` sources, `LAWS.bend`, and `PROOF.bend`; the derivation would run a pinned Bend compiler. CUDA would be an optional overlay. Default configurations would stay CPU-only. + +The dev shell would keep the peace treaty and, when asked, offer rustc, maturin, crane, and a pinned Bend binary. + +Open questions: -Level 2 can run a wheel. It is not a closed world. Do not claim it is bit-identical. +- Should PyO3 packages be built with crane, for incremental Rust caching, or is a lighter Cargo lock import plus build-system overlays enough? +- What is the bridge from Bend 2 into CPython: a maintained bend-python build, or a thin CFFI/PyO3 wrapper around Bend-generated C? +- The intent is that an open or false proof stops the product. How should `LAWS.bend` and `PROOF.bend` be wired so that holds, and so the extension imports from the same interpreter as the pure Python? +- How is the CUDA overlay selected so it stays opt-in, and so CPU and CUDA are each one closure rather than a leak between the two? +- Agent grading today is flake eval, an import inside the declared env, `uv sync --locked`, and the same closure. When a project contains Bend components, that list would grow a proof check. What is the smallest CI addition that does that and stays quiet for pure-Python projects? +- What should the reference template contain once any of this is real: one pure-Python control-plane example (this repo, now), one non-trivial PyO3 extension, one non-trivial Bend 2 component, and a documented CUDA recipe? + +The kinds of work this is aimed at: specialist-model fine-tuning and distillation where a kernel or an evaluation gate is proved; shape-safe data planes that hand arrays to PyTorch or JAX; policy or tool components with machine-checked behavior; a training or inference artifact any collaborator can `nix run`. Dense contiguous linear algebra stays in PyTorch or JAX. Maturin and a Bend bridge are the two extension paths being considered first. Other native build systems can join later through the same overlay mechanism. CPU and proofs are the part to get right first; Bend's GPU backends are a later question, wherever those backends themselves allow it. + +The peace treaty and the dual-lockfile model stay. Level 2 stays a named escape hatch. + +If you are cloning the template today, you are cloning Level 3. ## Init ```bash # GitHub template, or: -nix flake new -t github:Hedronite/python-gate ./my-app +nix flake new -t github:Hedronite/pyra ./my-app cd ./my-app -# Rename the project: pyproject.toml [project] name, [project.scripts], -# src//, and `projectName` in flake.nix. Then: +# Rename the project when you clone it: +# - pyproject.toml [project] name and [project.scripts] +# - src// +# - tool.uv.build-backend.module-name +# - projectName in flake.nix +# This repo keeps a python-gate script and a python_gate module as aliases +# for the old name. Drop both in your project. uv lock ``` @@ -42,9 +84,11 @@ Without Nix (inner loop): ```bash uv sync -uv run python-gate +uv run pyra ``` +`uv run python-gate` runs the same entry point. That alias exists so the old script name keeps working. + With Nix (same world, locked door): ```bash @@ -52,6 +96,8 @@ nix develop # store venv + peace treaty nix run .# # the product; network-free ``` +`nix run .#python-gate` is the same product under the old app name. + ## Add a dependency uv writes the expression. Nix consumes it. Do not hand-translate `uv.lock` into `python3Packages`. @@ -79,33 +125,50 @@ The Nix shell and uv share one interpreter. They do not share authority over the On NixOS, if you must run an **unpatched** FHS binary, set `NIX_LD` and `NIX_LD_LIBRARY_PATH` (nix-ld). **Never set `LD_LIBRARY_PATH` as the default.** That is a Level 2 leak into a Level 3 world. -## Must not +## Closed-world rules + +These keep the Level 3 world one world. They are the rules of what this template builds today. -- Keep a second venv (`.venv`) beside the store venv. `uv run` inside `nix develop` is how you grow a second world. -- Use poetry2nix. It is unmaintained. This template is uv + uv2nix. -- Claim Level 2 (nix-ld / `LD_LIBRARY_PATH`) is bit-identical. -- Edit `uv.lock` or `flake.lock` by hand. -- Hand-translate `uv.lock` into `python3Packages`. -- Set `LD_LIBRARY_PATH` as the default on NixOS. +- One venv: the store env. A `.venv` beside it is a second world. Inside `nix develop`, that is what a bare `uv run` grows. +- uv + uv2nix. poetry2nix is unmaintained. +- Level 2 (nix-ld / `LD_LIBRARY_PATH`) is not bit-identical. +- `uv.lock` and `flake.lock` are generated. They are not hand-edited. +- Dependencies enter through uv. They are not a hand-written `python3Packages` translation. +- On NixOS, `LD_LIBRARY_PATH` is not the default. An unpatched FHS binary uses `NIX_LD` and `NIX_LD_LIBRARY_PATH`. ## Agent grading A change is done when all of these pass: -1. **Flake eval.** `nix flake show` lists `packages..default`, `devShells..default`, and `templates.default`. Systems: `x86_64-linux`, `aarch64-linux`, `aarch64-darwin` (nixos-unstable dropped Intel Macs). -2. **Import inside the declared env.** `nix develop --command python -c "import python_gate, rich"` succeeds. -3. **`uv sync --locked`.** The non-Nix inner loop still reproduces from `uv.lock`. +1. **Flake eval.** `nix flake show` lists `packages..default`, `apps..default`, `devShells..default`, and `templates.default`. Systems: `x86_64-linux`, `aarch64-linux`, `aarch64-darwin` (nixos-unstable dropped Intel Macs). `apps.python-gate` is the old app name, still present. +2. **Import inside the declared env.** `nix develop --command python -c "import pyra, python_gate, rich"` succeeds. `python_gate` is the alias of `pyra`. +3. **`uv sync --locked`.** The non-Nix inner loop still reproduces from `uv.lock`. `uv run pyra` and `uv run python-gate` both run. 4. **Same closure.** `nix run .#` and `nix develop` use the same Python set (same interpreter, same uv2nix overlay). The product is the wrapped application, not a hope. CI on `ubuntu-latest` runs `nix build` / `nix run` (Determinate Nix) and `uv sync --locked` (setup-uv). Green means both proofs held, not that one was skipped. +A future Bend proof check belongs on this list only once a project contains Bend components. It is not part of grading today. + +## Names + +| | Current | Alias kept | +| --- | --- | --- | +| Distribution | `pyra` | | +| Import | `import pyra` | `import python_gate` | +| Console script | `pyra` | `python-gate` | +| `nix run` | `nix run .#` (`apps.default` runs `bin/pyra`) | `nix run .#python-gate` | +| Flake check | `pyra-runs` | `python-gate-runs` (same derivation) | +| Template | `templates.default` | URL is `github:Hedronite/pyra`. The output name is `default` | + +`packages.default`, `devShells.default`, and `templates.default` kept their attribute names. + ## Example app -`src/python_gate` is a console script plus `rich`. `nix run .#` prints versions and `square(12)` — a pure function, no HTTP. That is the point: the closure is enough. +`src/pyra` is a console script plus `rich`. `src/python_gate` re-exports it. `nix run .#` prints versions and `square(12)` — a pure function, no HTTP. That is the point: the closure is enough. ## Further reading -Short doctrine: [`docs/theses.md`](docs/theses.md). +Short doctrine, and the Level 4 notes: [`docs/theses.md`](docs/theses.md). Upstream builders (2026 map — do not use last year's): diff --git a/docs/brand/pyra-social-preview.jpg b/docs/brand/pyra-social-preview.jpg new file mode 100644 index 0000000..210c0eb Binary files /dev/null and b/docs/brand/pyra-social-preview.jpg differ diff --git a/docs/theses.md b/docs/theses.md index 641bbc2..ed63b01 100644 --- a/docs/theses.md +++ b/docs/theses.md @@ -1,5 +1,7 @@ # Twelve theses +These are the working doctrine of Pyra, formerly python-gate. They describe the Level 3 world this repository builds today. What Level 4 might become is below the theses. It is a direction. It is not code in this tree. + 1. **The Python environment is part of the program.** An interpreter, a set of wheels, and a script are one artifact. Shipping the script alone is a wish. 2. **A lockfile is not a closure.** `uv.lock` pins packages. `flake.lock` pins builders. The closure is the store paths those two proofs evaluate to. @@ -22,4 +24,24 @@ 11. **One Python set.** `devShells.default` and `packages.default` share the same interpreter and the same uv2nix overlay. Editables on the shell are a lens, not a second world. The peace treaty (`UV_NO_SYNC`, `UV_PYTHON`, `UV_PYTHON_DOWNLOADS=never`, unset `PYTHONPATH`) is how they stay one. -12. **This template is Level 3.** No CUDA, no PyO3, no native extension in v1. Level 4 is a different door. Do not pretend Level 2 is this one. +12. **What this template builds today is Level 3.** Pure Python. No native extension in the closure. Level 4 — PyO3/maturin kernels, Bend 2 components with machine-checked laws, optional CUDA, same store path — is the direction being explored. Level 2 is still not bit-identical. + +## Where Level 4 might go + +The same gate, a wider world. Ordinary Python would stay the control plane. Native code would be allowed in only when Nix evaluates it, and it would live in the same virtualenv. + +Three ideas are in play: + +- **PyO3 / maturin** for performance-critical or ecosystem-shaped kernels (sampling, tokenizers, loaders, rust-numpy interop, small CUDA kernels). +- **Bend 2** for components whose properties are stated as laws and checked by the Bend compiler before the extension is built. Shape-safe transforms, proved round-trips, tree-parallel reductions, policy pieces. +- **CUDA as an explicit mode.** CPU would remain what you get when you ask for nothing. torch, JAX, and anything else that needs the GPU would receive `cudaPackages` only under that mode. + +Open questions, none of them settled here: + +- crane, or a lighter Cargo lock import plus build-system overlays, for the Rust build? +- bend-python, or a thin CFFI/PyO3 wrapper around Bend-generated C, as the bridge into CPython? +- The intent is that an open or false Bend proof stops the product from building. How should `LAWS.bend` and `PROOF.bend` be wired so that is true, and so the resulting extension imports like any other module? +- How is the CUDA overlay selected so it stays opt-in and still evaluates to one closure? +- When a project actually contains Bend components, agent grading would grow a proof check. What is the smallest addition that does that and stays quiet on pure-Python projects? + +The pure-Python path, the peace treaty, and the two lockfiles stay in every version of this picture. This direction is not a plan to replace PyTorch or JAX for dense linear algebra. Maturin and a Bend bridge are the two extension paths under discussion. Other native build systems can be added later by the same overlay mechanism, if they earn it. Bend on GPU is available only where Bend's own backends allow it; CPU and proofs are the part worth getting right first. diff --git a/flake.nix b/flake.nix index e4caacd..7614e1a 100644 --- a/flake.nix +++ b/flake.nix @@ -1,5 +1,5 @@ { - description = "Closed-world Python: Nix is the gate, uv owns the expression"; + description = "Pyra (formerly python-gate): closed-world Python. Nix is the gate, uv owns the expression"; inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; @@ -44,8 +44,10 @@ ]; forAllSystems = lib.genAttrs systems; - # Rename this (and src/, and [project.scripts]) when you clone the template. - projectName = "python-gate"; + # Rename this when you clone the template, along with [project] name, + # [project.scripts], src//, and tool.uv.build-backend.module-name. + # `python-gate` / `python_gate` are compatibility aliases for the old name. + projectName = "pyra"; workspace = uv2nix.lib.workspace.loadWorkspace { workspaceRoot = ./.; }; @@ -73,18 +75,23 @@ ); in { - # `nix flake new -t github:Hedronite/python-gate` + # `nix flake new -t github:Hedronite/pyra` + # Output name is `default`. templates.default = { path = ./.; - description = "Closed-world Python: Nix is the gate, uv owns the expression"; + description = "Pyra (formerly python-gate): closed-world Python. Nix is the gate, uv owns the expression"; welcomeText = '' - # python-gate + # Pyra + Formerly python-gate. Nix is the gate. uv owns the expression. `nix run` is the product. - - Non-Nix: `uv sync` then `uv run python-gate` + - Non-Nix: `uv sync` then `uv run pyra` + (`uv run python-gate` is the same script) - Nix / CI: `nix develop` and `nix run .#` + This template is Level 3 (pure Python). Level 4 is a direction, + not a feature of this tree. Commit both lockfiles. A lockfile is not a closure. See README.md for the ladder, the peace treaty, and agent grading. ''; @@ -111,6 +118,11 @@ type = "app"; program = "${self.packages.${system}.default}/bin/${projectName}"; }; + # Former name. Same product as `default`. + python-gate = { + type = "app"; + program = "${self.packages.${system}.default}/bin/python-gate"; + }; }); devShells = forAllSystems ( @@ -151,11 +163,16 @@ let pkgs = nixpkgs.legacyPackages.${system}; app = self.packages.${system}.default; + pyraRuns = pkgs.runCommand "pyra-runs" { } '' + ${app}/bin/pyra > $out + # Compat script. Same entry point as pyra. + ${app}/bin/python-gate > /dev/null + ''; in { - "${projectName}-runs" = pkgs.runCommand "${projectName}-runs" { } '' - ${app}/bin/${projectName} > $out - ''; + pyra-runs = pyraRuns; + # Former check attribute. Same derivation as pyra-runs. + python-gate-runs = pyraRuns; } ); }; diff --git a/pyproject.toml b/pyproject.toml index 3cfd92f..8b6ad10 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,18 +1,27 @@ [project] -name = "python-gate" +name = "pyra" version = "0.1.0" description = "Closed-world Python: Nix is the gate, uv owns the expression" readme = "README.md" requires-python = ">=3.12" license = "MIT" -authors = [{ name = "Hedronite" }] +authors = [{ name = "Evan Eugene Pincham" }] dependencies = [ "rich>=13", ] [project.scripts] -python-gate = "python_gate:main" +pyra = "pyra:main" +# Former console script. Same entry point as `pyra`. +python-gate = "pyra:main" [build-system] requires = ["uv_build>=0.11.0,<0.13.0"] build-backend = "uv_build" + +# `python_gate` is the pre-rename import path. Both modules ship in this +# distribution so `import pyra` and `import python_gate` keep working. +# Drop `python_gate` from this list when you clone the template and do not +# want the alias. +[tool.uv.build-backend] +module-name = ["pyra", "python_gate"] diff --git a/src/pyra/__init__.py b/src/pyra/__init__.py new file mode 100644 index 0000000..8954e8c --- /dev/null +++ b/src/pyra/__init__.py @@ -0,0 +1,28 @@ +"""Tiny closed-world example. The product is `nix run`; this module is the payload. + +Pyra was formerly python-gate. ``import python_gate`` remains as an alias. +""" + +from importlib.metadata import version + +from rich.console import Console +from rich.table import Table + + +def square(n: int) -> int: + """A pure function. `nix run` must not need the network.""" + return n * n + + +def greet() -> str: + return f"{version('pyra')} · rich {version('rich')} · square(12)={square(12)}" + + +def main() -> None: + table = Table(title="Pyra · closed world") + table.add_column("proof") + table.add_column("value") + table.add_row("pyra", version("pyra")) + table.add_row("rich", version("rich")) + table.add_row("square(12)", str(square(12))) + Console().print(table) diff --git a/src/python_gate/__init__.py b/src/python_gate/__init__.py index 7060133..21de36f 100644 --- a/src/python_gate/__init__.py +++ b/src/python_gate/__init__.py @@ -1,25 +1,9 @@ -"""Tiny closed-world example. The product is `nix run`; this module is the payload.""" +"""Compatibility alias for :mod:`pyra`. -from importlib.metadata import version +python-gate was renamed to Pyra. Prefer ``import pyra``. +This module stays so existing ``import python_gate`` keeps working. +""" -from rich.console import Console -from rich.table import Table +from pyra import greet, main, square - -def square(n: int) -> int: - """A pure function. `nix run` must not need the network.""" - return n * n - - -def greet() -> str: - return f"{version('python-gate')} · rich {version('rich')} · square(12)={square(12)}" - - -def main() -> None: - table = Table(title="python-gate · closed world") - table.add_column("proof") - table.add_column("value") - table.add_row("python-gate", version("python-gate")) - table.add_row("rich", version("rich")) - table.add_row("square(12)", str(square(12))) - Console().print(table) +__all__ = ["greet", "main", "square"] diff --git a/uv.lock b/uv.lock index 887f7fe..f702dd9 100644 --- a/uv.lock +++ b/uv.lock @@ -1,5 +1,5 @@ version = 1 -revision = 3 +revision = 5 requires-python = ">=3.12" [[package]] @@ -33,7 +33,7 @@ wheels = [ ] [[package]] -name = "python-gate" +name = "pyra" version = "0.1.0" source = { editable = "." } dependencies = [