feat(lm): sglang LM block — preflight the hardware, repair the runtime, serve from a named venv

CHANGELOG.md

@@ -81,6 +81,18 @@
 
 ## [Unreleased]
 
+### Added (sglang LM block, #199)
+- IR gained `SGLangServer` and an optional `IRDocument.lm` block: a graph can
+  declare the LM endpoint it wants (`provider: sglang`, `model`, `host`,
+  `port`, `args`, `startup_timeout_s`) instead of the caller passing
+  `--lm-url`/`--lm-model`. `stargraph run` resolves it before the first node —
+  attaching to a server already serving that model, else spawning
+  `python -m sglang.launch_server` and terminating it at run end. The block is
+  **excluded from the structural graph hash**: an endpoint is an environment
+  binding, not topology. `args` and a non-loopback `host` are operator-only —
+  a graph-declared value is refused unless re-stated as `--sglang-arg` /
+  `--sglang-host`. JSON schemas + openapi.json regenerated.
+
 ### Added (batteries-included, #189)
 - IR `RuleSpec.when` now accepts mapping sugar alongside the raw CLIPS
   string: `{node: <node-id>, <mirror-field>: <value>, ...}` compiles to

docs/how-to/authoring-format.md

@@ -51,6 +51,77 @@ works: the [prebuilt kinds](../reference/nodes/prebuilt.md), `tool`,
 `NodeBase`. Bare tool ids get `@1` appended (`std.web_search` →
 `std.web_search@1`).
 
+### `lm` (optional)
+
+Pin the model the graph's LLM nodes run against, so the graph carries its
+own endpoint instead of depending on the caller's flags:
+
+```yaml
+lm:
+  provider: sglang            # the only provider today
+  model: microsoft/phi-4      # --model-path, and the id the server must report
+  port: 41002                 # default 30000
+  args: [--attention-backend, triton]   # passed to sglang.launch_server verbatim
+  startup_timeout_s: 600      # weights take minutes on big models
+```
+
+`stargraph run` resolves it before the first node: it attaches to a server
+already serving that model on the port (and leaves it running), otherwise
+it launches `python -m sglang.launch_server`, waits for the endpoint to
+answer, and terminates it at the end of the run. The derived base URL +
+model configure the DSPy LM, so `--lm-url`/`--lm-model` become unnecessary
+(and are rejected alongside it). `--sglang-*` flags override this block
+field by field — `--sglang-port 41010` re-points it without editing the
+graph.
+
+Two fields are **operator-only**, because a graph file can be less trusted
+than the person running it — and this block is the only way a graph reaches
+a subprocess at all (every process-spawning std tool sits behind the
+default-deny capability gate):
+
+- `args` — passthrough argv into `sglang.launch_server`. A graph-declared
+  value is refused; re-state it as `--sglang-arg` to allow it.
+- a non-loopback `host` — the derived endpoint receives `--lm-key` and every
+  prompt. Refused unless the operator passes the same `--sglang-host`.
+
+`model`, `port` and `startup_timeout_s` stay graph-declarable.
+
+The block is not part of the graph hash: an endpoint is an environment
+binding, not topology.
+
+Before it launches anything, `stargraph run` checks that the machine and the
+interpreter agree:
+
+- **Hardware** is read from the vendor tools (`nvidia-smi`, `rocm-smi`,
+  `xpu-smi`, `npu-smi`), never from torch. `torch.cuda.is_available()` answers
+  "was this torch built with CUDA", not "does this box have a GPU" -- a
+  CPU-only wheel on a two-GPU machine says `False`.
+- **The runtime** is probed inside the interpreter that would be spawned. A
+  missing sglang, or a torch built for the wrong accelerator, is reported with
+  the install command for *that* platform. `--install-runtime` runs it;
+  without the flag nothing is installed and the run stops. Kernel drivers are
+  never touched -- a missing or too-old CUDA/ROCm driver can only be reported.
+- **A CPU torch is repaired explicitly**, because installing sglang does not
+  do it: `2.11.0+cpu` satisfies sglang's `torch==2.11.0` pin (PEP 440 ignores
+  the local version), so the resolver is happy and the CPU wheel stays.
+  Repair therefore runs in rounds -- install sglang, re-probe, then
+  force-reinstall torch off the CUDA index at the pin *that* sglang resolved
+  to. A round that changes nothing stops the loop rather than retrying.
+  SGLang publishes plain wheels for NVIDIA only, so ROCm, XPU, Ascend NPU and
+  Apple Metal are reported with a pointer to their platform page rather than a
+  command that would not work.
+- **The interpreter** is stargraph's own unless `--sglang-python` names
+  another one (a venv directory works). Preflight, weight fetch and launch all
+  move together, so a graph can run from a CPU-only venv and still serve from
+  the venv that has a CUDA sglang. It is a flag, never an `lm:` key -- naming
+  the interpreter to execute is operator-only.
+- **The weights** are fetched before the server starts, so `startup_timeout_s`
+  measures server boot rather than racing a multi-gigabyte download.
+- **The format** is validated, not rewritten. A GGUF repo is refused (that is
+  llama.cpp's format; sglang serves safetensors) with the servable repo named,
+  and an FP8 checkpoint on pre-sm_89 hardware warns. The graph always runs the
+  weights it declares.
+
 ### `routes`
 
 Declaration order is the default flow: with no rule firing, execution
@@ -111,6 +182,9 @@ overwritten.
 - Value routes branch on `verdict` only — standardize on it (both
   `classify` and `judge` already emit it).
 - One graph per file; `state` fields are flat primitives/containers.
+- `lm:` is honoured by `stargraph run` only — `stargraph serve` binds its
+  LM from its own `--lm-*` flags at boot, one endpoint for every graph it
+  serves.
 - For anything the sugar can't say (custom fact templates, multi-field
   `when` conditions, verifiers), write IR — see
   [Build a graph](build-graph.md).

docs/reference/cli.md

@@ -77,9 +77,66 @@ on `failed`.
 | `--lm-model NAME`    | str              | _(none)_               | LLM model identifier (e.g. `gpt-oss:20b`).                                 |
 | `--lm-key KEY`       | str              | `placeholder`          | API key for the LLM endpoint (`placeholder` works for ollama).             |
 | `--lm-timeout SEC`   | int              | `60`                   | LLM call timeout in seconds.                                               |
+| `--sglang-model NAME`| str              | _(none)_               | Serve this model with SGLang for the run; sets `--lm-url`/`--lm-model` from it. |
+| `--sglang-host HOST` | str              | `127.0.0.1`            | SGLang bind/probe host.                                                    |
+| `--sglang-port PORT` | int              | `30000`                | SGLang port.                                                               |
+| `--sglang-arg ARG`   | str (repeatable) | _(empty)_              | Extra argv passed through to `sglang.launch_server` verbatim.              |
+| `--sglang-timeout SEC` | int            | `600`                  | Seconds to wait for a launched SGLang server to answer.                    |
+| `--sglang-python PATH` | str            | _(this interpreter)_   | Interpreter (or venv directory) to serve from. |
+| `--install-runtime`  | flag             | `false`                | Install the sglang build matching the detected accelerator before launching. |
 
 `--quiet` and `--verbose` are mutually exclusive. `--lm-url` and
-`--lm-model` must be supplied together (or neither).
+`--lm-model` must be supplied together (or neither). A spawned SGLang server is
+preflighted first: hardware is detected from the vendor tools, the spawn
+interpreter's sglang/torch build is checked against it, and the weights are
+fetched before the startup clock starts. Without `--install-runtime` a runtime
+that cannot serve is reported with the exact install command and the run stops;
+nothing is ever installed implicitly, and kernel drivers are never touched.
+With the flag, repair runs in rounds: sglang first, then -- if the torch beside
+it is a CPU (or otherwise mismatched) build -- a `--force-reinstall` off the
+CUDA index at the pin sglang resolved to. Installing sglang alone does not fix
+that torch: `2.11.0+cpu` satisfies `torch==2.11.0`, so the resolver leaves it
+where it is.
+
+`--sglang-python` points the whole spawn -- preflight, weight fetch and launch
+-- at another interpreter, so the venv running stargraph never has to become
+the venv serving the model. A venv directory is accepted and resolved to its
+`bin/python`. There is deliberately no `lm:` key for it: the interpreter is
+argv into a subprocess, the same class of operator-only value as `args` and a
+non-loopback `host`.
+
+The `--sglang-*` flags bind the run to a local
+[SGLang](https://docs.sglang.ai/) server, and derive `--lm-url` /
+`--lm-model` from it — so they conflict with those two flags. Before the
+first node runs, `stargraph run` probes `http://host:port/v1/models`:
+
+- a server already serving that model is **attached to** and left running;
+- a server serving a *different* model is a loud error (pick another port);
+- nothing listening means one is launched
+  (`python -m sglang.launch_server --model-path ...`, plus every
+  `--sglang-arg` verbatim), waited on until it answers, and terminated —
+  process group included — when the run ends.
+
+The same binding can be declared in the graph itself as an `lm:` block
+(see [Author a graph in simple YAML](../how-to/authoring-format.md)); the
+flags override it field by field. A graph-declared block may set
+`model`/`port`/`startup_timeout_s` only — passthrough `args` and a
+non-loopback `host` are operator-only and must be re-stated as
+`--sglang-arg` / `--sglang-host`. Neither is part of the graph hash: like
+`--lm-url`, an endpoint is an environment binding, not topology.
+
+The end-of-run summary counts LM calls off the DSPy client, not off the event
+bus -- a `kind: dspy` node calls its LM directly and publishes no event. DSPy's
+disk cache (`~/.dspy_cache`) is on by default and is left on, so the line names
+cache hits when there are any:
+
+```
+✓ done in 1.2s  (1 steps, 1 llm calls, 1 cached)
+```
+
+A cached run reached no server. `--summary-json` carries the same three numbers
+as `llm_call_count`, `llm_cache_hits` and `tool_call_count` (tool calls are
+counted from `ToolCallEvent`, separately from LM calls).
 
 **Examples**
 
@@ -92,6 +149,11 @@ stargraph run graphs/triage.yaml --inspect
 
 # Bind a local LLM for dspy nodes
 stargraph run graphs/triage.yaml --lm-url http://localhost:11434 --lm-model gpt-oss:20b
+
+# Boot SGLang for the run (attaches instead if :41002 already serves it)
+stargraph run graphs/triage.yaml \
+  --sglang-model microsoft/phi-4 --sglang-port 41002 \
+  --sglang-arg=--attention-backend --sglang-arg=triton
 ```
 
 See also: [Concepts: IR](../concepts/ir.md),

docs/reference/ir-schema.md

@@ -61,6 +61,7 @@ and `nodes`; every other section defaults to an empty list / dict.
 | `parallel`     | `list[ParallelBlock]` | no       | `[]`       | Top-level parallel/join declarations.                                      |
 | `governance`   | `list[PackMount]`     | no       | `[]`       | Mounted Bosun rule packs.                                                  |
 | `migrate`      | `list[MigrateBlock]`  | no       | `[]`       | Hash-to-hash migration descriptors for resume.                             |
+| `lm`           | `SGLangServer \| None` | no      | `None`     | Local LM endpoint bound to the run (attach-or-launch SGLang). Environment binding, not topology — excluded from the structural hash. |
 
 ```yaml
 ir_version: "1.0.0"

docs/reference/openapi.json

@@ -1942,6 +1942,49 @@
             "title": "RuleSpec",
             "type": "object"
           },
+          "SGLangServer": {
+            "additionalProperties": false,
+            "description": "Declared LM endpoint: an SGLang server bound to this graph for a run.\n\n``stargraph run`` resolves this block before the first node executes: it\nprobes ``http://{host}:{port}/v1/models`` and **attaches** when a server\nalready serves ``model`` (left running afterwards -- it is not ours),\notherwise it spawns ``python -m sglang.launch_server``, waits for the\nendpoint to answer, and terminates it when the run ends. The derived\nbase URL + ``model`` configure the DSPy LM, so declaring this block is\nequivalent to passing ``--lm-url``/``--lm-model`` at a live endpoint.\n\nNot part of the structural graph hash: like ``--lm-url``, the endpoint is\nan environment binding, not graph topology.\n\nAttributes:\n    provider: Only ``\"sglang\"`` today; the discriminator for future\n        providers.\n    model: ``--model-path`` value; must equal the id the server reports\n        in ``/v1/models`` when attaching to an already-running server.\n    host: Bind address / probe host.\n    port: Bind port (SGLang's own default is 30000).\n    args: Extra argv passed through to ``sglang.launch_server``\n        verbatim (e.g. ``[\"--attention-backend\", \"triton\"]``).\n        Operator-only: a graph-declared value is refused unless the\n        operator re-states it as ``--sglang-arg`` (argv into a\n        subprocess is a code-execution surface, and a graph file can\n        be less trusted than the operator running it). The same holds\n        for a non-loopback ``host``, which would receive the API key\n        and every prompt.\n    startup_timeout_s: How long to wait for a spawned server to answer\n        before failing the run. Big models take minutes to load.",
+            "properties": {
+              "args": {
+                "items": {
+                  "type": "string"
+                },
+                "title": "Args",
+                "type": "array"
+              },
+              "host": {
+                "default": "127.0.0.1",
+                "title": "Host",
+                "type": "string"
+              },
+              "model": {
+                "title": "Model",
+                "type": "string"
+              },
+              "port": {
+                "default": 30000,
+                "title": "Port",
+                "type": "integer"
+              },
+              "provider": {
+                "const": "sglang",
+                "default": "sglang",
+                "title": "Provider",
+                "type": "string"
+              },
+              "startup_timeout_s": {
+                "default": 600,
+                "title": "Startup Timeout S",
+                "type": "integer"
+              }
+            },
+            "required": [
+              "model"
+            ],
+            "title": "SGLangServer",
+            "type": "object"
+          },
           "SkillRef": {
             "additionalProperties": false,
             "description": "Reference to a skill by namespaced id.",
@@ -2046,6 +2089,17 @@
             "title": "Ir Version",
             "type": "string"
           },
+          "lm": {
+            "anyOf": [
+              {
+                "$ref": "#/$defs/SGLangServer"
+              },
+              {
+                "type": "null"
+              }
+            ],
+            "default": null
+          },
           "migrate": {
             "items": {
               "$ref": "#/$defs/MigrateBlock"

examples/README.md

@@ -9,6 +9,7 @@ golden test (`tests/integration/test_examples.py`) and must reach
 | `hello.yaml` | Smallest graph: 2 nodes, 2 rules, 1 state field, no LLM | `stargraph run examples/hello.yaml --inputs message=hello` |
 | `pipeline.yaml` | Three-step linear routing, one Fathom rule per hop | `stargraph run examples/pipeline.yaml --inputs message=hello` |
 | `research-bot.yaml` | Authoring format (no `ir_version`): react + judge + verdict-routed feedback loop in ~20 lines | `stargraph run examples/research-bot.yaml --lm-url <url> --lm-model <model> --inputs question="..."` |
+| `sglang-qa.yaml` | The `lm:` block: the graph carries its own SGLang endpoint, so no `--lm-url`/`--lm-model` | `stargraph run examples/sglang-qa.yaml --inputs question="..."` |
 
 Trace rule firings without executing nodes:
 
@@ -19,8 +20,11 @@ stargraph run examples/hello.yaml --inspect
 ## What's intentionally *not* here
 
 Most examples use `echo`/`halt` nodes, so they stay self-contained and fast
-(`research-bot.yaml` is the exception: it demonstrates LLM nodes and needs
-`--lm-url`/`--lm-model`; its golden test drives it with a scripted stub LM).
+(`research-bot.yaml` and `sglang-qa.yaml` are the exceptions: both demonstrate
+LLM nodes. `research-bot.yaml` needs `--lm-url`/`--lm-model` and its golden test
+drives it with a scripted stub LM; `sglang-qa.yaml` declares its own endpoint and
+its golden test attaches to a loopback stub server, exercising the real attach
+path without a GPU).
 For the features that need more wiring, read the full graphs under
 [`demos/`](../demos/):
 

examples/sglang-qa.yaml

@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# examples/sglang-qa.yaml — a graph that carries its own LM endpoint.
+#
+# The `lm:` block is the point: instead of the caller supplying
+# --lm-url/--lm-model, the graph names the model it wants and `stargraph run`
+# resolves the endpoint before the first node executes. If a server is already
+# serving that model on that port it attaches and leaves it running; otherwise
+# it launches `python -m sglang.launch_server`, waits for the endpoint to
+# answer, and terminates it when the run ends.
+#
+# Run it (weights are fetched on first use; add --install-runtime to have
+# stargraph install the sglang build matching your GPU):
+#     stargraph run examples/sglang-qa.yaml --inputs question="what routes stargraph?"
+#
+# Point it at a server you already have, without editing this file:
+#     stargraph run examples/sglang-qa.yaml --sglang-port 41010 --inputs question="..."
+#
+# See the lowered IR (the `lm:` block becomes an SGLangServer model, and is
+# deliberately excluded from the structural graph hash — an endpoint is an
+# environment binding, not topology):
+#     stargraph compile examples/sglang-qa.yaml
+id: sglang-qa
+lm:
+  provider: sglang            # the only provider today
+  model: LiquidAI/LFM2.5-1.2B-Instruct
+  port: 30000                 # sglang's own default
+  startup_timeout_s: 600      # first run downloads weights
+state:
+  question: str
+  answer: str
+nodes:
+  # Every key except `kind` becomes the node's config, so this is a plain
+  # dspy.Predict over the declared signature — bound to the endpoint above.
+  ask: {kind: dspy, signature: "question -> answer"}
+routes:
+  ask: done