Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .agents/skills/gpustack-operator-docs/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ one, not to widen the overview.
| The local disk tier itself: what it renders, its bucket, its eviction, its host directory | `docs/modules/kv-cache/local-disk-tier.md` |
| Configuring a node that is mostly disk: why a group is never disk alone, the thin-segment shape, the memory floor | `docs/modules/kv-cache/disk-heavy-nodes.md` |
| A `KVCachePool` or `KVCachePoolBinding`: the grant, the reuse domain, a quota ceiling or grant, what a full quota does | `docs/modules/kv-cache/pool.md` |
| Standing a cache up end to end, or which object comes first: the pasteable four-object sequence | `docs/modules/kv-cache/walkthrough.md` |
| Standing a cache up end to end, or which object comes first: the pasteable four-object sequence | `docs/walkthroughs/kv-cache/shared-cache.md` |
| How a **Pod** consumes a pool: the inject label and annotations, the injected keys per engine, a refusal, the isolation record | `docs/modules/kv-cache/injection.md` |
| A `ModelArtifact`: its sources, resolution and revalidation, the manifest digest, how a `ModelDeployment` or an `Instance` mounts or downloads it, claim placement, the weight identity in KV keys | `docs/modules/model-delivery/artifact.md` |
| The `image` source of a `ModelArtifact`: the digest contract, building weights into an image, image-volume delivery, the version floors, double storage, kubelet image GC, registry mirrors | `docs/modules/model-delivery/image-source.md` |
Expand All @@ -70,6 +70,9 @@ one, not to widen the overview.
| A `NodeModelStore` or the `model-manager` plugin: a field and its writer, the status guard, mount authorization, materialization, a failure reason, collection, a metric | `docs/modules/model-delivery/node-store.md` |
| Running node delivery: the chart values, where the node's configuration comes from, reading a node, the watermark cap, switching delivery, where replicas land and turning the preference off, upgrading, removing the cache | `docs/modules/model-delivery/operations.md` |
| The `ModelDeployment` contract: the inherited reuse domain, the three override tiers, the owned-key table, the runner-image formula, prefill/decode pairing, the topology-placement field contract | `docs/modules/model-deployment/deployment.md` |
| A recorded Elastic EP expansion, its captured CR output and inference checks | `docs/walkthroughs/model-deployment/elastic-ep.md` |
| A recorded model prefetch sequence and captured artifact, prefetch and node-store output | `docs/walkthroughs/model-delivery/prefetch.md` |
| A fixed External DP group, its minimum arguments, per-rank requests, or measured Router coverage | `docs/walkthroughs/model-deployment/external-dp.md` |
| A `ModelDeployment` metrics snapshot, which series each field reads per engine, role and router, cache-hit scope or Pod scrape annotation | `docs/modules/model-deployment/metrics.md` |
| What a `ModelDeployment` status condition or published field means, and how to read them when a deployment misbehaves | `docs/modules/model-deployment/status.md` |
| How a prefill role and a decode role are paired: the connector each engine and router renders, `spec.router` and its fields, `spec.kvTransfer`, roles on different hardware, a role's own Service | `docs/modules/model-deployment/prefill-decode.md` |
Expand All @@ -89,7 +92,7 @@ one, not to widen the overview.
| A subcommand, one of its flags, its exit codes, its invocation | `docs/reference/commands.md` |
| A user-visible capability, a vendor's slicing support, the install flow | `README.md` |
| Vendor prerequisites, vendor GPU Operator coexistence | `docs/getting-started/vendor-prerequisites.md` |
| A recorded run with real output | `docs/getting-started/walkthrough.md`, or the walkthrough section of `docs/modules/devices/nvidia-mig.md` |
| A recorded run with real output | `docs/walkthroughs/devices/scheduling.md`, or the walkthrough section of `docs/modules/devices/nvidia-mig.md` |

A decision *record* — why an approach was chosen over another — belongs in `specs/`, not in `docs/`.
The docs state the rule that resulted; a `> **Why**` note carries only as much rationale as a reader
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ resources. [Architecture](../../getting-started/architecture.md) introduces the

---

**See also** — [Device Discovery](discovery.md) · [Walkthrough](../../getting-started/walkthrough.md)
**See also** — [Device Discovery](discovery.md) · [Walkthrough](../../walkthroughs/devices/scheduling.md)

**Next** → [Admission](admission.md) — the admission gates a request passes.
```
Expand Down Expand Up @@ -81,7 +81,7 @@ something that belongs in a list, a table, or another page.
| Cap | Limit | Exempt |
|---|---|---|
| prose paragraph | 5 lines / 500 rendered characters | fenced blocks, table rows, list *markers*, `> **Why**` notes, the footer |
| page length | 1000 lines | `docs/getting-started/walkthrough.md`, `docs/operate/*`, `docs/modules/devices/*-mig.md` — recordings and runbooks |
| page length | 1000 lines | `docs/walkthroughs/devices/scheduling.md`, `docs/operate/*`, `docs/modules/devices/*-mig.md` — recordings and runbooks |
| `##` sections | 10 per page, `## Contents` not counted | `docs/README.md`, `docs/reference/*` — both are lookup tables |

A paragraph nested under a list item is prose a reader still has to get through, so it is measured on
Expand All @@ -106,7 +106,7 @@ it as follows — when a page starts serving two modes at once, that is the mome

| Mode | Reader is… | Our pages |
|---|---|---|
| Tutorial | learning by doing | `README.md` Usage examples, `docs/getting-started/walkthrough.md`, the MIG walkthrough, `docs/modules/kv-cache/walkthrough.md` |
| Tutorial | learning by doing | `README.md` Usage examples, `docs/walkthroughs/devices/scheduling.md`, the MIG walkthrough, `docs/walkthroughs/kv-cache/shared-cache.md` |
| How-to | achieving a goal | `docs/operate/*`, `docs/modules/devices/*-mig.md`, `docs/operate/migration/*`, `docs/contribute/development.md`, a domain runbook (`docs/modules/model-delivery/operations.md`) |
| Reference | looking something up | `docs/modules/devices/requests.md`, `docs/reference/settings.md`, `docs/reference/*` |
| Explanation | building understanding | `docs/getting-started/architecture.md` and the deep pages under `docs/modules/` (contract pages read as reference, mechanism pages as explanation) |
Expand Down
31 changes: 29 additions & 2 deletions .agents/skills/gpustack-operator-docs/references/page-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ backend and links to it. And never the *enforcement* the Binding does not do: th
it is a grant and not an isolation boundary belongs here, but the mechanism a real boundary would need
is not this page's to design.

## `docs/modules/kv-cache/walkthrough.md`
## `docs/walkthroughs/kv-cache/shared-cache.md`

**Owns** — the ORDER, and nothing else: the four objects a working shared cache takes, one pasteable
manifest each, the check to run after each one, and the three configurations that read as working and
Expand Down Expand Up @@ -332,7 +332,7 @@ contract and does not teach.

**Never** — implementation. A rule's *enforcement point* is a link to `admission.md`.

## `docs/getting-started/walkthrough.md`
## `docs/walkthroughs/devices/scheduling.md`

**Owns** — recorded runs with real `kubectl` output and before/after comparisons.

Expand Down Expand Up @@ -428,3 +428,30 @@ supersession is recorded that way because it has no new spec to be recorded in.
`**Corrected after shipping.**`. A spec naming another spec by file name is the case that forces one:
when that file leaves the tree the name resolves to nothing, and leaving it alone preserves a pointer
rather than a record.

## `docs/walkthroughs/model-deployment/elastic-ep.md`

**Owns** — the user sequence for elastic TP2/DP2→4 expansion, captured CR output,
short inference checks, log inspection, and application release. Distinguishes independent replicas
from an elastic collective and states which path was measured.

**Never** — private engine code, raw infrastructure identities, or broad support claims inferred from
one runtime configuration. The Elastic EP guide owns the profile contract; the resize spec owns design analysis.

## `docs/walkthroughs/model-deployment/external-dp.md`

**Owns** — the fixed External DP serving sequence, minimum argument dependencies, captured CR output,
per-rank inference checks, Router coverage limits, and application cleanup. Links shared model prefetch steps.

**Never** — imply online collective resize from fixed-group inference, or claim all ranks receive Router
traffic from HTTP success alone. P/D combinations remain with their owning guide and follow-up issue.

## `docs/walkthroughs/model-delivery/prefetch.md`

**Owns** — the shared model preparation example, its fixed model revision and captured artifact,
prefetch, and node-store output. Model-serving walkthroughs link here before creating workloads.

## `docs/walkthroughs/_index.md`

**Owns** — the walkthrough entry point and module choices. Each module keeps its examples under
its own directory; capability guides link to the relevant example.
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ H2_CAP=10 # "##" sections in one page

# Cap exemptions, declared here rather than escaped inline. Each is a space-separated glob list.
# A recording and a runbook are as long as the hardware makes them.
LINE_CAP_EXEMPT='docs/getting-started/walkthrough.md docs/operate/* docs/modules/devices/*-mig.md'
LINE_CAP_EXEMPT='docs/walkthroughs/devices/scheduling.md docs/operate/* docs/modules/devices/*-mig.md'
# The index is a table of contents and a reference page is a lookup table: both are meant to be flat.
H2_CAP_EXEMPT='docs/README.md docs/reference/*'

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ spec:

The `ModelDeployment` below names the binding, which grants this namespace access to the cache.
Choose member nodes, quotas, block size and dtype for your engine before applying these resources.
The [KV Cache Walkthrough](./docs/modules/kv-cache/walkthrough.md) explains what to check at each step.
The [KV Cache Walkthrough](./docs/walkthroughs/kv-cache/shared-cache.md) explains what to check at each step.

### Model Delivery

Expand Down Expand Up @@ -299,7 +299,7 @@ ssh -i ~/.ssh/id_ed25519 -p 2222 root@127.0.0.1
```

The SSH session enters the workload container, where the accelerator and workspace are available.
The [walkthrough](./docs/getting-started/walkthrough.md) shows the resulting resources. Pods can
The [walkthrough](./docs/walkthroughs/devices/scheduling.md) shows the resulting resources. Pods can
request accelerators directly; see
[Accelerator Requests](./docs/modules/devices/requests.md) for complete manifests.

Expand Down
26 changes: 19 additions & 7 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ The capability guides are [Heterogeneous Devices](modules/devices/_index.md),

For a first workload, follow [Usage](../README.md#usage), then use
[Accelerator Requests](modules/devices/requests.md) to choose its resource keys. The
[Walkthrough](getting-started/walkthrough.md) records a run on a four-node cluster. Workloads that
need RDMA also need [RDMA Operations](modules/rdma/operations.md).
[Device Scheduling Walkthrough](walkthroughs/devices/scheduling.md) records a run on a four-node cluster.
Workloads that need RDMA also need [RDMA Operations](modules/rdma/operations.md).

For cluster operations, read [Architecture](getting-started/architecture.md) and [Installation
Modes](operate/installation-modes.md) first. The [settings](reference/settings.md), [vendor
Expand All @@ -27,6 +27,8 @@ and [RDMA](modules/rdma/operations.md) runbooks as those features are enabled. H
has separate runbooks for [NVIDIA](modules/devices/nvidia-mig.md),
[T-Head](modules/devices/thead-mig.md) and [Hygon](modules/devices/hygon-mig.md).

For complete examples, use the [Walkthroughs](walkthroughs/_index.md) entry point.

Start an upgrade with the relevant page under [migration](operate/migration/to-subcharts.md).

For code changes, read [Architecture](getting-started/architecture.md), then the relevant deep page in the
Expand Down Expand Up @@ -83,7 +85,7 @@ A skill link names a procedure; the entry's other fields carry shared facts.
- ID: `kv-cache`
- Use for: standing up a shared inference cache; pool grants and quotas; attaching a workload to a pool.
- Aliases: `KVCacheBackend`, `KVCachePool`, `Mooncake`, `prefix cache`, `injection`
- Guides: [KV Cache](modules/kv-cache/_index.md); setup order in [KV Cache Walkthrough](modules/kv-cache/walkthrough.md); store and grants in [KV Cache Backend](modules/kv-cache/backend.md) and [KV Cache Pool](modules/kv-cache/pool.md).
- Guides: [KV Cache](modules/kv-cache/_index.md); setup order in [KV Cache Walkthrough](walkthroughs/kv-cache/shared-cache.md); store and grants in [KV Cache Backend](modules/kv-cache/backend.md) and [KV Cache Pool](modules/kv-cache/pool.md).
- Specs: [backend](../specs/2026-08-28-kv-cache-backend.md) · [pool](../specs/2026-08-28-kv-cache-pool.md) · [injection](../specs/2026-08-28-kv-cache-injection.md) · [media and scaling](../specs/2026-09-05-kv-cache-media-and-scaling.md) · [high availability](../specs/2026-09-06-kv-cache-backend-high-availability.md)
- Code: store rendering and read-back in [pkg/worker/kvcache](../pkg/worker/kvcache/) · per-engine client config in [pkg/worker/kvcache/inject](../pkg/worker/kvcache/inject/) · `kv_cache_*.go` reconcilers under [pkg/worker/controllers/worker](../pkg/worker/controllers/worker/)
- Related: [Model Deployment](#model-deployment) (deployments and Pods consume a pool through injection)
Expand All @@ -94,7 +96,7 @@ A skill link names a procedure; the entry's other fields carry shared facts.
- ID: `model-delivery`
- Use for: giving model weights a stable identity; node caching and delivery; prefetch, peer sync, progress views and cache operations.
- Aliases: `ModelArtifact`, `NodeModelStore`, `ModelStore`, `ModelPrefetch`, `model cache`, `Hugging Face`
- Guides: [Model Delivery](modules/model-delivery/_index.md); the artifact contract in [Model Artifact](modules/model-delivery/artifact.md); the node cache in [Node Model Store](modules/model-delivery/node-store.md); the runbook in [Model Store Operations](modules/model-delivery/operations.md).
- Guides: [Model Delivery](modules/model-delivery/_index.md); the artifact contract in [Model Artifact](modules/model-delivery/artifact.md); the node cache in [Node Model Store](modules/model-delivery/node-store.md); the runbook in [Model Store Operations](modules/model-delivery/operations.md); a measured example in [Model Prefetch Walkthrough](walkthroughs/model-delivery/prefetch.md).
- Specs: [model artifact](../specs/2026-09-25-model-artifact.md) · [node model store](../specs/2026-09-25-node-model-store.md) · [prefetch](../specs/2026-09-27-model-prefetch.md) · [peer sync](../specs/2026-09-27-model-peer-sync.md) · [image source](../specs/2026-09-27-model-image-source.md)
- Code: Hub resolution in [pkg/modelartifact](../pkg/modelartifact/) · the per-node plugin and cache in [pkg/modelmanager](../pkg/modelmanager/) · cache configuration in [pkg/modelstore](../pkg/modelstore/)
- Related: [Model Deployment](#model-deployment) (deployments mount or download artifacts) · [KV Cache](#kv-cache) (the weight identity enters KV keys) · [GPU Instances](#gpu-instances) (an Instance can mount weights)
Expand All @@ -105,7 +107,7 @@ A skill link names a procedure; the entry's other fields carry shared facts.
- ID: `model-deployment`
- Use for: describing a serving workload's model, engine and roles; prefill/decode pairing; replica routing, status and metrics.
- Aliases: `ModelDeployment`, `prefill`, `decode`, `router`, `vLLM`, `SGLang`, `replica`
- Guides: [Model Deployment](modules/model-deployment/_index.md); the role contract in [Model Deployment Configuration](modules/model-deployment/deployment.md); pairing in [Prefill and Decode](modules/model-deployment/prefill-decode.md); replica selection and Router observation in [Routing](modules/model-deployment/routing.md); what a status field means in [Status](modules/model-deployment/status.md); retirement and drain in [Shutdown](modules/model-deployment/shutdown.md); in-place collective resize in [Elastic EP](modules/model-deployment/elastic-ep.md).
- Guides: [Model Deployment](modules/model-deployment/_index.md); the role contract in [Model Deployment Configuration](modules/model-deployment/deployment.md); pairing in [Prefill and Decode](modules/model-deployment/prefill-decode.md); replica selection and Router observation in [Routing](modules/model-deployment/routing.md); what a status field means in [Status](modules/model-deployment/status.md); retirement and drain in [Shutdown](modules/model-deployment/shutdown.md); in-place collective resize in [Elastic EP](modules/model-deployment/elastic-ep.md); a recorded run in [Elastic EP Walkthrough](walkthroughs/model-deployment/elastic-ep.md); fixed per-rank HTTP serving in [External DP Walkthrough](walkthroughs/model-deployment/external-dp.md).
- Specs: [consolidation](../specs/2026-09-18-kvcache-and-model-deployment-consolidation.md) · [role replica admission](../specs/2026-09-19-role-replica-admission-unit.md) · [P/D pairing and router](../specs/2026-09-12-model-deployment-pd-pairing-and-router.md) · [router implementations](../specs/2026-09-19-model-deployment-router-implementations.md) · [router qualification and drain](../specs/2026-10-01-s1-router-qualification-and-drain.md) · [elastic EP resize](../specs/2026-10-01-s2-elastic-ep-resize.md)
- Code: `model_deployment_*.go` reconcilers under [pkg/worker/controllers/worker](../pkg/worker/controllers/worker/) · native resize client in [pkg/worker/elasticengine](../pkg/worker/elasticengine/) · webhooks in [pkg/worker/webhooks/worker](../pkg/worker/webhooks/worker/) · router image under [pack/llm-router](../pack/llm-router/)
- Related: [KV Cache](#kv-cache) (a deployment can attach a shared cache) · [Model Delivery](#model-delivery) (weights reach replicas through an artifact) · [Topology Aware](#topology-aware) (the placement field constrains replicas)
Expand Down Expand Up @@ -196,7 +198,6 @@ A skill link names a procedure; the entry's other fields carry shared facts.
| [KV Cache Local Disk Tier](modules/kv-cache/local-disk-tier.md) | Configure local disk storage. |
| [KV Cache on Disk-Heavy Nodes](modules/kv-cache/disk-heavy-nodes.md) | Size memory on disk-heavy nodes. |
| [KV Cache Pool](modules/kv-cache/pool.md) | Grant and limit cache use. |
| [KV Cache Walkthrough](modules/kv-cache/walkthrough.md) | Create a working cache from backend to workload. |
| [KV Cache Injection Reference](modules/kv-cache/injection.md) | Attach a Pod to a pool. |

### Model delivery
Expand Down Expand Up @@ -233,12 +234,23 @@ A skill link names a procedure; the entry's other fields carry shared facts.
| [Accelerated Instances](modules/instances/_index.md) | Start an accelerator-backed workspace. |
| [Instance Metrics Reference](reference/instance-metrics.md) | Read an Instance’s resource use. |

### Walkthroughs

| Page | Description |
|---|---|
| [Walkthroughs](walkthroughs/_index.md) | Choose a complete example by module. |
| [Device Scheduling Walkthrough](walkthroughs/devices/scheduling.md) | Follow a recorded accelerator scheduling run. |
| [KV Cache Walkthrough](walkthroughs/kv-cache/shared-cache.md) | Create a shared cache from backend to workload. |
| [Model Prefetch Walkthrough](walkthroughs/model-delivery/prefetch.md) | Cache weights and verify captured model CR output. |
| [Model Deployment Walkthroughs](walkthroughs/model-deployment/_index.md) | Choose a serving shape and prepare GPUs. |
| [Elastic EP Walkthrough](walkthroughs/model-deployment/elastic-ep.md) | Grow DP width with TP fixed and verify inference. |
| [External DP Walkthrough](walkthroughs/model-deployment/external-dp.md) | Verify per-rank inference and Router coverage. |

### Start here

| Page | Description |
|---|---|
| [Architecture](getting-started/architecture.md) | See the operator’s four-stage path. |
| [Walkthrough](getting-started/walkthrough.md) | Follow a recorded cluster run. |
| [Vendor Prerequisites](getting-started/vendor-prerequisites.md) | Prepare each manufacturer’s driver. |

### Cluster operations and upgrades
Expand Down
Loading
Loading