Skip to content

Commit ff8bebd

Browse files
SniderVirgil
andcommitted
docs(go): flesh out orm + inference, promote to content sidebar
orm - Stateless typed-bridge model: callers build intent, Mediums transport it - orm.Define schema declaration with field constraints - Bridge[T] typed query builder — Where/OrderBy/Limit/Get/First/Count - Insert/Update/Delete via Bridge or package-level helpers - From() aliases for joins + With() for eager-loaded relations - WhereGroup for compound (A AND (B OR C)) predicates - Polyglot story per RFC §12 — Go/PHP/TS produce identical Schema JSON inference - The Backend contract every GPU-specific runtime registers against - Stdlib-only — compiles on every platform regardless of GPU - Quick-start with blank-import backend registration pattern - LoadModel + Generate streaming token loop with options table - LoadOption vs GenerateOption split (load-time vs runtime knobs) - LoadTrainable for fine-tuning workflows (gradients + Step) - Service Register pattern, ProbeBus for token-level introspection - Capability registry for backend discovery Sidebar - Promoted orm + inference into "Packages — content" — total now 14 pages with real examples - Removed both from "Packages — index" stub list Co-Authored-By: Virgil <virgil@lethean.io>
1 parent 71b867d commit ff8bebd

3 files changed

Lines changed: 306 additions & 14 deletions

File tree

astro.config.mjs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ export default defineConfig({
6161
{ label: 'process', slug: 'go/process' },
6262
{ label: 'webview', slug: 'go/webview' },
6363
{ label: 'forge', slug: 'go/forge' },
64+
{ label: 'orm', slug: 'go/orm' },
65+
{ label: 'inference', slug: 'go/inference' },
6466
{ label: 'agent', slug: 'go/agent' },
6567
{ label: 'mcp', slug: 'go/mcp' },
6668
{ label: 'miner', slug: 'go/miner' },
@@ -93,12 +95,10 @@ export default defineConfig({
9395
{ label: 'html', slug: 'go/html' },
9496
{ label: 'i18n', slug: 'go/i18n' },
9597
{ label: 'ide', slug: 'go/ide' },
96-
{ label: 'inference', slug: 'go/inference' },
9798
{ label: 'log', slug: 'go/log' },
9899
{ label: 'ml', slug: 'go/ml' },
99100
{ label: 'mlx', slug: 'go/mlx' },
100101
{ label: 'netops', slug: 'go/netops' },
101-
{ label: 'orm', slug: 'go/orm' },
102102
{ label: 'p2p', slug: 'go/p2p' },
103103
{ label: 'rag', slug: 'go/rag' },
104104
{ label: 'scm', slug: 'go/scm' },

src/content/docs/go/inference.mdx

Lines changed: 155 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: inference
3-
description: Local inference adaptersDefault, LoadModel, LoadTrainable.
3+
description: Local text generationthe Backend contract every GPU-specific runtime implements.
44
head:
55
- tag: meta
66
attrs:
@@ -12,7 +12,12 @@ head:
1212
content: 'dappco.re/go/inference https://github.com/dappcore/go-inference https://github.com/dappcore/go-inference/tree/main{/dir} https://github.com/dappcore/go-inference/blob/main{/dir}/{file}#L{line}'
1313
---
1414

15-
Local inference adapters — Default, LoadModel, LoadTrainable. Built on the [`core/go`](/go/) primitives.
15+
The shared contract every text-generation backend implements. `TextModel`
16+
+ `Backend` + `Token` + `Message` are stdlib-only types that compile on
17+
every platform regardless of GPU availability; GPU-specific runtimes
18+
(Metal, ROCm, CUDA, …) register themselves at import time. Built on
19+
[`core/go`](/go/)`LoadModel` returns a [`Result`](/go/result/), zero
20+
external dependencies.
1621

1722
## Install
1823

@@ -23,10 +28,154 @@ go get dappco.re/go/inference@latest
2328
## Import
2429

2530
```go
26-
import "dappco.re/go/inference"
31+
import (
32+
"dappco.re/go/inference"
33+
34+
// Pick one or more — each blank import registers a backend
35+
_ "forge.lthn.ai/core/go-mlx" // "metal" backend on darwin/arm64
36+
// _ "forge.lthn.ai/core/go-rocm" // "rocm" backend on linux/amd64
37+
// _ "forge.lthn.ai/core/go-cuda" // "cuda" backend on linux/amd64
38+
)
39+
```
40+
41+
The base package compiles without any backend; calls fail cleanly with
42+
`no backend registered` rather than refusing to build. This is what lets
43+
the same binary target a laptop without a GPU and a Mac Studio without
44+
two `GOOS` builds.
45+
46+
## Quick start
47+
48+
```go
49+
r := inference.LoadModel("/path/to/safetensors/model/")
50+
if !r.OK { return r }
51+
model := r.Value.(inference.TextModel)
52+
defer model.Close()
53+
54+
for tok := range model.Generate(ctx, "Hello", inference.WithMaxTokens(256)) {
55+
fmt.Print(tok.Text)
56+
}
57+
```
58+
59+
Backend selection is automatic — the registry picks Metal on Apple
60+
Silicon, ROCm on Linux+AMD, CUDA on Linux+NVIDIA, in that preferred
61+
order — but you can pin explicitly:
62+
63+
```go
64+
r := inference.LoadModel(path, inference.WithBackend("metal"))
2765
```
2866

29-
## Status
67+
`inference.List()` returns the backend names that registered at import
68+
time, which is useful for runtime config + diagnostics.
69+
70+
## Generate options
71+
72+
Every generation call takes a variadic `GenerateOption`:
73+
74+
| Option | Effect |
75+
|---|---|
76+
| `WithMaxTokens(n)` | Cap output length |
77+
| `WithTemperature(t)` | Sampling temperature (0 = greedy) |
78+
| `WithTopK(k)` | Restrict sampling to top K logits |
79+
| `WithTopP(p)` | Nucleus sampling threshold |
80+
| `WithStopTokens(ids...)` | Halt on any of these token IDs |
81+
| `WithRepeatPenalty(p)` | Penalise repeated tokens |
82+
| `WithLogits()` | Emit raw logits per token (for analysis) |
83+
84+
## Load options
85+
86+
`LoadOption` configures the model **at load time** — runtime knobs that
87+
the backend can't change without reloading:
88+
89+
| Option | Effect |
90+
|---|---|
91+
| `WithBackend(name)` | Pin a specific backend instead of auto-select |
92+
| `WithContextLen(n)` | Override the model's default context length |
93+
| `WithGPULayers(n)` | How many transformer layers live on GPU vs CPU |
94+
| `WithParallelSlots(n)` | Number of concurrent generation slots |
95+
| `WithAdapterPath(path)` | Layer a LoRA adapter over the base model |
96+
97+
## Trainable models
98+
99+
For fine-tuning workflows, `LoadTrainable` returns a model that exposes
100+
gradients alongside generation:
101+
102+
```go
103+
r := inference.LoadTrainable(path, inference.WithAdapterPath("lora/"))
104+
if !r.OK { return r }
105+
trainable := r.Value.(inference.TrainableModel)
106+
107+
// Same generation surface
108+
for tok := range trainable.Generate(ctx, "prompt") { /* ... */ }
109+
110+
// Plus the training surface
111+
trainable.Backward(loss)
112+
trainable.Step()
113+
```
114+
115+
Backend support for the trainable path is opt-in — backends that ship
116+
inference-only will return `not supported`.
117+
118+
## Service registration
119+
120+
The canonical `core/go` Service shape lets a Core instance host an
121+
inference runtime that every consumer reaches through actions:
122+
123+
```go
124+
c := core.New(core.Options{})
125+
126+
if r := inference.RegisterCore(c); !r.OK { return r }
127+
128+
// Or with options
129+
svc := inference.NewService(inference.Options{
130+
DefaultBackend: "metal",
131+
ModelPath: "/srv/models/gemma-4-e2b",
132+
})
133+
if r := svc(c); !r.OK { return r }
134+
```
135+
136+
## Probe bus
137+
138+
`NewProbeBus(sinks...)` wires up token-by-token introspection — useful
139+
for metrics, eval harnesses, or live UIs that show what the model is
140+
"thinking":
141+
142+
```go
143+
bus := inference.NewProbeBus(
144+
inference.NewTelemetrySink(c),
145+
inference.NewLogitSink(eval),
146+
)
147+
148+
r := inference.LoadModel(path, inference.WithProbeBus(bus))
149+
```
150+
151+
Each `ProbeSink` receives every token + (optionally) the logits behind
152+
it. The base package ships the bus + interface; concrete sinks live in
153+
consumer code.
154+
155+
## Capabilities
156+
157+
Each registered backend declares a `Capability` list that the registry
158+
exposes for discovery. A consumer can pick a backend based on what it
159+
supports (training? batched inference? logits? KV-cache snapshots?):
160+
161+
```go
162+
for _, name := range inference.List() {
163+
caps := inference.BackendCaps(name)
164+
for _, cap := range caps {
165+
fmt.Printf("%s: %s (%s)\n", name, cap.ID, cap.Status)
166+
}
167+
}
168+
```
169+
170+
## Sibling packages
171+
172+
- [`go/ai`](/go/ai/) — higher-level multi-provider orchestrator built on
173+
inference plus remote API clients
174+
- [`go/mlx`](/go/mlx/) — Apple Silicon Metal backend that registers as
175+
`"metal"`
176+
- [`go/rag`](/go/rag/) — retrieval-augmented pipeline that consumes
177+
inference for the generation stage
178+
179+
## Source
30180

31-
Page in flight — content fills in as the package converges. Source of truth
32-
today is the README and inline docstrings in the repo.
181+
[`github.com/dappcore/go-inference`](https://github.com/dappcore/go-inference) — full source, contract tests, and the capability registry.

src/content/docs/go/orm.mdx

Lines changed: 149 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: orm
3-
description: Typed query layer over core/go/store — query builder, repositories, materialised views.
3+
description: Typed fluent communications bridge — schema, intent, Medium-transported.
44
head:
55
- tag: meta
66
attrs:
@@ -12,8 +12,11 @@ head:
1212
content: 'dappco.re/go/orm https://github.com/dAppCore/orm https://github.com/dAppCore/orm/tree/main{/dir} https://github.com/dAppCore/orm/blob/main{/dir}/{file}#L{line}'
1313
---
1414

15-
Typed query layer over [`core/go/store`](/go/store/) — query builder,
16-
repositories, materialised views. Built on the [`core/go`](/go/) primitives.
15+
A typed, fluent communications bridge over backend Media. Callers build
16+
**intent**; a Medium **transports** it. The same `Schema` declaration
17+
lands against DuckDB today, Postgres tomorrow, a Borg DataNode next week
18+
— the bridge is stateless, the backend's capability is what changes.
19+
Built on [`core/go`](/go/) — every query returns a [`Result`](/go/result/).
1720

1821
## Install
1922

@@ -27,7 +30,147 @@ go get dappco.re/go/orm@latest
2730
import "dappco.re/go/orm"
2831
```
2932

30-
## Status
33+
## Declare a Schema
3134

32-
Page in flight — content fills in as the package converges. Source of truth
33-
today is the README and inline docstrings in the repo.
35+
`orm.Define` collects field declarations into a `Schema` value. Field
36+
builders are chainable so constraints stay close to the column they
37+
describe:
38+
39+
```go
40+
type User struct {
41+
ID int64
42+
Name string
43+
Email string
44+
}
45+
46+
func (User) Schema() orm.Schema {
47+
return orm.Define(func(b *orm.Builder) {
48+
b.Name("users")
49+
b.PK("id")
50+
b.String("name").NotNull()
51+
b.String("email").Unique()
52+
})
53+
}
54+
55+
type Post struct {
56+
ID int64
57+
UserID int64
58+
Title string
59+
}
60+
61+
func (Post) Schema() orm.Schema {
62+
return orm.Define(func(b *orm.Builder) {
63+
b.Name("posts")
64+
b.PK("id")
65+
b.Int64("user_id")
66+
b.String("title")
67+
})
68+
}
69+
```
70+
71+
Schemas are values — pass them around, serialise them to JSON via
72+
`SchemaFromJSON`, ship them across the polyglot boundary (per RFC §12 the
73+
same `Schema()` declaration ports across Go, PHP, and TS implementations
74+
producing identical JSON shape).
75+
76+
## Mount a Medium
77+
78+
A Medium is what carries intent to a real backend. Today: in-memory
79+
(`NewMemium`), DuckDB (via [`go/store`](/go/store/)), more to follow.
80+
Mount once on a Core and every Bridge call routes through:
81+
82+
```go
83+
c := core.New()
84+
mem := orm.NewMemium()
85+
orm.Mount(c, "default", mem)
86+
```
87+
88+
## Query — typed Bridge
89+
90+
`Bridge[T]` is the typed query builder. Construct one from a Schema +
91+
Core, chain predicates, terminate with a fetch verb that returns
92+
`core.Result`:
93+
94+
```go
95+
bridge := orm.From[User](c)
96+
97+
r := bridge.
98+
Where("email", "=", "a@b.com").
99+
First()
100+
if !r.OK { return r }
101+
user := r.Value.(*User)
102+
103+
// Multi-row + ordering + limit
104+
r := orm.From[Post](c).
105+
Where("user_id", "=", user.ID).
106+
OrderBy("id", "desc").
107+
Limit(10).
108+
Get()
109+
posts := r.Value.([]*Post)
110+
111+
// Aggregates
112+
r := orm.From[Post](c).Where("user_id", "=", user.ID).Count()
113+
n := r.Value.(int)
114+
```
115+
116+
The fluent surface mirrors Eloquent's reading rhythm without dragging in
117+
Eloquent's machinery. Every predicate, every order, every join is just
118+
intent in a Go value until the Medium turns it into a backend call.
119+
120+
## Write — Insert, Update, Delete
121+
122+
Writes go through the same Bridge or the package-level helpers:
123+
124+
```go
125+
// Through the Bridge
126+
bridge.Insert(&User{Name: "alice", Email: "a@b.com"})
127+
bridge.Where("id", "=", 42).Update(map[string]any{"name": "renamed"})
128+
bridge.Where("id", "=", 42).Delete(&User{})
129+
130+
// Package-level — multi-row insert
131+
orm.Insert(c, &User{Name: "bob"}, &User{Name: "carol"})
132+
orm.Delete(c, &User{ID: 7})
133+
```
134+
135+
Every write returns `core.Result` — failures surface the bridge intent
136+
that didn't transport plus the underlying Medium error, so the call site
137+
sees what was attempted.
138+
139+
## Joins + relations
140+
141+
`Bridge.From(...)` aliases a related Schema for cross-table predicates;
142+
`Bridge.With(...)` declares eager-load relations the Medium should
143+
pre-fetch:
144+
145+
```go
146+
users := orm.From[User](c).
147+
From(orm.A{"posts": Post{}.Schema()}).
148+
Where("posts.title", "like", "%golang%").
149+
With("posts").
150+
Get()
151+
```
152+
153+
## Predicate grouping
154+
155+
Compound `(A AND (B OR C))` predicates with `WhereGroup`:
156+
157+
```go
158+
bridge.
159+
Where("status", "=", "active").
160+
WhereGroup(func(g *orm.Group) {
161+
g.Where("role", "=", "admin").
162+
OrWhere("role", "=", "owner")
163+
}).
164+
Get()
165+
```
166+
167+
## Sibling packages
168+
169+
- [`go/store`](/go/store/) — the persistence layer most orm Mediums route to
170+
- [`go/io`](/go/io/) — Medium transport contract orm extends
171+
- [`go/api`](/go/api/) — the polyglot boundary orm Schemas cross at runtime
172+
173+
## Source
174+
175+
[`github.com/dAppCore/orm`](https://github.com/dAppCore/orm) — full source,
176+
RFC, and the IMPLEMENTATION_PLAN that sequences the Go v1 build.

0 commit comments

Comments
 (0)