Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ export default defineConfig({
label: 'Fly',
items: [
{ label: 'Deploy Fly Machines Offline', slug: 'tutorials/local-fly' },
{ label: 'Reconcile Fly Machines', slug: 'tutorials/fly-machines-reconcile' },
{ label: 'Fly Deploy with Checkpoint Rollback', slug: 'tutorials/fly-deploy-rollback' },
{ label: 'Managed Agents Worker on Sprites', slug: 'tutorials/sprites-managed-agent-worker' },
],
Expand Down
164 changes: 164 additions & 0 deletions docs/src/content/docs/tutorials/fly-machines-reconcile.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
---
title: Reconcile Fly Machines
description: flyApply reconciles typed Machines against the live Fly control plane with no state file — create, no-op re-apply, in-place update, and owned-only prune that leaves foreign machines alone. Runs offline against a persistent mudflaps emulator, or real Fly by dropping one override.
---

import { Aside } from '@astrojs/starlight/components';

The [`fly-reconcile`](https://github.com/INTENTIUS/chant/tree/main/examples/fly-reconcile)
example shows chant's differentiator on Fly: a **stateless reconcile**. Fly's
Machines API has no server-side declarative apply and chant keeps **no state
file**, so `flyApply` does the reconcile itself — it diffs your typed resources
against what the platform reports, and prunes only the machines it owns.

Where [`local-fly`](/chant/tutorials/local-fly/) boots and tears down the
emulator each run, this example points at a **persistent** mudflaps, so you run
the op repeatedly and watch the reconcile converge.

## The infra

`src/infra.ts` is one app, a persistent volume, and two machines — `web` (which
mounts the volume) and `worker`:

```ts
import { App, Machine, Volume, MachineConfig, MachineGuest, MachineMount, Fly } from "@intentius/chant-lexicon-fly";

const app = new App({ name: "fly-reconcile-demo", org_slug: Fly.OrgSlug });
const data = new Volume({ name: "data", region: "iad", size_gb: 1 });
const guest = new MachineGuest({ cpu_kind: "shared", cpus: 1, memory_mb: 256 });

const web = new Machine({
name: "web",
region: "iad",
config: new MachineConfig({
image: "flyio/hellofly:latest",
guest,
mounts: [new MachineMount({ volume: "data", path: "/data" })],
}),
});
const worker = new Machine({ name: "worker", region: "iad", config: new MachineConfig({ image: "flyio/hellofly:latest", guest }) });

export { app, data, web, worker };
```

The serializer stamps `managed-by: chant` into each machine's `config.metadata`
on its own — the ownership marker the owned-only prune reads back. `flyApply`
creates the volume before the machine that mounts it (the dependency is on the
wire — the volume is POSTed first), and FLY011 checks at build time that the
mount names a declared volume.

## The op

`ops/fly.op.ts` is `Build → Apply` only — no emulator lifecycle, so state
persists between runs:

```ts
import { Op, phase, build } from "@intentius/chant/op";
import { flyApplyStep } from "@intentius/chant-lexicon-fly";

export default Op({
name: "fly",
overview: "Reconcile the plan against a running mudflaps (create / update / prune)",
taskQueue: "fly",
phases: [
phase("Build", [build(".", { script: "build:fly" })]),
phase("Apply", [flyApplyStep("dist/fly.json", { endpoint: "http://localhost:4280", prune: true })]),
],
});
```

## Run it

Boot mudflaps once, then run the op:

```bash
docker run -d --rm -p 4280:4280 --name mudflaps ghcr.io/intentius/mudflaps:0.4.1

cd examples/fly-reconcile
npm install

chant run fly
```

The first run creates everything:

```
created: app/fly-reconcile-demo
created: volume/fly-reconcile-demo/data
created: machine/fly-reconcile-demo/web
created: machine/fly-reconcile-demo/worker
```

<Aside type="note">
In a dev checkout, run through the workspace CLI —
`npx tsx ../../packages/core/src/cli/main.ts run fly` — rather than a globally
linked `chant`, which double-loads modules here.
</Aside>

## Watch it reconcile

The point is what re-running does. State lives on the platform, not in a file, so
`flyApply` reads it back each time.

**Re-apply is a no-op.** Run it again with no source change — nothing is touched:

```
unchanged: app/fly-reconcile-demo
noop: volume/fly-reconcile-demo/data
noop: machine/fly-reconcile-demo/web
noop: machine/fly-reconcile-demo/worker
```

**A config change updates in place.** Edit `web`'s `config.image` in
`src/infra.ts` and re-run — `flyApply` leases the machine, updates it, and waits
the new instance to `started`. `worker`, unchanged, still no-ops:

```
updated: machine/fly-reconcile-demo/web
noop: machine/fly-reconcile-demo/worker
```

**Removing a machine prunes it.** Delete the `worker` machine from `src/infra.ts`
and re-run. Prune is on, so the machine chant no longer declares is destroyed —
it carries the `managed-by: chant` marker:

```
noop: machine/fly-reconcile-demo/web
pruned: fly-reconcile-demo/worker
```

**Foreign machines survive.** Create a machine directly in mudflaps, without the
ownership marker, then re-run:

```bash
curl -X POST http://localhost:4280/v1/apps/fly-reconcile-demo/machines \
-H 'content-type: application/json' \
-d '{"name":"legacy","region":"iad","config":{"image":"flyio/hellofly:latest"}}'
```

The owned-only prune leaves `legacy` alone — chant deletes only what it owns, so
a resource created outside chant is never touched. Authority stays with the
platform; chant reconciles its own slice of it.

Stop the emulator when done:

```bash
docker rm -f mudflaps
```

## Run it against real Fly

Drop the endpoint override and give it a token — the same op, no code change:

```ts
phase("Apply", [flyApplyStep("dist/fly.json", { prune: true })]), // no endpoint
```

```bash
export FLY_API_TOKEN=...
chant run fly
```

With no `endpoint`, `flyApply` falls through to `FLY_FLAPS_BASE_URL` (or Fly's
`https://api.machines.dev`) and authenticates with the token. The reconcile is
the same against real Fly — the emulator is just an endpoint swap.
51 changes: 51 additions & 0 deletions examples/examples.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import type { PostSynthContext } from "@intentius/chant/lint/post-synth";
import { k8sPlugin } from "@intentius/chant-lexicon-k8s/plugin";
import deployOp from "./getting-started/deploy.op";
import flyDeployOp from "./local-fly/ops/fly.op";
import flyReconcileOp from "./fly-reconcile/ops/fly.op";
import agentTaskOp from "./sprites-agent-task/ops/agent-task.op";
import guardedTaskOp from "./sprites-agent-task/ops/guarded-task.op";
import managedAgentSessionOp from "./sprites-managed-agent-worker/ops/managed-agent-session.op";
Expand Down Expand Up @@ -237,6 +238,56 @@ describe("local-fly deploy Op (#744)", () => {
});
});

// ── Fly reconcile — fly-reconcile (#868) ─────────────────────────────
// A multi-machine stack whose op reconciles against a *running* mudflaps (no
// boot/teardown), so re-runs show create → no-op → update → owned-only prune.
// Build-validated here (the live reconcile is documented in the tutorial); a
// separate block compile-validates the Build → Apply op.

describeExample(
"fly-reconcile",
{
lexicon: "fly",
serializer: [flySerializer],
outputKey: "fly",
examplesDir: import.meta.dirname,
},
{
checks: (output) => {
const plan = JSON.parse(output) as Record<
string,
{ endpoint: string; method: string; body: Record<string, any> }
>;
const reqs = Object.values(plan);
const app = reqs.find((r) => r.endpoint === "/v1/apps");
expect(app!.body.app_name).toBe("fly-reconcile-demo");
// A volume, created before the machine that mounts it.
const volume = reqs.find((r) => /\/v1\/apps\/[^/]+\/volumes$/.test(r.endpoint));
expect(volume!.body.name).toBe("data");
// Two machines, each carrying the managed-by marker the prune reads back.
const machines = reqs.filter((r) => /\/v1\/apps\/[^/]+\/machines$/.test(r.endpoint));
expect(machines.map((m) => m.body.name).sort()).toEqual(["web", "worker"]);
for (const m of machines) expect(m.body.config.metadata["managed-by"]).toBe("chant");
// web mounts the volume by name.
const web = machines.find((m) => m.body.name === "web");
expect(web!.body.config.mounts).toEqual([{ volume: "data", path: "/data" }]);
},
},
);

describe("fly-reconcile op (#868)", () => {
test("compiles to a Build → Apply reconcile with prune on", () => {
const props = (flyReconcileOp as unknown as {
props: { name: string; phases: Array<{ name: string; steps: Array<{ fn: string; args?: Record<string, any> }> }> };
}).props;
expect(props.name).toBe("fly");
expect(props.phases.map((p) => p.name)).toEqual(["Build", "Apply"]);
const apply = props.phases.find((p) => p.name === "Apply")!.steps[0];
expect(apply.fn).toBe("flyApply");
expect(apply.args?.prune).toBe(true);
});
});

// ── Sprites agent task — sprites-agent-task (#762) ───────────────────
// Pure activity-sequence Ops (no build, no serialized plan). The live run is
// documented in the example README (against the in-process fake or real
Expand Down
3 changes: 3 additions & 0 deletions examples/fly-reconcile/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
node_modules/
dist/
package-lock.json
62 changes: 62 additions & 0 deletions examples/fly-reconcile/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# fly-reconcile

One Fly **app + a volume + two machines** (`web` mounts the volume), reconciled
against a **running** mudflaps by `flyApply`. Where [`local-fly`](../local-fly)
boots and tears down the emulator each run, this example points at a persistent
mudflaps so you can run it repeatedly and watch the reconcile: create → no-op →
in-place update → owned-only prune. No `flyctl`, no state file.

| Resource | Serializes to | Applied by |
|----------|---------------|------------|
| App + Volume + 2 Machines | flaps create bodies (JSON) | `flyApply` (prune on) |

The op is `Build → Apply` only — it does not manage the emulator, so state
survives between runs. `flyApply` GET-then-creates each resource, updates changed
machines in place, and (with `prune: true`) destroys chant-owned machines no
longer declared. A machine created directly in mudflaps, without the
`managed-by: chant` marker, is left untouched.

## Run it

Boot mudflaps once, then run the op as many times as you like:

```bash
docker run -d --rm -p 4280:4280 --name mudflaps ghcr.io/intentius/mudflaps:0.4.1

npm install
chant run fly # first run: creates app + volume + web + worker
chant run fly # second run: no-op — nothing changed
```

Then exercise the reconcile:

- **Update in place.** Change `web`'s `config.image` in `src/infra.ts` and
re-run — `flyApply` leases the machine, updates it, and waits `started`.
- **Owned-only prune.** Delete the `worker` machine and re-run — the prune
destroys it, because it carries the `managed-by: chant` marker.
- **Foreign resources survive.** Create a machine directly in mudflaps
(`curl -X POST .../machines`) and re-run — the prune leaves it alone; it has no
ownership marker.

Stop the emulator when done:

```bash
docker rm -f mudflaps
```

## Real Fly

Drop the endpoint override and give it a token — the same op, no code change:

```ts
// ops/fly.op.ts
phase("Apply", [flyApplyStep("dist/fly.json", { prune: true })]), // no endpoint
```

```bash
export FLY_API_TOKEN=...
chant run fly
```

With no `endpoint`, `flyApply` falls through to `FLY_FLAPS_BASE_URL` (or Fly's
`https://api.machines.dev`) and sends `Authorization: Bearer $FLY_API_TOKEN`.
6 changes: 6 additions & 0 deletions examples/fly-reconcile/chant.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import type { ChantConfig } from "@intentius/chant";

// The fly lexicon (loads the flaps applier + mudflaps lifecycle activities:
// flyApply / flapsUp / flapsDown) plus temporal (the base activities: chantBuild,
// httpCheck). `chant run fly` resolves each Op step's `fn` against these.
export default { lexicons: ["fly", "temporal"] } satisfies ChantConfig;
27 changes: 27 additions & 0 deletions examples/fly-reconcile/ops/fly.op.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { Op, phase, build } from "@intentius/chant/op";
import { flyApplyStep } from "@intentius/chant-lexicon-fly";

/**
* Reconcile the app + volume + two machines against a **running** mudflaps —
* `chant run fly`. Unlike the local-fly deploy Op, this one does not boot or tear
* down the emulator: you start mudflaps once and run this repeatedly, so the
* state persists across runs and the reconcile is observable (re-apply no-op,
* in-place update, owned-only prune).
*
* docker run -d --rm -p 4280:4280 --name mudflaps ghcr.io/intentius/mudflaps:0.4.0
*
* `flyApply` GET-then-creates each resource, updates changed machines in place,
* and — with `prune: true` — destroys chant-owned machines no longer declared. A
* machine created directly in mudflaps (no `managed-by: chant` marker) is left
* untouched. To target real Fly, drop the `endpoint` override and set
* `FLY_FLAPS_BASE_URL` / `FLY_API_TOKEN`.
*/
export default Op({
name: "fly",
overview: "Reconcile the plan against a running mudflaps (create / update / prune)",
taskQueue: "fly",
phases: [
phase("Build", [build(".", { script: "build:fly" })]),
phase("Apply", [flyApplyStep("dist/fly.json", { endpoint: "http://localhost:4280", prune: true })]),
],
});
19 changes: 19 additions & 0 deletions examples/fly-reconcile/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"name": "fly-reconcile-example",
"version": "0.0.1",
"private": true,
"type": "module",
"scripts": {
"build:fly": "chant build src --lexicon fly -o dist/fly.json",
"build": "npm run build:fly",
"lint": "chant lint src"
},
"dependencies": {
"@intentius/chant": "*",
"@intentius/chant-lexicon-fly": "*",
"@intentius/chant-lexicon-temporal": "*"
},
"devDependencies": {
"typescript": "*"
}
}
36 changes: 36 additions & 0 deletions examples/fly-reconcile/src/infra.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import { App, Machine, Volume, MachineConfig, MachineGuest, MachineMount, Fly } from "@intentius/chant-lexicon-fly";

// A small multi-resource Fly stack: one app, a persistent volume, and two
// machines — `web` (which mounts the volume) and `worker`. It exists to show the
// reconcile flyApply does — create → no-op re-apply → in-place update →
// owned-only prune. The serializer stamps `managed-by: chant` into each
// machine's `config.metadata`, the ownership marker the prune reads back.
//
// `org_slug` is required by the Machines API. `Fly.OrgSlug` resolves from
// `FLY_ORG` at build time, defaulting to `personal` offline.
const app = new App({ name: "fly-reconcile-demo", org_slug: Fly.OrgSlug });

// A persistent volume. flyApply creates it before any machine that mounts it —
// the dependency order is on the wire (the volume is POSTed first).
const data = new Volume({ name: "data", region: "iad", size_gb: 1 });

const guest = new MachineGuest({ cpu_kind: "shared", cpus: 1, memory_mb: 256 });

const web = new Machine({
name: "web",
region: "iad",
config: new MachineConfig({
image: "flyio/hellofly:latest",
guest,
// The mount references the declared volume by name (FLY011 checks this).
mounts: [new MachineMount({ volume: "data", path: "/data" })],
}),
});

const worker = new Machine({
name: "worker",
region: "iad",
config: new MachineConfig({ image: "flyio/hellofly:latest", guest }),
});

export { app, data, web, worker };
Loading
Loading