Skip to content

ci: measure a plugin's rebuild loop on Linux, per backend and incremental - #292

Merged
foxnne merged 3 commits into
mainfrom
ci/plugin-loop
Oct 10, 2026
Merged

ci: measure a plugin's rebuild loop on Linux, per backend and incremental#292
foxnne merged 3 commits into
mainfrom
ci/plugin-loop

Conversation

@foxnne

@foxnne foxnne commented Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Part of #250. The compile is now nearly all of a plugin's rebuild loop (#285). This sets up its measurement on Linux CI. Stacked on #287, whose reload log the bench's --app stage reads. Merge #287 first, then this PR's base moves to main.

What changes

  • .github/workflows/plugin-loop.yml, a measurement, not a gate (not part of ci-ok). It runs on demand, and on PRs touching scripts/plugin-loop/**, the workflow itself or sdk/plugin_sdk.zig.
    • Linux only, plugins/image.
    • Debug and ReleaseFast, each with Zig's default backend, LLVM forced, and self-hosted forced: cold, nothing changed, and a one-line edit.
    • Then three edits under zig build --watch -fincremental, each timed from save to installed.
    • Results go to the job summary as a table, beside the runner's cores, memory and Zig version.
  • A zig_version input: re-measuring on a newer Zig is one manual run (Actions → Plugin rebuild loop → Run workflow → 0.17.0 or master). Zig 0.17 is close to release and expected to carry incremental compilation further. A plugin that no longer builds on a newer Zig fails the run, which is worth knowing early too.
  • FIZZY_PLUGIN_USE_LLVM=1|0 (sdk/plugin_sdk.zig) forces a plugin build's code generator. Unset, Zig chooses. It's an environment variable rather than a -D option because fizzy's own build creates several plugins on one builder, and an option may be declared only once.
  • scripts/plugin-loop/bench.py:
    • --backend, --watch N, and --summary FILE.
    • A combination that can't build becomes a "failed" row instead of ending the run.
    • It defaults to plugins/image: examples: moved to fizzyedit/example-app and example-plugin #282 moves examples/ out of the repo, which would take the old default with it. --app still needs a plugin fizzy doesn't bundle, and the docstring says so.
  • plans/AGENTS_PLAN.md: the compile lever points at the workflow, and notes Zig 0.17.

Why CI rather than a workstation. Forcing the self-hosted backend on an arm64 Mac, beside other sessions' compiles, coincided with a kernel panic: watchdog timeout, swap exhausted. An experimental backend on a large compile can use far more memory than LLVM. Runners are disposable.

SDK impact

  • None
  • Core-only or additive: sdk/plugin_sdk.zig (build-side) reads FIZZY_PLUGIN_USE_LLVM. Unset, nothing changes. test-sdk-version passes with the recorded fingerprint unchanged.
  • Fingerprint moved
  • SDK release

Verified

  • macOS: all five gates pass (test-integration 33/33 steps, 347/347). A light local run of the bench (Debug, default and llvm, no watch, plugins/image) printed and wrote its summary table. I didn't run the self-hosted backend or the watch locally, on purpose.
    • That run showed default and forced llvm differing on arm64 macOS (one-line edit 2.7 s vs 5.0 s). Machine load also changed during it, so it means nothing until the Linux numbers are in.
  • Linux: this PR's own run of the new workflow is the first measurement.
  • Windows / Web: n/a.

Follow-ups

  • Read the first Linux run, then decide the compile lever: incremental under --watch (the agent plugin's loop could keep one running), a smaller compile unit for plugins, or waiting for Zig 0.17.

🤖 Generated with Claude Code

@foxnne foxnne mentioned this pull request Oct 9, 2026
Merged
2 tasks done
foxnne and others added 2 commits October 9, 2026 13:46
@foxnne @claude
app: a rebuilt plugin loads without freezing a frame
aba2bc6 
A plugin rebuilt under a running fizzy used to be reloaded on the UI
thread, and its `dlopen` — on macOS 180-220 ms for a file the system has
not seen, which a rebuild always is — froze the person's frame for that
long. The `dlopen` was the whole cost: the plugin's `register` takes
~30 us.

`PluginLoader` splits into `open` (copy, `dlopen`, ABI and SDK checks;
nothing of the host's, so any thread) and `register` (on the UI thread);
`loadAndRegister` is the two in a row, so startup is unchanged. The
watcher's reload (`PluginReloads`) opens the new build on a worker while
the old one keeps running, and a later frame swaps them: ~8 ms on the UI
thread (unload ~4 ms, register ~4 ms), the ~175 ms open off it.

Part of #250.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@foxnne @claude
ci: measure a plugin's rebuild loop on Linux, per backend and increme…
c2c516d 
…ntal

`.github/workflows/plugin-loop.yml` (on demand, and on PRs touching the
loop's files; not part of ci-ok) runs scripts/plugin-loop/bench.py on
Linux for `plugins/image`: cold, nothing changed and one-line-edit builds
in Debug and ReleaseFast with Zig's default backend, LLVM forced and
self-hosted forced, and edits under `zig build --watch -fincremental`,
written to the job summary. A `zig_version` input re-measures on a newer
Zig — 0.17 is expected to carry incremental compilation further.

`FIZZY_PLUGIN_USE_LLVM=1|0` (read by sdk/plugin_sdk.zig) forces a plugin
build's code generator. The bench gains `--backend`, `--watch N` and
`--summary`, reports a combination that cannot build as a row, and
defaults to `plugins/image` (examples/ is moving out of the repo, #282).

Compiler experiments run here, not on a shared workstation: forcing the
self-hosted backend locally, beside other sessions' compiles, took a
machine's memory down.

Part of #250.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@foxnne
foxnne force-pushed the app/reload-off-thread branch from ccded80 to aba2bc6 Compare October 9, 2026 18:47
@foxnne

foxnne commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator Author

Heads-up: examples/hello-plugin, the default plugin of scripts/plugin-loop/bench.py (--plugin default and the comment in copy_plugin), is gone from main. #282 moved it to fizzyedit/example-plugin, and the rest of examples/ is leaving with replay-app. A built-in such as plugins/image has the same ../../sdk path, so copy_plugin's rewrite works on it unchanged. Or the bench could clone example-plugin and zig fetch --save=fizzy the packed SDK, as scripts/check-examples.sh does. Left for this PR rather than changed under it.

🤖 Generated with Claude Code

@foxnne foxnne mentioned this pull request Oct 9, 2026
Merged
2 of 5 tasks
Base automatically changed from app/reload-off-thread to main October 9, 2026 19:47
foxnne added a commit that referenced this pull request Oct 9, 2026
@foxnne @claude
docs: plans move to the private fizzyedit/plans (#296)
8a0fe73 
## What changes

Fizzy's plans move to a private repo,
[`fizzyedit/plans`](https://github.com/fizzyedit/plans), open to the
`fizzyedit` organization. Where fizzy is going can then be worked out
without every edit being public.

- **Moved (15):** every `docs/*_PLAN.md`, both `REVIEW_*` docs,
`TRANSITIONS_HANDOFF.md`, `LIB_CHECKPOINT.md`. The plans repo has them
as of `main` `a2b4e0a7`, plus the `AGENTS_PLAN.md` edits from #287,
#290, #291 and #292, so those PRs drop their doc changes.
- **Staying public:** `PLUGINS.md`, `AUTOMATION.md`, `DEPENDENCIES.md`,
the live-resize measurements, release notes, `demos/`.
- **Test data:** the markdown layout tests embedded
`docs/PLUGIN_MANIFEST_PLAN.md` as their table-heavy sample (and the
markdown bench as prose). It moves to
`tests/data/markdown_sample_tables.md` as a frozen copy: that text was
already public in this repo's history, the plan is essentially done, and
later edits to it stay private. I tried a synthetic sample of the same
shape first, and it exposed a real preview bug: a large table of
wrapping rows gets a different height before and after it is laid out
(`block heights do not depend on where the reader is` and `rapid
scrolling` fail). That's filed separately rather than fixed here.
- **References:** in code comments and docs, a plan path reads
`plans/X.md` (48 files). Markdown links point into the plans repo, which
works for members.
- **`CLAUDE.md`** says what `plans/X.md` means, where to clone it, and
that a session without access works from the code and public docs.
**`CONTRIBUTING.md`** says plans and their issues live there, and that a
PR here carries what a reviewer without access needs.

**What this doesn't do:** fizzy's history still holds every plan as it
was. Moving them hides future edits only; rewriting history would break
every clone.

## SDK impact

- [x] None (comments and docs only)

## Verified

- [x] macOS: all five gates pass after the sweep. `zig build
test-integration`: 33/33 steps, 347/347. No reference to a moved file is
left.

## Follow-ups

- #282 (another session) edits `docs/AGENTS_PLAN.md` and
`docs/WORKSPACES_PLAN.md`. Its plan edits belong in the plans repo once
this lands.
- My open #287, #290, #291 and #292 drop their `AGENTS_PLAN.md` edits,
which are already in the plans repo.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@foxnne
foxnne marked this pull request as ready for review October 10, 2026 01:08
@foxnne
foxnne enabled auto-merge (squash) October 10, 2026 01:08
@foxnne @claude
Merge main into ci/plugin-loop
51de9f5 
scripts/plugin-loop/bench.py: this branch already carries #287's reload-log changes; it stands, with
#296's link to plans/AGENTS_PLAN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@foxnne
foxnne merged commit d4d9071 into main Oct 10, 2026
9 of 11 checks passed
@foxnne
foxnne deleted the ci/plugin-loop branch October 10, 2026 01:27
@foxnne foxnne mentioned this pull request Oct 10, 2026
Merged
2 of 8 tasks
foxnne added a commit that referenced this pull request Oct 10, 2026
@foxnne
sdk: release 0.2.20 (#322)
d440491 
## What changes

`sdk_version` goes from 0.2.19 to 0.2.20. Merging tags `sdk-v0.2.20`,
publishes its tarball,
and asks the store plugins to repin.

**Why now.** The plugin boundary's fingerprint has not moved since
`sdk-v0.2.19`
(`0x7b455e7a…` both), so plugins built against 0.2.19 still load. But
`core/` and `sdk/replay`
changes merged since then reach plugins only through a release, and the
agent plugin
(`fizzyedit/agent`, private until it can pin a released SDK) needs one
of them: #295's
profiler on request.

## SDK impact

- [ ] None
- [ ] Core-only or additive: reaches plugins at the next SDK release
- [ ] Fingerprint moved: recorded in `sdk/src/version.zig`,
`sdk_version` untouched, PR labelled `sdk`
- [x] SDK release: bumps `sdk_version`, lists the `sdk` PRs since the
last `sdk-v*` tag

**The `sdk` PRs since `sdk-v0.2.19`:** none; the fingerprint is
unchanged.

**What reaches plugins (`core/`, `sdk/`):**
- #295: the profiler records on request (`core.profile.want`) and
reports as ZON
(`core.profile.report`). `Profiler` grew a field (`abi` 3 → 4), so a
plugin records its own
sections into the host's profiler again only once rebuilt against this
release.
- #310: `replay.Input.synthetic`, which tells a tape's key from the
OS's.
- #293: the replay `Player` plays and seeks every bundled demo exactly.
- #313: `core.dialogs` changes for a run of the real app that plays a
tape to a verdict.
- #312: `core/gfx`'s programs and the liquid-glass shader, now that the
native backend is a
  package of its own.
- #292: `sdk/plugin_sdk.zig` honours `FIZZY_PLUGIN_USE_LLVM`, for
measuring a plugin's rebuild
  loop per code generator.

## Verified

- [x] macOS:
  - `zig build` and `test-sdk-version` on this head.
  - `scripts/pack-sdk.sh` packs `fizzy-sdk-v0.2.20.tar.gz`.
- Every store plugin (fresh clones of pixi, zig, drive, atlas and
ghostty `main`) builds
against this tree's `sdk/` with `zig build --fork=<sdk>` (31/31 and
28/28 steps), installing
into a throwaway profile. The fork matched, and the built plugin reports
`pinned fizzy SDK: 0.2.20`. With the fingerprint unchanged, the repins
should open as plain
    version bumps.
- The agent plugin (path dependency on `main`'s `sdk/`) builds, 21/21
steps.
- [ ] Windows:
- [ ] Linux: CI.
- [ ] Web:

## Follow-ups

- Each repin PR merged and its plugin tagged (the plugin repos' tags
stay the maintainer's).
- `fizzyedit/agent` repins from its local path to the `sdk-v0.2.20`
tarball.
- fizzyedit/example-plugin repins once `PLUGIN_DISPATCH_TOKEN` can reach
it (#281, still open).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
foxnne added a commit that referenced this pull request Oct 10, 2026
@foxnne @claude
examples: replay-app moved to fizzyedit/example-app (#301)
f549b4c 
Part of #281. **Merge after fizzyedit/example-app#2**: this PR's CI
clones example-app's `main` and builds its replay app, which lands there
in #2.

## What changes

`examples/replay-app` moves to fizzyedit/example-app, as `zig build
run-replay`, now that SDK 0.2.19 ships `replay`. Fizzy's `examples/`
folder is gone.

- `scripts/check-examples.sh` also swaps example-app's `fizzy_sdk` pin
for this tree's SDK, packed as a release would be, and builds the replay
app. So replay is still built against every pull request.
- The CI step that built `examples/replay-app` by path is gone.
- `sdk/replay_module.zig`, `docs/AUTOMATION_PLAN.md` and
`docs/AGENTS_PLAN.md` point at example-app's replay app.

**Not changed here:** `scripts/plugin-loop/bench.py` still defaults to
`examples/hello-plugin`, which #282 removed. #292 is rewriting that
script, so it is noted there rather than changed under it.

## SDK impact

- [x] None (a doc comment in `sdk/replay_module.zig`)

## Verified

- [x] macOS:
- `scripts/check-examples.sh` against this tree and example-app with #2:
example-plugin 28/28 steps; minimal, studio, endless 30/30; studio as
data 31/31; the replay app 15/15.
- `zig build` (43/43), `test-integration` (33/33 steps, 349 tests),
`check-web`, `test-sdk-version`.
- [ ] Windows:
- [ ] Linux: CI, once example-app#2 has merged.
- [ ] Web:

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@foxnne