Skip to content

Add math in labels: zero-dep lookup tier + opt-in Typst tier - #9

Open
gabo515 wants to merge 202 commits into
devfrom
feat/typst-math
Open

Add math in labels: zero-dep lookup tier + opt-in Typst tier#9
gabo515 wants to merge 202 commits into
devfrom
feat/typst-math

Conversation

@gabo515

@gabo515 gabo515 commented Jun 10, 2026

Copy link
Copy Markdown
Owner

Any label may embed $...$ math (LaTeX-ish syntax). Rendering has two tiers that share $-detection and the command→symbol table (src/render/math.rs):

Lookup tier — always compiled, zero dependencies:

  • to_unicode lowers math to inline Unicode: Greek, operators, super/sub (all-or-nothing, falling back to a clean ^(2q) when a glyph is missing), \frac{a}{b}a/b, \sqrt{x}√(…). Never emits a stray \ or $.
  • Baseline for every backend; the only tier the terminal can use. SVG, PNG, PDF, and terminal all route plain $...$ labels through it.

Typst tier — feature math, opt-in:

  • Links the Typst compiler as a library (minimal World: bundled fonts, no package/network access) and typesets the whole label (text + math) into one SVG fragment (SVG/PDF) or pixmap (PNG), embedded into kuva output. Real 2-D math: stacked fractions, radicals, large operators with limits.
  • Whole-label rendering means no fontdue text measurement and no per-segment layout — svg_math just places one fragment; raster blits/rotate-blits one pixmap. Math color follows the label color (injected into the Typst source).
  • On compile failure (e.g. Typst's mcm c) the label degrades to the lookup tier with a one-time warning.
  • Bundles only New Computer Modern Math (~1 MB, gzipped) + reuses the bundled DejaVu Sans for text, instead of the ~15 MB typst-assets. Excluded from full so cargo ci-test and example builds stay lean; enable explicitly, e.g. --features math,png.

Also adds the typst markup backend (TypstBackend): emits a CETZ-based .typ document for external typst compile, sharing the $...$ → Typst-math translation with the math tier.

Tests: exhaustive exact-match units on the lookup tier (render::math); structural tests on the typst tier (SVG embed, PNG composite incl. rotated y-labels, PDF via usvg, color injection). Docs: Reference → Math in Labels.

Description

Type of change

  • New plot type
  • New feature / API addition
  • Bug fix
  • Documentation / assets only
  • Refactor / housekeeping

Checklist

Library (new plot type)

  • src/plot/<name>.rs — struct + builder methods
  • src/plot/mod.rspub mod + re-export
  • src/render/plots.rsPlot enum variant + bounds() / colorbar_info() / set_color()
  • src/render/render.rsrender_<name>(), added to render_multiple() match, skip_axes if pixel-space
  • src/render/layout.rsauto_from_plots() extended if categories needed

Tests

  • New test file in tests/ with ≥ basic render + SVG content + legend tests
  • cargo test --features cli,full — all existing tests still pass

CLI (if applicable)

  • src/bin/kuva/<name>.rs — Args struct (with /// doc comment) + run()
  • src/bin/kuva/main.rs — module, Commands variant, match arm
  • scripts/smoke_tests.sh — at least one invocation
  • tests/cli_basic.rs — SVG output test + content verification test
  • docs/src/cli/index.md — subcommand entry
  • man/kuva.1 — regenerated (./target/debug/kuva man > man/kuva.1)

Documentation

  • examples/<name>.rs — Rust example for doc asset generation
  • scripts/gen_docs.sh — invocations added; bash scripts/gen_docs.sh runs clean
  • docs/src/plots/<name>.md — documentation page with embedded SVGs
  • docs/src/SUMMARY.md — link added
  • docs/src/gallery.md — gallery card added
  • README.md — plot types table updated

Visual inspection

  • Opened test_outputs/ — new plot SVGs look correct
  • Scanned neighbouring plots in test_outputs/ for layout regressions
  • bash scripts/smoke_tests.sh — all existing smoke test outputs still look correct
  • No text clipped, no legend overlap, no spurious axes on pixel-space plots

Housekeeping

  • CHANGELOG.md — entry added under ## [Unreleased]
  • README.md — item marked done in TODO section if applicable

@gabo515

gabo515 commented Jun 10, 2026

Copy link
Copy Markdown
Owner Author

@claude review this jawn plz

@claude

claude Bot commented Jun 10, 2026

Copy link
Copy Markdown

Claude finished @gabo515's task in 3m 33s —— View job


PR Review: Math in Labels (lookup + Typst tiers)

Read the full diff against origin/dev. Overall this is a high-quality, well-architected PR — the two-tier design is clean, the feature gating is careful, and the lookup tier is exhaustively unit-tested. Notes below; most are minor. Couldn't run cargo test/cargo build (sandbox blocks cargo here), so the structural/runtime findings are from reading.

Strengths

  • Tier split is clean. to_unicode (zero-dep, always-on) and the #[cfg(feature="math")] Typst tier share $-detection (contains_math/split_segments) and the symbol table (command_to_unicode), diverging only on structure. Graceful fall-through to the lookup tier on compile failure in all three backends (svg.rs:234, raster.rs:2023, terminal always-lookup).
  • Feature hygiene. math is deliberately excluded from full (Cargo.toml:42-43) with a clear rationale comment; fonts mod and unpremultiply are correctly gated. today() returns a fixed date (typst.rs:763) → deterministic output. Fonts/Library/FontBook cached via OnceLock.
  • Lookup tier is genuinely robust — all-or-nothing super/subscript fallback, the "never emit a stray \ or $" guarantee, and ~25 exact-match unit tests including the quadratic-formula full chain.

Issues

1. [Medium] Duplicate element IDs across embedded Typst SVG fragments.
When a plot has more than one $...$ label, each fragment from extract_inner (render/math.rs:675) carries its own <defs id="glyph"> plus content-hashed <symbol id="g…">. I confirmed this in the committed asset — docs/src/assets/math/mixed.svg has two id="glyph" and shared g<hash> symbol ids:

2 id="glyph"
1 id="gF742789BD91EBE9EAEBCF0F8597DBBA1"
...

Because the symbol ids are content-hashed, identical glyphs collide to identical content, so browsers/usvg render correctly (first def wins, and it's the same path). But it's technically-invalid SVG (duplicate id), which strict validators/sanitizers and some downstream consumers may reject or mangle. Recommend namespacing per fragment — prefix each fragment's ids with a per-label counter and rewrite the matching href="#…" in embed_label.

2. [Medium] Two divergent Typst escapers; the markup backend's is incomplete.

  • render::math::escape_markup (render/math.rs:661) escapes # $ * _ \ < > @ \ " [ ]` ✅
  • backend/typst.rs::write_typst_escaped (typst.rs:480) escapes only \ # [ ] $ @missing _, *, `, <, >.

So a label like a_b, *x*, or rate < 5 rendered through the typst markup backend would be mis-typeset as emphasis/strong/label syntax, while the same label through the math tier escapes correctly. Recommend consolidating onto one shared escaper. Fix this →

3. [Low / perf] No memoization of Typst compiles. render_label_svg/render_label_pixmap run a full typst::compile for every math label (render/math.rs:561,573). Fonts/Library/Book are cached, but a plot with many math tick labels recompiles per-label each render. Consider a process cache keyed by (label, size_bits, color).

4. [Low] Non-rotated raster blit lacks the inward-nudge clamp. blit_pixmap_rotated (raster.rs:979) and SVG embed_label (svg_math.rs:35) both nudge an over-margin label back inside the clip, but the non-rotated blit_pixmap path (raster.rs:2043-2050) just rounds and blits — a wide non-rotated math label near the left/top edge can clip at the canvas boundary. Inconsistent across the three paths.

5. [Low] Color injection drops named CSS colors. typst_fill (render/math.rs:650) handles only Color::Rgb and Color::Css starting with #; anything else → None → black. A label colored with a named color (e.g. "red") renders black in the math tier but correctly-colored via the lookup/plain path. Map the common CSS names (you already have a palette in typst.rs:517).

Nits

  • warn_once grows unbounded & isn't truly "once". render/math.rs:684 keeps a HashSet<String> of every distinct failing label for the process lifetime; the doc/comment says "one-time warning" but it's once per distinct label. Minor.
  • Doc drift: typst.rs module example shows @preview/cetz:0.3.1 (typst.rs:26) but Default pins "0.5.2" (typst.rs:73).
  • extract_inner fragility: uses the first > to find the end of <svg …> (render/math.rs:676). Safe for typst-svg output, but a one-line comment noting the "no > inside attribute values" assumption would help.
  • Edge case in to_unicode: a bare braceless command as a script operand, e.g. $x^\alpha$, has read_script_group grab only \, leaving alpha as literal text → xalpha. $x^{\alpha}$ works. Unusual input; worth a test if you care.

Tests / docs

Coverage is strong: exhaustive lookup-tier exact-match units, structural Typst-tier tests (SVG embed, PNG composite incl. rotated y-labels, PDF via usvg, color injection), plus the Reference → Math doc page. Nothing missing for the scope.

None of these are blockers. #1 and #2 are the two I'd address before merge (invalid-SVG output and the markup-backend escaping gap); the rest are polish.
· branch feat/typst-math

A vector-field plot: each arrow has a tail at (x, y) and a vector (u, v).
- `from_function()` samples a closure on a regular grid
- auto-scaled arrow length (longest ≈ one grid cell, span/√n heuristic) or
  explicit `with_scale`; proportional heads so every arrow looks like an arrow
- pivot modes (Tail/Middle/Tip); optional magnitude colormap + colorbar
- `tight_bounds` for dense fields; `with_clip_to_plot_area()`; combo helper
  `with_magnitude_colormap(cmap, label)`
- CLI: `kuva quiver`

Shared refactors: `render_utils::arrow_head_path` (used by Quiver, Network,
TextAnnotation) and `colorbar_linear` in `render/plots.rs`.
@gabo515
gabo515 force-pushed the feat/typst-math branch 2 times, most recently from 978fd99 to edc1845 Compare June 10, 2026 22:47
Psy-Fer and others added 9 commits June 11, 2026 12:38
…-Fer#87)

> **Re-submission of Psy-Fer#70.** Apologies for the churn — Psy-Fer#70 was
accidentally
> closed when I synced my fork's `dev` branch to upstream. That PR used
`dev`
> itself as the head branch, so the sync dropped the commits and GitHub
> auto-closed it (it can't be reopened — "no new commits on the
branch"). This
> re-submits the **identical** work on a dedicated `feature/quiverplot`
branch,
> rebased onto current `dev`, so a routine fork-sync can't take it down
again.
> Sorry for any confusion on your end.

## Description

Adds **`QuiverPlot`** — a 2-D vector-field renderer. Every arrow has a
tail at `(x, y)` and a vector `(u, v)` in data coordinates; the renderer
maps these through the axis transform, shrinks the shaft to butt against
a triangular head, and wires the whole thing into the standard kuva
layout / legend / colorbar pipeline.

Highlights:

- **Zero-config rendering.** `QuiverPlot::from_function((xlo, xhi, nx),
(ylo, yhi, ny), |x, y| (u, v))` samples a closure on a regular grid; the
longest arrow is auto-scaled to roughly one grid cell (`0.9 × span /
√n`), so a naïve call produces a sensible plot regardless of the units
of `(u, v)`.
- **Arrow heads always look like arrows.** Head length is proportional
to shaft length (`0.28 × shaft`, clamped to `[4, 14]` px), so tiny
arrows still have a visible head and long arrows don't grow gigantic
ones. Explicit pixel overrides via `with_head(len, half_w)` /
`with_head_length` / `with_head_width`.
- **Three pivot modes.** `QuiverPivot::Tail` (default), `Middle`
(centered on `(x, y)`), `Tip` (arrow points *into* `(x, y)`).
- **Magnitude colormap** with automatic colorbar via
`with_magnitude_colormap(cmap, label)`. Per-arrow color priority:
per-arrow override > colormap > plot-level color.
- **Bounds + clipping.** Default bounds include arrow tips. Opt-in
`with_tight_bounds()` uses tails-only bounds; `with_clip_to_plot_area()`
/ `with_no_clip()` control clipping independently.
- **Interactive tooltips.** Each arrow wraps in `<g class="tt" data-x
data-y data-u data-v data-mag>` with a native `<title>` (x, y, u, v,
|v|, θ) for `--interactive` mode.

### Refactors (touch existing plots)

- **`render_utils::arrow_head_path`** — shared arrow-head triangle
helper. Replaces three inline copies: `NetworkPlot` directed edges,
`TextAnnotation` arrows, and (new) `QuiverPlot`.
- **`colorbar_linear(cmap, min, max, label)`** in `render/plots.rs` —
consolidates six near-identical `Arc<Fn>` colorbar closures across
`Heatmap`, `DotPlot`, `DicePlot`, `Contour`, `Clustermap`, and `Quiver`.
The 3-D `colorbar_from_z` path routes through it too.

## Type of change

- [x] New plot type
- [x] Refactor / housekeeping (shared helpers above)

---

## Checklist

### Library (new plot type)
- [x] `src/plot/quiver.rs` — `QuiverPlot` + `QuiverArrow` +
`QuiverPivot`; full builder API; `from_function`; internal-accessor unit
tests
- [x] `src/plot/mod.rs` — `pub mod quiver` + re-exports
- [x] `src/render/plots.rs` — `Plot::Quiver` + `bounds()` /
`colorbar_info()` (via `colorbar_linear`) / `set_color()` /
`estimated_primitives()` / `From<QuiverPlot>`
- [x] `src/render/render.rs` — `add_quiver` + `render_quiver`, wired
into `render_multiple` + palette auto-cycle
- [x] `src/render/layout.rs` — `has_colorbar` / `has_legend` checks
- [x] `src/prelude.rs` — re-exports

### Tests
- [x] `tests/quiver_basic.rs` — 18 integration tests (render + shape
counts, `from_function` sampling, colormap→colorbar, tight-bounds
clip-path, proportional heads, legend, empty-plot, per-arrow color
precedence). Encoding-resilient color assertions.
- [x] `src/plot/quiver.rs` unit tests — pivot modes, auto-scale,
explicit scale, empty/zero fallbacks
- [x] `tests/cli_basic.rs` — CLI SVG output + colorbar canvas widening +
`--arrow-scale`/`--auto-scale` mutual-exclusion error
- [x] `cargo test --features cli,full` — **1467 tests pass, 0 fail**
(rebased onto current `dev`)

### CLI
- [x] `src/bin/kuva/quiver.rs` — `QuiverArgs` + `CliPivot` `ValueEnum` +
`run()`
- [x] `src/bin/kuva/main.rs` — module + `Commands::Quiver` + match arm
- [x] `scripts/smoke_tests.sh` — 7 quiver invocations; **183/183 pass**
- [x] `scripts/terminal_plots.sh` — quiver entry
- [x] `docs/src/cli/index.md` — subcommand entry
- [x] `man/kuva.1` — regenerated

### Documentation
- [x] `examples/quiver.rs` →
`docs/src/assets/quiver/{basic,colormap,source}.svg` (regenerated
against current `dev`)
- [x] `examples/data/quiver.tsv` — sample field data
- [x] `scripts/gen_docs.sh` — `quiver` added
- [x] `docs/src/plots/quiver.md` + `docs/src/cli/quiver.md` — full pages
with flag tables + embedded SVG
- [x] `docs/src/SUMMARY.md` + `docs/src/gallery.md` — links + gallery
card

### Housekeeping
- [x] `CHANGELOG.md` — `[Unreleased]` Added (QuiverPlot) + Changed
(refactors)

---

*Rebased onto current `dev`; the only merge conflict was `CHANGELOG.md`.
Builds + clippy clean across `cli,full`. Doc-asset SVGs regenerated
against current `dev`.*
Psy-Fer and others added 29 commits August 10, 2026 15:05
.with_stacked, and .with_weights methods. Fix broken legend
Rebased onto the krilla-era dev: the zero-dep lookup tier already merged
via Psy-Fer#89 and evolved there, so this commit now adds only the typst work.

- feature `math`: links the Typst compiler as a library and typesets any
  label containing $...$ into an SVG fragment (SVG backend) or pixmap
  (PNG), embedded at the label position with color injection and
  per-label ID namespacing. Compile failure falls back to the lookup
  tier with a one-time warning per distinct label.
- PDF: math fragments flow through the krilla pipeline unchanged
  (SvgBackend -> usvg -> krilla-svg draw_svg) — the xlink:href->href
  rewrite is exactly what usvg needs; no pdf.rs changes required.
- feature `typst` (zero deps, now in `full`): TypstBackend emits a
  CETZ-based .typ document for external `typst compile`; $...$ regions
  pass through as native Typst math.
- bundles NewCM Math (~0.75 MB gzip) instead of the 15 MB typst-assets;
  fonts module gains newcm_math() behind the `math` gate.
Per James's proposal on Psy-Fer#88: krilla (the pdf backend since dev@e855bf5)
is Typst's own PDF stack, so `pdf` was already the crate's one
deliberately heavy, higher-MSRV feature. The typst math tier now rides
it instead of being a separate `math` feature:

- Cargo.toml: `pdf` pulls typst/typst-svg/typst-render/typst-library;
  the `math` feature is gone. `full` (and cargo ci-test) now compile
  the typst tree.
- MSRV: no floor change — typst 0.14 needs 1.89, under krilla's 1.92
  documented in [package.metadata.msrv] pdf_feature. The pdf-msrv CI
  job now covers typst implicitly; comments updated.
- cfg gates and prose flip feature="math" -> feature="pdf" across
  backends, fonts, render::math, and the math test suites (which now
  run under cargo ci-test instead of needing a separate invocation).
- smoke_tests.sh: --math-bin machinery removed — the standard cli,full
  binary now exercises the typst tier directly.
- PDF embedding needs no code: math fragments flow through
  SvgBackend -> usvg -> krilla-svg draw_svg unchanged (verified by
  tests/math_pdf.rs).
Closes the two gaps James flagged in the June review of Psy-Fer#88:

- Primitive::Path (arrowheads in quiver/network, chord ribbons, sankey
  flows, venn outlines) now emits CETZ merge-path calls. Kuva's path
  vocabulary is closed (absolute M/L/C/Q/A/Z): quadratics are promoted
  to cubics, arcs converted endpoint->center per SVG F.6.5 and split
  into <=90-degree cubic approximations. Opacity maps to
  transparentize(), dasharrays to Typst dash tuples.
- ClipStart/ClipEnd now clip: the element stream is split into chunks
  at clip boundaries; each chunk renders as a #place'd canvas, and
  clipped chunks sit inside box(clip: true) at the clip rect with the
  inner canvas shifted back so coordinates stay page-absolute. Scenes
  without clips keep the old single-canvas output shape.
- Every canvas now opens with an invisible full-page rect, anchoring
  CETZ's auto-fitted bounding box to the page so absolute positions
  hold even when no drawing touches the page corners.

All emitted documents (scatter, quiver, clip, math) verified to
compile with typst 0.14 CLI and visually inspected as PNG.
Closes the TextPlot gap from the June review of Psy-Fer#88 ("can't do plot
body text"). Design: inline fragment splicing — kuva keeps its own
markdown parsing and word-wrapping; each $...$ in a body compiles to a
typst fragment that flows through the line-breaker as one unbreakable
word. Surrounding text stays real, selectable text in SVG/PDF output.

- TextSpan gains a `math` flag; parse_inline_markup splits spans at
  $...$ boundaries under `pdf` (without it, lookup-tier lowering is
  unchanged).
- wrap_rich_spans sizes a math word by its typeset width in
  mean-char units, tracks flush adjacency so punctuation after math
  ("$x^2$.") stays tight, and never breaks inside a fragment.
- Lines with fragments taller than the text line grow their leading by
  the ascent/descent overshoot, so stacked fractions never collide
  with neighbouring lines.
- SVG backend lays math-bearing lines out manually (bundled-font
  metrics for text runs, fragment width for math) and embeds fragments
  at the shared baseline; raster blits pixmaps in its pen loop;
  terminal always lowers math spans to inline Unicode.
- render_label_svg is now memoized per process (each fragment is
  needed twice: wrapping metrics + draw; axis labels repeat across
  re-renders).
- Fragments advance like words: the 0.3em safety margin baked into the
  fragment page (FRAGMENT_MARGIN_EM) is subtracted from the inline
  advance and the embed starts a margin early.

Verified visually in PNG (raster), SVG (resvg), and PDF (krilla):
wrapping, bold+math mixing, tall-fragment leading, punctuation
adjacency all correct.
- docs/reference/math.md: restructured around the two tiers — lookup
  tables stay as the zero-dep reference; new Typst-tier section covers
  the pdf feature, compile-failure fallback, the not-LaTeX identifier
  rule, and TextPlot body splicing (with a new committed asset).
- examples/math.rs: regenerated all assets with --features full, so
  the committed SVGs now show typeset math (the docs say so
  explicitly). Fixed two typst-hostile bodies that would have silently
  fallen back to the lookup tier: E = mc^2 -> m c^2, 4ac -> 4 a c.
  Added a TextPlot body-splice example.
- CHANGELOG: body-splicing entry; TypstBackend entry now records full
  primitive coverage (Path via merge-path, clip via box(clip: true)).
page.frame.baseline() returns size.y (the frame height) when unset,
which is always the case for page frames. Using it directly placed the
fragment's bottom edge at the surrounding text baseline, so every
formula floated visibly above the line it was spliced into.

The real baseline lives on the inner paragraph-line frame nested inside
the page (has_baseline() is true there). Walk the frame tree, take the
topmost line-baseline position, add up y offsets on the way.

Assets regenerated: math now sits on the text baseline in wrapped
TextPlot bodies and in labels; before/after diff is visible on every
committed math SVG.
The previous scene put both circles fully inside the clip region, so
test_outputs/typst_clip.typ rendered as two dots with a bottom label —
technically a correct primitive-emission check, visually useless as a
clip demo. Add a large blue circle centered on the clip's left edge so
half is clipped away; keep the small red circle fully inside as a
sanity control. Test still asserts on the primitive emission.
Rustfmt-clean these files so PR review isn't cluttered by post-merge
fmt drift (James's project-wide rustfmt pass on 2026-05-04 established
the current style; new work should land fmt-clean).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants