Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

414 changes: 414 additions & 0 deletions docs/canvas-architecture-m0.md

Large diffs are not rendered by default.

101 changes: 101 additions & 0 deletions moli-benchmark/fixtures/canvas/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Canvas 2D M0 baseline fixtures

Reproducible benchmark inputs + runner for the Canvas 2D re-architecture
(see `docs/canvas-architecture-m0.md`). These measure the **pre-migration**
cost model so M6 can compare old vs new.

Contents:

- `workloads.py` — JS workload definitions (deterministic, emits self-contained
HTML). Workloads cover the proposal §11.1 matrix: many small path fills and
strokes with a single readback; many small rect, image, and text draws; a
readback-after-every-draw workload (where batching is impossible); a mixed
draw/clear/pixel-write sequence; repeated clean `getImageData`; and
canvas-to-canvas/self-drawing.
- `runner.py` — CDP runner that launches `moli serve`, navigates each workload
as a `data:` URL, and writes results to `results/<timestamp>_baseline.json`.
- `results/` — captured raw baselines (see below for what is captured so far).

## Workload shape

Every workload HTML defines `SIZE` and `N`, builds a canvas and a 2D context,
runs a synchronous draw loop, then sets:

```js
globalThis.__canvasResult = {
workload, size, ops,
recordMs, // cumulative time of the draw-call loop (input/recording cost)
flushMs, // time of the single forced pixel observation
totalMs, // total wall time
probe, acc, // sanity pixel/accumulator values
};
```

## Prerequisites

Building Moli (not this runner): the full workspace build pulls `aws-lc-sys`,
which compiles with bindgen, so a `cargo` build requires libclang.

```sh
sudo apt-get install libclang-dev clang
cargo build --release -p moli
```

Python deps for `moli_benchmark` (which this runner imports): see
`moli-benchmark/pyproject.toml` (`websockets`, `pillow`).

## Running

```sh
python moli-benchmark/fixtures/canvas/runner.py \
--binary ./target/release/moli \
--sizes 256 1024 2048 \
--ops 100 1000
```

Results are written to `results/<UTC ISO timestamp>_baseline.json`, a list of
rows keyed by `workload`, `size`, `ops`, plus `recordMs`/`flushMs`/`totalMs`
(and `probe`/`acc`).

## Captured baseline (this machine)

Recorded during M0 on branch `canvas_2D_moli`, commit `4d6e0373`, debug build.

### Native cost model (`moli-canvas/tests/baseline_cost.rs`, no V8)

Models the current design's per-draw full-plane copy + format-conversion work.
Run with:

```sh
cargo test -p moli-canvas --test baseline_cost -- --nocapture
```

Raw (debug) output:

| canvas | ops | bytes/plane | full copies/draw | bytes copied (total) | copy secs | convert secs |
|---|---|---|---|---|---|---|
| 256² | 100 | 262,144 | 2 | 52,428,800 | 0.0015 | 0.162 |
| 1024² | 1000 | 4,194,304 | 2 | 8,388,608,000 | 0.229 | 26.04 |
| 2048² | 1000 | 16,777,216 | 2 | 33,554,432,000 | 1.204 | 100.99 |

These numbers expose the structural problem the proposal targets: cost scales
with canvas area even for small shapes, because every draw copies the whole
plane and (for path fills) allocates a fresh full-canvas raster to composite.

### Environment note on the JS/CDP baseline

The full `moli-renderer-v8` test suite and the CDP benchmark **could not be
executed in the M0 capture environment** because rebuilding `aws-lc-sys`
requires libclang/bindgen, which is not installed here (the once-built debug
binary predates this session). `workloads.py` + `runner.py` are validated for JS
correctness (see the Node harness notes in the commit) but the wall-clock JS
orders of magnitude were not measured against a live Moli instance in this
session. They must be captured as part of M6 on a machine with libclang and a
built `moli serve`.

### JS correctness baseline (existing regression suite)

`moli-canvas` native tests pass (22/22). The full
`canvas_paths`/`canvas_arguments` JS regressions require the `moli-renderer-v8`
test build (blocked by libclang here):
`cargo nextest run -p moli-renderer-v8 --lib canvas_paths canvas_arguments`.
44 changes: 44 additions & 0 deletions moli-benchmark/fixtures/canvas/results/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Canvas 2D M0 baseline results

Raw captured numbers for the pre-migration Canvas 2D cost model. The CDP
wall-clock rows are captured by `runner.py` and land here as
`<UTC ISO timestamp>_baseline.json`; none have been produced yet because the
capture environment lacks libclang to rebuild `aws-lc-sys` (see
`README.md` in this directory).

## Native cost model (this machine)

Branch `canvas_2D_moli`, commit `4d6e0373`, **debug** build,
`moli-canvas/tests/baseline_cost.rs`, run via:

```sh
cargo test -p moli-canvas --test baseline_cost -- --nocapture
```

### Arithmetic byte-cost evidence (asserted; instant)

The current design performs **two full-plane byte copies per ordinary draw**
(`with_canvas_like_pixels_mut`: copy backing view to a Vec, mutate, write back),
so the cost is linear in canvas area regardless of paint size:

| canvas | ops | bytes/plane | full copies/draw | bytes copied (total) |
|---|---|---|---|---|
| 256² | 100 | 262,144 | 2 | 52,428,800 |
| 1024² | 1000 | 4,194,304 | 2 | 8,388,608,000 |
| 2048² | 1000 | 16,777,216 | 2 | 33,554,432,000 |

Moving 33.5 GB to draw 1,000 small shapes on a 2048² canvas is the structural
problem this project removes.

### Reduced timing matrix (debug; kept fast so the check suite stays quick)

| canvas | ops | full-copy secs | premultiply-convert secs | encode (×10) secs |
|---|---|---|---|---|
| 256² | 100 | 0.0016 | 0.175 | 0.015 |
| 1024² | 100 | 0.0288 | 2.582 | 2.090 |

The full-timing matrix at 1000 ops is intentionally not run in the crate test to
keep `cargo nextest` fast; it can be measured at M6 on a machine with a built
browser via `moli-benchmark/fixtures/canvas/runner.py`.

Machine / toolchain: x86_64-unknown-linux-gnu, rustc 1.96.1 (debug).
122 changes: 122 additions & 0 deletions moli-benchmark/fixtures/canvas/runner.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
"""CDP runner for the Canvas 2D M0 baseline.

Launches a Moli CDP instance (`moli serve`), loads each canvas workload through
`Page.navigate` on a `data:` URL, and records per-workload timings to
`results/baseline.json`.

The workload HTML is generated by `workloads.py` in the same directory
(deterministic, no external network), and the timing loop is synchronous JS so
wall-clock cost maps to the draw/observe phases the architecture targets.

Prerequisite to build Moli (this script does not build): the full workspace
build pulls `aws-lc-sys`, which compiles with bindgen, so a `cargo` build
requires libclang. Build with:

sudo apt-get install libclang-dev clang
cargo build --release -p moli

Run (anywhere, once dependencies are installed):

python moli-benchmark/fixtures/canvas/runner.py \
--binary ./target/release/moli \
--sizes 256 1024 2048 --ops 100 1000

Results are written (with an ISO timestamp) into
`<local>/results/<timestamp>_baseline.json` next to this runner.
"""

from __future__ import annotations

import argparse
import asyncio
import json
import sys
import urllib.parse
from datetime import datetime, timezone
from pathlib import Path

HERE = Path(__file__).resolve().parent
BENCHMARK_ROOT = HERE.resolve().parents[1]

# Runtime sys.path wiring (the runner is a standalone script, not a package):
# - `workloads` is a sibling module in the same directory.
# - `moli_benchmark` lives in the sibling directory that also holds `moli-benchmark/`.
sys.path.insert(0, str(HERE))
sys.path.insert(0, str(BENCHMARK_ROOT))

from workloads import WORKLOADS, html_for_workload # noqa: E402

from moli_benchmark.raw_cdp import RawCdpClient, connect_raw_cdp # noqa: E402
from moli_benchmark.serve import start_moli_serve, stop_moli_serve # noqa: E402

DEFAULT_SIZES = (256, 1024, 2048)
DEFAULT_OPS = (100, 1000)


def _data_url(html: str) -> str:
return "data:text/html;charset=utf-8," + urllib.parse.quote(html)


async def _evaluate_json(client: RawCdpClient, expression: str) -> dict | None:
message_id = await client.send(
"Runtime.evaluate",
{"expression": expression, "returnByValue": True},
)
reply, _events = await client.recv_until_id(message_id)
result = (reply.get("result") or {}).get("result") or {}
value = result.get("value")
return value if isinstance(value, dict) else None


async def _run_workload(client: RawCdpClient, name: str, size: int, ops: int) -> dict:
url = _data_url(html_for_workload(name, size, ops))
message_id = await client.send("Page.navigate", {"url": url})
await client.recv_until_id(message_id)
payload = await _evaluate_json(client, "JSON.stringify(globalThis.__canvasResult)")
row = {"workload": name, "size": size, "ops": ops}
if payload:
row.update(
{k: payload.get(k) for k in ("recordMs", "flushMs", "totalMs", "acc", "probe")}
)
return row


async def _drive(binary: Path, sizes: list[int], ops_list: list[int]) -> list[dict]:
handle = start_moli_serve(binary, 30.0)
rows: list[dict] = []
try:
client = await connect_raw_cdp(handle.endpoint)
try:
# Deterministic ordering: size (outer), ops (middle), workload (inner).
for size in sizes:
for ops in ops_list:
for name in WORKLOADS:
rows.append(await _run_workload(client, name, size, ops))
finally:
await client.close()
finally:
stop_moli_serve(handle)
return rows


def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--binary", required=True, help="path to the moli CDP binary")
parser.add_argument("--sizes", nargs="+", type=int, default=list(DEFAULT_SIZES))
parser.add_argument("--ops", nargs="+", type=int, default=list(DEFAULT_OPS))
args = parser.parse_args()

print(f"[canvas-baseline] binary={args.binary} sizes={args.sizes} ops={args.ops}")
rows: list[dict] = asyncio.new_event_loop().run_until_complete(
_drive(Path(args.binary), args.sizes, args.ops)
)
out_dir = HERE / "results"
out_dir.mkdir(parents=True, exist_ok=True)
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
out_path = out_dir / f"{stamp}_baseline.json"
out_path.write_text(json.dumps(rows, indent=2))
print(f"[canvas-baseline] wrote {out_path}")


if __name__ == "__main__":
main()
Loading
Loading