diff --git a/ATTRIBUTIONS.md b/ATTRIBUTIONS.md index dd0744f..9ad81bb 100644 --- a/ATTRIBUTIONS.md +++ b/ATTRIBUTIONS.md @@ -133,3 +133,11 @@ Vendored WordPerfect-family test files. Per-file provenance (upstream path, revi - **License:** CDLA-Permissive-1.0 - **Used here:** 2 financial-report pages (`images/doclaynet_page_01..02.jpg`) from the `test` split (`NASDAQ_ATRI_2003.pdf` p24, `NYSE_MGM_2004.pdf` p49). - **Modifications:** authoritative PDF text cells linearized in reading order (top-to-bottom, then left-to-right). No OCR and no model used; text is DocLayNet's verbatim PDF-layer cell text. + +## Diagram fixtures (this repository) + +- **Source:** authored here, not derived from any third-party dataset. The first four Graphviz fixtures were contributed by Parman Mohammadalizadeh (@MannXo) in xberg-io/test_documents#1. +- **License:** same as this repository. +- **Used here:** `diagrams/svg/*.svg` plus their `diagrams/src/*.{dot,mmd,puml,fodg}` sources, `diagrams/manifest.json`, and `ground_truth/dot/*.dot`. +- **Rendering tools:** the source graphs are ours; the SVGs are what each renderer emitted from them. Graphviz 15.1.1 (`dot`/`neato`) is EPL-1.0, Mermaid CLI 11.16.0 is MIT, PlantUML 1.2026.0 is GPL, LibreOffice 26.2.5.2 is MPL-2.0. None of these licenses reaches the output: rendering a document is not a derivative work of the renderer, and PlantUML says so explicitly in its FAQ. No tool binary, jar, font or stylesheet is redistributed here. +- **Modifications:** every fixture whose producer states its own graph also ships as a `*_geometry.svg` variant with that metadata removed by `scripts/strip_svg_graph_metadata.py` — Graphviz writes the node ids and the full edge list into `` elements and again into XML comments, Mermaid into `id="L_start_auth_0"`, PlantUML into `id="Read config-to-Open input"`, and a fixture that names its own answer cannot measure geometry recovery. Stripping removes no geometry: each variant renders to a byte-identical PNG. `nested_transforms.svg`, `icon_nodes.svg`, `mixed_page.svg` and the three `negative_*.svg` files are hand-written. Ground truth is the source graph restated by node label, so it is independent of any recogniser's numbering. diff --git a/README.md b/README.md index 64cda62..24c1d8a 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,31 @@ -# xberg PDF→Markdown benchmark corpus +# xberg test corpus -Ground truth for the xberg PDF→Markdown benchmark. **Every document here is reproduced from a pinned -upstream source and gated against an independent text oracle** — see `ground_truth/corpus_manifest.json` -for per-document provenance and verdicts. Do not hand-edit GT; change the builder and re-run. +Test documents and ground truth for [xberg](https://github.com/xberg-io/xberg): about 1,400 +documents spanning 105 file extensions, and 1,465 ground-truth files saying what extracting them +should produce. 638 of the documents are binaries served from a bucket rather than committed here. + +Consumers use it as a git submodule (`xberg/test_documents`) and as the corpus behind the +PDF→Markdown benchmark harness. Two things live here and they are governed differently: + +- **Documents.** The inputs. Text formats are committed to git; binaries are not (see below). +- **Ground truth.** What correct extraction looks like. Always committed, always text. ## Getting the documents -**The binary documents are not in this repository.** Git tracks only text — ground truth, manifests -and scripts. The binaries live in the public bucket `gs://xberg-test-documents`, content-addressed by -sha256 and pinned by `corpus.lock.json`, which maps each path to the object that belongs there. +**The binary documents are not in this repository, and they are not in Git LFS either.** Git +tracks only text — ground truth, manifests, scripts, and the text-format fixtures. The binaries live +in the public Google Cloud Storage bucket `gs://xberg-test-documents`, content-addressed by sha256 +and pinned by `corpus.lock.json`, which maps each repository path to the object that belongs there. + +The bucket is world-readable, so fetching needs no credentials, no `gcloud`, and no SDK: + +```sh +python3 scripts/fetch_corpus.py # everything, ~580 MiB +python3 scripts/fetch_corpus.py --include 'pdf/**' # just the PDFs +``` + +Files already present with the right hash are skipped, so re-running is cheap. Several consumers +reference fixtures through `include_bytes!`, so the bytes must be on disk before `cargo build` runs. In CI, use the shared action, which fetches only what a job needs and caches on the manifest: @@ -19,98 +36,167 @@ In CI, use the shared action, which fetches only what a job needs and caches on pdf/fake_memo.pdf ``` -Locally, materialise the files into their paths with the same manifest and the same anonymous -HTTPS. Several consumers reference fixtures through `include_bytes!`, so the bytes must be on disk -before `cargo build` runs: +Anything else can read `corpus.lock.json` and fetch over plain HTTPS directly: ```text -python3 scripts/fetch_corpus.py # everything, ~578 MiB -python3 scripts/fetch_corpus.py --include 'pdf/**' # just the PDFs +https://storage.googleapis.com/xberg-test-documents/objects/<sha256> ``` -Files already present with the right hash are skipped, so re-running is cheap. Anything else can -read `corpus.lock.json` and fetch over plain HTTPS directly — the bucket is world-readable, so no -credentials or SDK are involved: +### Why a bucket and not Git LFS -```text -https://storage.googleapis.com/xberg-test-documents/objects/<sha256> -``` +The corpus moved off Git LFS. LFS bills bandwidth per clone and makes every consumer authenticate +against it, including CI jobs that need three files out of six hundred. Content-addressed objects in +a public bucket are anonymous to read, cache trivially, and let a job fetch exactly the paths it +uses. `scripts/corpus-patterns.txt` was lifted verbatim from the old `.gitattributes` filter list so +the publish set did not change in the move. -`python3 scripts/verify_corpus.py --bucket xberg-test-documents` checks that every pinned object is -still served with the pinned size; add `--sample N` to download some in full and verify their hashes. +## What is in git and what is in the bucket -### Adding or changing a document +`scripts/corpus-patterns.txt` is the authority. It uses gitignore semantics: a pattern with no `/` +matches a basename at any depth. Every line in it is mirrored in `.gitignore`, and +`scripts/test_publish_corpus.py` fails if the two drift apart. -Put the file in its directory (`.gitignore` keeps it out of git), then publish it and commit the -refreshed pin: +**Bucket-managed** (never committed): `.pdf .doc .docx .odt .rtf .msg .pst .xls .xlsx .xlsm .xlsb +.xlam .xla .ods .ppt .pptx .pptm .ppsx .odp .key .epub .fb2 .pages .numbers .hwp .hwpx .png .jpg +.jpeg .gif .bmp .webp .tiff .tif .heif .heic .avif .jp2 .jpx .jpm .j2k .j2c .mj2 .ppm .pnm .pgm +.pbm .wav .mp3 .zip .tar .tgz .gz .7z .dbf`, plus `ground_truth/structured/parsebench/*.jsonl`. -```text -python3 scripts/publish_corpus.py --bucket xberg-test-documents -git add corpus.lock.json && git commit -``` +**Git-tracked**: everything else — all ground truth, and the text-format fixtures +(`.svg .xml .html .md .rst .org .tex .typ .json .yaml .csv .tsv .eml .opml .ipynb .wp .wpd .wpg +.dot .mmd .puml .fodg` and friends). -This needs write access to the bucket, so CI cannot do it — a CI checkout contains no binaries to -publish. Outside contributors should open a pull request describing the fixture; a maintainer -publishes it. `scripts/corpus-patterns.txt` declares which files count as corpus binaries. +The split is by *format*, not by size: a fixture whose bytes a human can read in a diff belongs in +git, because that is where review happens. -## Reproduce +## Adding or changing a document -The corpus builder lives in the **xberg** repository, not here: +1. Put the file in its directory. If it is a bucket-managed extension, `.gitignore` keeps it out of + git automatically. +2. Add ground truth under `ground_truth/<ext>/<stem>.{txt,md}` — these **are** committed. +3. Register it with the benchmark harness if it should be scored: a descriptor JSON in + `xberg/tools/benchmark-harness/fixtures/` (see below). +4. Record provenance. Anything from a third party needs an entry in `ATTRIBUTIONS.md` and, for a + multi-file set, a `PROVENANCE.md` with per-file sha256 (`wordperfect/PROVENANCE.md` is the + template). +5. **If the file is bucket-managed**, a maintainer publishes it and commits the refreshed pin: -```text -python tools/benchmark-harness/scripts/build_corpus.py --stage all -``` + ```sh + python3 scripts/publish_corpus.py --bucket xberg-test-documents --dry-run # check first + python3 scripts/publish_corpus.py --bucket xberg-test-documents + git add corpus.lock.json && git commit + ``` + + This needs write access to the bucket, so **CI cannot do it** — a CI checkout contains no + binaries to publish. Outside contributors should open a pull request describing the fixture and a + maintainer publishes it. +6. **Publish before you push the refreshed lock file.** CI verifies that every pinned object + resolves from the bucket; a lock file that names an object nobody uploaded fails the build. +7. If the fixture uses an extension not yet in `scripts/corpus-patterns.txt` and it should be + bucket-managed, add the pattern to **both** that file and `.gitignore`, with identical text. + +Never `git add` a corpus binary. `publish_corpus.py` refuses to run if one was committed. -This is the ONLY sanctioned way to modify the corpus. It acquires the pinned sources, normalizes GT to -canonical GFM, gates each doc, and writes `pdf/`, `ground_truth/pdf/<stem>.{md,txt}`, the fixtures, and -this file. Re-running with the same pins is deterministic. +## Layout + +| directory | what | +|---|---| +| `pdf/`, `pdf_scanned/`, `charts/` | 247 PDFs: born-digital, scanned, and chart-heavy | +| `docx/`, `doc/`, `odt/`, `rtf/`, `wordperfect/`, `hwp/`, `hwpx/` | word-processor formats, current and legacy | +| `pptx/`, `ppt/`, `odp/` | presentations | +| `xlsx/`, `xls/`, `data_formats/`, `csv/`, `dbf/` | spreadsheets and tabular data | +| `images/`, `images_extra/` | raster fixtures; `images_extra/` is one file per exotic codec | +| `xml/`, `html/`, `markdown/`, `markup/`, `rst/`, `org/`, `latex/`, `typst/`, `docbook/`, `jats/` | markup and text formats | +| `diagrams/` | vector diagram fixtures for node/edge recovery — see `diagrams/README.md` | +| `email/`, `epub/`, `fictionbook/`, `iwork/`, `jupyter/`, `opml/`, `archives/`, `audio/` | everything else | +| `vendored/` | third-party corpora kept verbatim with their own provenance | +| `ground_truth/` | expected output, one subdirectory per source extension | +| `scripts/` | the corpus tooling: fetch, publish, verify | + +## Ground truth + +`ground_truth/<ext>/<stem>.txt` is plaintext ground truth, scored as text-F1. +`ground_truth/<ext>/<stem>.md` is canonical-GFM ground truth, scored structurally. Both may exist; +for pure-OCR fixtures they are byte-identical by design. + +Two kinds are not plain text: + +- `ground_truth/dot/<stem>.dot` — the node/edge graph a diagram fixture draws, as Graphviz DOT keyed + by node label. Indexed by `diagrams/manifest.json`, which also carries provenance, the recovery + class, and the page/bbox of each graph within its document. An empty file means the fixture is a + negative control, and the manifest says why. See `diagrams/README.md`. +- `ground_truth/structured/` — field- and formula-level extraction targets, with their own manifest. + +Two files index the rest: + +- `ground_truth/corpus_manifest.json` — the immutable benchmark manifest: per-document hashes, + source, license, revision, normalization transforms, oracle verdict and scores, cohorts, size + tier, and tune/eval role, under one frozen top-level hash. **Generated — do not hand-edit.** +- `ground_truth/ground_truth_mapping.json` — a flat `stem → path` index. + +The authoritative binding between a document and its ground truth is neither of those: it is the +per-document descriptor in `xberg/tools/benchmark-harness/fixtures/*.json`, which names the +document, its type, its expected frameworks, and its ground-truth files. + +## Licensing -## Sources +`test_documents` is part of xberg, which is MIT-licensed, public, non-commercial open source. The +MIT `LICENSE` covers **our own work** — the tooling, the manifests, the prose. It does not +relicense third-party content: every source document and its upstream ground truth keeps its own +license. See `LICENSES.md` and `ATTRIBUTIONS.md`. -| dataset | license | GT provenance | role | -|---|---|---|---| -| [lazyc/READoc](https://huggingface.co/datasets/lazyc/READoc) @`HEAD` | MIT | arXiv GT = author LaTeX→pandoc (no tables); GitHub GT = author README rendered to PDF | document | -| [llamaindex/ParseBench](https://huggingface.co/datasets/llamaindex/ParseBench) @`HEAD` | Apache-2.0 | only table.jsonl ships expected_markdown (HTML tables); human-verified | page | -| [bsmock/FinTabNet.c](https://huggingface.co/datasets/bsmock/FinTabNet.c) @`HEAD` | CDLA-Permissive-2.0 | financial-statement table crops; GT = canonicalized cell structure rendered to GFM | page | -| [federalregister.gov](https://www.federalregister.gov) @`2026-07-` | US-PD (17 U.S.C. §105) | OFR/GPO full-text XML → GFM (headings, label blocks, GPOTABLE pipe tables) | document | +Datasets are handled in two classes: -Excluded on purpose: **OmniDocBench** (research-only / non-commercial — incompatible with this MIT repo) -and **Nougat** (weights CC-BY-NC; corpus not distributed). +- **vendor** — permissively licensed (MIT / Apache-2.0 / BSD / CC-BY / CC0 / CDLA-Permissive / + MPL-2.0 / US public domain). Committed here, with attribution. +- **reference** — non-commercial, ShareAlike, or research-only. **Never redistributed here.** + Fetched to local staging on demand and used only for non-commercial benchmarking. Their manifest + entries carry the source URL and license for provenance. -## How the data was modified +This keeps the public repo free of content it cannot redistribute while still letting the +benchmark use data that cannot be shipped. -Upstream GT is not committed verbatim — it is normalized to canonical GFM so it can be scored -consistently. The transforms are declared once in `scripts/normalize_gt.py` and applied by -`build_corpus.py`; this section and the per-doc `transforms` field in the manifest are generated from -the build ledger, so they always match what actually ran. +## Reproducing the benchmark corpus -| transform | applies to | what it changes | +The builder for the PDF→Markdown benchmark slice lives in the **xberg** repository, not here: + +```sh +python tools/benchmark-harness/scripts/build_corpus.py --stage all +``` + +It acquires the pinned sources, normalizes ground truth to canonical GFM, gates each document +against an independent text oracle, and writes `pdf/`, `ground_truth/pdf/<stem>.{md,txt}` and the +manifest. Re-running with the same pins is deterministic. This is the only sanctioned way to modify +that slice; hand-added fixtures elsewhere in the corpus are not covered by it. + +| dataset | license | ground-truth provenance | |---|---|---| -| `math_display` | ReaDoc arXiv | display math \[…\] → $$…$$ | -| `math_inline` | ReaDoc arXiv | inline math \(…\) → $…$ | -| `double_bold` | ReaDoc arXiv | merge pandoc doubled bold **** (bold-close+bold-open) | -| `trailing_ws` | all sources | strip trailing whitespace | -| `blank_runs` | all sources | collapse >2 blank lines to one | +| [lazyc/READoc](https://huggingface.co/datasets/lazyc/READoc) | MIT | arXiv GT = author LaTeX→pandoc (no tables); GitHub GT = author README rendered to PDF | +| [llamaindex/ParseBench](https://huggingface.co/datasets/llamaindex/ParseBench) | Apache-2.0 | only `table.jsonl` ships expected markdown (HTML tables); human-verified | +| [bsmock/FinTabNet.c](https://huggingface.co/datasets/bsmock/FinTabNet.c) | CDLA-Permissive-2.0 | financial-statement table crops; GT = canonicalized cell structure rendered to GFM | +| [federalregister.gov](https://www.federalregister.gov) | US-PD (17 U.S.C. §105) | OFR/GPO full-text XML → GFM | -Applied this build: +Excluded on purpose: **OmniDocBench** (research-only) and **Nougat** (weights CC-BY-NC, corpus not +distributed). -- **2577** documents had at least one normalization applied. -- `math_inline`: 231969 substitutions across the corpus. -- `math_display`: 26997 substitutions across the corpus. -- `trailing_ws`: 26990 substitutions across the corpus. -- `double_bold`: 3490 substitutions across the corpus. -- `html_to_gfm`: 503 substitutions across the corpus. +Upstream ground truth is not committed verbatim — it is normalized to canonical GFM so it can be +scored consistently. The transforms are declared once in `scripts/normalize_gt.py` and applied by +the builder; the per-document `transforms` field in the manifest records what actually ran. -## Layout +## Verifying + +```sh +python3 scripts/verify_corpus.py --bucket xberg-test-documents # every pin resolves +python3 scripts/verify_corpus.py --bucket xberg-test-documents --sample 20 # download and re-hash +python3 -m unittest discover -s scripts -v # tooling tests +``` -- `pdf/<stem>.pdf` — source document. -- `ground_truth/pdf/<stem>.md` — normalized canonical-GFM GT (scored by the harness). -- `ground_truth/pdf/<stem>.txt` — plaintext GT (text-F1). -- `ground_truth/corpus_manifest.json` — immutable manifest: per-doc hashes, source+license+revision, - transforms, oracle verdict+scores, cohorts, size tier, tune/eval role; one frozen top-level hash. +CI (`.github/workflows/verify-corpus.yaml`) runs exactly these on every push and pull request. It +needs no credentials: it proves the manifest is still fetchable rather than trying to publish. -## Manifest, cohorts, tiers, roles +The unittest suite also covers the corpus itself where the corpus can contradict its own answer +key — `scripts/test_diagram_manifest.py` checks `diagrams/manifest.json` against the files and the +ground truth on disk. One further check needs a renderer and so is not part of CI: -See the plan and `corpus_manifest.json`. Cohorts tag execution mode (native-clean / native-corrupt-font -/ selective-OCR / forced-OCR) and diagnostic strata (tables, multicolumn, formulas, …). Size tiers -`smoke ⊂ core ⊂ extended` and a `tune`/`eval` role per doc support fast iteration without overfitting. +```sh +python3 scripts/check_diagram_ground_truth.py # needs graphviz; skips cleanly without it +``` diff --git a/diagrams/README.md b/diagrams/README.md new file mode 100644 index 0000000..1e90801 --- /dev/null +++ b/diagrams/README.md @@ -0,0 +1,218 @@ +# Diagram fixtures + +Fixtures for node/edge recovery — xberg-io/xberg#579. `manifest.json` is the index: it says what +each file is, where it came from, why it is in the set, and what the correct answer is. + +Ground truth lives in `ground_truth/dot/<stem>.dot`, written as a Graphviz graph keyed **by node +label** rather than by generated id, so it does not depend on how any one recogniser numbers its +output. It records what the file draws, not what any implementation currently returns. + +## Two classes of fixture + +`manifest.json` tags every fixture `class: "A"` or `class: "B"`, and the class decides how it +should be scored. + +**Class A — the file states its graph.** ODF `draw:connector` carries `draw:start-shape` and +`draw:end-shape`; PowerPoint `p:cxnSp` carries `<a:stCxn id= idx=/>`; SmartArt carries +`dgm:ptLst`/`dgm:cxnLst`; `.drawio` edges carry `source=`/`target=`; a fenced ` ```mermaid ` block +is the graph in plain text. Recovery is lossless, so a correct implementation scores **exactly +1.0**. A miss here is a bug, not a threshold to tune. + +**Class B — geometry only.** SVG and vector PDF give you shapes, strokes and text positions, and +the graph has to be inferred. Scoring is precision/recall against a threshold. + +One Class A fixture ships today: `src/libreoffice_connectors.fodg`. Everything else is Class B. + +## The metadata problem + +Every diagram tool writes the graph it just laid out back into its own output, each in its own +way: + +```xml +<!-- graphviz: the node id, the whole edge, and both again in comments --> +<g id="edge1" class="edge"><title>a->b + + + + + + + + +``` + +That is the complete edge list, verbatim, in the file — and xberg's SVG extractor already collects +`` (`SVG_TEXT_ELEMENTS` in `extraction/xml.rs`). A recogniser that reads it scores 4/4 nodes +and 4/4 edges on every Graphviz fixture without inspecting a single coordinate. + +Reading it is not cheating: when a file states its graph, using it is the correct and exact thing +to do — that is what Class A *is*. It is simply a different capability from inferring a graph from +geometry, and measuring the two together measures neither. So each affected fixture ships twice, as +emitted and stripped: + +```sh +python3 scripts/strip_svg_graph_metadata.py \ + diagrams/svg/graphviz_flow.svg diagrams/svg/graphviz_flow_geometry.svg +``` + +The stripper works on two producer-agnostic rules rather than a list of the dialects seen so far: +drop `<title>` inside an element group, and drop every identifier the document itself never refers +to. What stays is anything giving an element's *type* without naming its endpoints — `class="node"`, +`class="com.sun.star.drawing.ConnectorShape"` — because knowing a stroke is some connector still +leaves you the whole job of working out what it connects. XML comments go entirely; provenance +belongs in `manifest.json`, which is where a reader should be looking for it. + +The `*_geometry.svg` variants share the ground truth of their originals: same answer, arrived at a +harder way, and identical to the pixel — the stripped file renders to a byte-identical PNG. +`manifest.json` marks which fixtures need one with `states_graph_in_metadata`. + +Two producers need no variant. LibreOffice numbers its shapes `id1`..`id9` and says only what kind +of shape each one is; PlantUML's swimlane output carries no id, no class and no comment at all. +Both are honest by construction. + +## What each fixture exercises + +Positives, by producer: + +| file | producer | n/e | exercises | +|---|---|---|---| +| `svg/graphviz_flow.svg` | Graphviz | 4/4 | box, diamond and ellipse nodes; arrowheads; edge labels; a dashed edge; root `translate` with negative coordinates | +| `svg/graphviz_states.svg` | Graphviz | 4/4 | `doublecircle` — one node drawn as two concentric outlines; a pair of antiparallel edges | +| `svg/graphviz_network.svg` | Graphviz | 5/4 | undirected `--` edges, so no arrowhead anywhere; `neato` layout | +| `svg/graphviz_bidirectional.svg` | Graphviz | 3/3 | `dir=both` and `dir=back` — the arrowhead at the tail, so the edge reads the other way round | +| `svg/graphviz_clusters.svg` | Graphviz | 5/4 | two cluster containers that are **not** nodes; an edge crossing a container boundary | +| `svg/graphviz_selfloop.svg` | Graphviz | 4/5 | self-loops, one labelled; two edges that cross in mid-drawing and share no endpoint | +| `svg/graphviz_ortho.svg` | Graphviz | 5/5 | orthogonal elbow routing, the default in every non-Graphviz tool | +| `svg/graphviz_record.svg` | Graphviz | 3/2 | `shape=record`: one outline divided by internal rules, edges anchored to a named port | +| `svg/graphviz_cjk.svg` | Graphviz | 5/4 | stroke-only nodes with no fill at all; Japanese, Korean, Hebrew and Arabic labels | +| `svg/graphviz_large.svg` | Graphviz | 128/141 | scale, for the perf and memory profiling the review asked for | +| `svg/mermaid_flow.svg` | Mermaid 11.16.0 | 6/6 | a 4.4 KB CSS block, ten `<marker>` defs, HTML labels in `<foreignObject>`, and edge labels on an opaque background box that looks exactly like a small node | +| `svg/plantuml_activity.svg` | PlantUML 1.2026.0 | 5/4 | rounded activity shapes; unlabelled `(*)` terminals | +| `svg/plantuml_swimlane.svg` | PlantUML 1.2026.0 | 5/4 | three swimlane bands that are **not** nodes | +| `svg/libreoffice_connectors.svg` | LibreOffice 26.2.5.2 | 4/3 | `draw:custom-shape` enhanced geometry; glued connectors that stop a few units short of the outline, so endpoints match only by proximity | +| `src/libreoffice_connectors.fodg` | hand-authored | 4/3 | **Class A** — `draw:start-shape`/`draw:end-shape` name the endpoints outright | +| `svg/nested_transforms.svg` | hand-authored | 4/3 | nested `translate`/`scale` groups plus a viewBox that differs from the viewport, so nothing sits at the coordinate it is written at | +| `svg/icon_nodes.svg` | hand-authored | 4/3 | the AWS/Azure house style: a node is an icon glyph with its caption underneath and no outline at all | +| `svg/mixed_page.svg` | hand-authored | 3/2 | a whole page — heading, prose, a ruled table, and one figure. Recovery has to be selective *within* the page | +| `xml/org_chart.svg` | hand-authored | 9/3 | multi-line labels, six isolated nodes | +| `xml/flowchart.svg` | hand-authored | 4/3 | `marker-end` arrowheads, annotations outside every shape | + +`src/` holds every source — `.dot`, `.mmd`, `.puml`, `.fodg` — so all of the above is regenerable. + +## Ground-truth conventions + +- **Keyed by node label.** A recogniser's own numbering never enters into it. +- **Only labelled nodes.** An unlabelled decoration — PlantUML's start/stop markers, an arrowhead, + a pie chart's leader dot — has no key and is not a node, and an edge joining one is not an edge. +- **Containers are not nodes.** Cluster rectangles, swimlane bands and lane headers are absent from + the ground truth however closed their outlines look. +- **One record is one node.** Its key is the field texts as one multi-line label, the same way + `org_chart.dot` keys a two-line box, because splitting a record into one node per field is the + failure that fixture exists to catch. +- **Direction as drawn.** `dir=back` puts the arrowhead at the tail, so the ground truth records + what the drawing shows, not the order the source declared. +- **Undirected stays undirected** — `graph` and `--`, never restated as a `digraph`. +- **Comments carry no DOT syntax.** Not every consumer strips `//`, so a comment mentioning + `a -> b` gets scored as an edge. `test_diagram_manifest.py` enforces this. + +## Negatives + +Six fixtures have `"negative": true`, an empty ground-truth file and a stated `reason` in the +manifest. Recovering anything from them is a false positive, and that is worth a test of its own. +The empty file plus the reason is what keeps "not a diagram" and "not yet annotated" from being the +same bytes. + +- `svg/negative_ruled_table.svg` — **the most dangerous one.** A table drawn as ruling lines has the + same signature as a diagram: closed rectangular regions with text inside, joined by straight + strokes running from the edge of one region to the next. Whatever rejects it also has to leave + table detection working. +- `svg/negative_pie_chart.svg` — each label sits in a rounded box joined to its slice by a + two-segment leader line ending in a dot, which is exactly the shape of a labelled node wired up + by an elbow connector. +- `svg/negative_form.svg` — captions beside empty boxes, section rules, checkboxes, a signature + line. The rules run right up to the boxes, which is what an edge looks like. +- `xml/data_dashboard.svg` — a bar chart. Closed outlines and straight strokes, but the strokes are + axes and gridlines. +- `xml/simple_svg.svg` — two unconnected shapes and a label. +- `images/5_level_paging_and_5_level_ept_intel_revision_1_1_may_2017.svg` — despite the filename, + an **inferno flame graph**: 77 nested `<rect>`s, 81 `<text>` labels, a CSS `<style>` block, an + ECMAScript `<script>` and a gradient `<defs>`. Adjacency is stack containment, not connection, and + there is not one connector in the file. It was already shipping; the only thing missing was the + assertion that recovery returns nothing. + +`svg/mixed_page.svg` is the seventh case and the awkward one: a negative and a positive in the same +file, so the answer cannot be decided per file. + +## Regenerate + +```sh +brew install graphviz # 15.1.1 +brew install --cask libreoffice # 26.2.5.2 +curl -sLO https://github.com/plantuml/plantuml/releases/download/v1.2026.0/plantuml-1.2026.0.jar + +cd diagrams/src +for f in graphviz_flow graphviz_states graphviz_bidirectional \ + graphviz_clusters graphviz_ortho graphviz_record graphviz_cjk graphviz_large; do + dot -Tsvg "$f.dot" -o "../svg/$f.svg" +done +for f in graphviz_network graphviz_selfloop; do + neato -Tsvg "$f.dot" -o "../svg/$f.svg" +done +cd ../.. + +npx @mermaid-js/mermaid-cli@11.16.0 \ + -i diagrams/src/mermaid_flow.mmd -o diagrams/svg/mermaid_flow.svg -b transparent +java -jar plantuml-1.2026.0.jar -tsvg -o "$PWD/diagrams/svg" diagrams/src/*.puml +soffice --headless --convert-to svg --outdir diagrams/svg diagrams/src/libreoffice_connectors.fodg + +for f in graphviz_flow graphviz_states graphviz_bidirectional graphviz_network \ + graphviz_clusters graphviz_ortho graphviz_record graphviz_cjk graphviz_large \ + graphviz_selfloop mermaid_flow plantuml_activity; do + python3 scripts/strip_svg_graph_metadata.py \ + "diagrams/svg/$f.svg" "diagrams/svg/${f}_geometry.svg" +done + +python3 scripts/check_diagram_ground_truth.py # ground truth still matches what was drawn +python3 -m unittest discover -s scripts # manifest still matches the files +``` + +Output is stable for a given tool version; a different version may lay a graph out differently, +which changes coordinates but not the graph — and the ground truth is written in terms of the +graph. The four Graphviz fixtures added before this set regenerate byte-identically under 15.1.1. + +`nested_transforms.svg`, `icon_nodes.svg`, `mixed_page.svg` and the three `negative_*.svg` are +hand-authored and are not regenerated. + +## Checks + +`scripts/test_diagram_manifest.py` runs in CI with no renderer installed and asserts that every +indexed path exists, that the node and edge counts in the manifest match the ground truth, that an +undirected graph is written as one, that every negative has an empty ground truth and a reason, and +that each `*_geometry.svg` is byte-for-byte what stripping its parent produces — which is the only +check that does not depend on knowing how a given producer encodes its answer. + +`scripts/check_diagram_ground_truth.py` needs Graphviz and re-derives each graph from its `.dot` +source with `dot -Tplain`, diffing it against the hand-written ground truth. It reports a skip +rather than failing when Graphviz is absent. + +## Why generated rather than hand-drawn + +Hand-written SVG exercises almost none of what a real diagram tool emits. The four SVGs originally +in `xml/` have no transform chain, no arrowheads, no double borders and no curved connectors, so a +recogniser can pass all four while being wrong about every file a user would actually bring. +Running recovery over real `dot -Tsvg` output found four defects those fixtures could not reach: +every arrowhead read as a node; every edge terminating on the arrowhead rather than the shape +behind it; `doublecircle` split into two concentric nodes; and the midpoint of a straight two-point +connector computed as its endpoint, so straight edges could never carry a label at all. + +Rendering a known graph also gives ground truth for free — the source **is** the correct answer — +so the corpus measures recovery rather than freezing whatever the code currently returns. + +## Not yet here + +Vector PDF, PowerPoint and SmartArt connectors, DOCX canvas and VML, XLSX drawings, and the +diagram-native formats (`.drawio`, `.excalidraw`, `.bpmn`, `.vsdx`, `.graphml`) are all still +missing, as is draw.io's SVG dialect with its embedded `mxGraphModel`, and Excalidraw's, where one +logical stroke becomes many wobbly subpaths. So is a multi-page document with the diagram on page +N, which is what the `page` and `bbox` slots in the manifest exist for. See the plan on +xberg-io/xberg#1410. diff --git a/diagrams/manifest.json b/diagrams/manifest.json new file mode 100644 index 0000000..6f99991 --- /dev/null +++ b/diagrams/manifest.json @@ -0,0 +1,454 @@ +{ + "schema": 1, + "comment": [ + "Diagram fixtures for node/edge recovery (xberg-io/xberg#579). Each entry says what the", + "fixture is, where it came from, why it is in the set, and what the correct answer is.", + "", + "`class` is the recovery mode the fixture measures, and it decides how it should be scored:", + " A -- the file states its graph outright (connector endpoints, model ids, source/target", + " attributes, or diagram source text). Recovery is lossless, so a correct", + " implementation scores exactly 1.0. Anything less is a bug, not a tuning problem.", + " B -- the file carries geometry only. The graph has to be inferred from shapes, strokes", + " and text positions, so scoring is precision/recall against a threshold.", + "", + "`states_graph_in_metadata` marks a Class-B file that also happens to name its own graph.", + "Every producer does it somewhere: Graphviz writes one <title> per node holding the node id", + "and one per edge holding `src->dst`, and repeats both in XML comments; Mermaid names both", + "endpoints in `id=\"L_start_auth_0\"`; PlantUML in `id=\"Read config-to-Open input\"`. Such a", + "file cannot measure geometry recovery, because a recogniser can score perfectly without", + "reading a single coordinate -- and xberg's SVG extractor already reads <title>. Those", + "fixtures therefore ship twice, and `geometry_only_variant` names the second copy, produced", + "by `scripts/strip_svg_graph_metadata.py`. The two share one ground truth: same answer,", + "arrived at a harder way. Reading the metadata when it is there is correct behaviour -- it", + "is just a different capability, and the two are measured apart.", + "", + "Ground truth is keyed by node label, so an unlabelled decoration -- PlantUML's start and", + "stop markers, an arrowhead, a leader dot -- has no key and is not a node. Cluster, swimlane", + "and lane containers are not nodes either, however closed their outlines look." + ], + "fixtures": [ + { + "path": "diagrams/svg/graphviz_flow.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_flow_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_flow.dot", + "license": "authored-here", + "exercises": [ + "shape:box", "shape:diamond", "shape:ellipse", "arrowhead:normal", + "edge-label", "edge:dashed", "transform:root-translate-negative" + ], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_flow.dot", + "directed": true, "nodes": 4, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_states.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_states_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_states.dot", + "license": "authored-here", + "exercises": ["shape:doublecircle", "edges:antiparallel-pair", "edge-label"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_states.dot", + "directed": true, "nodes": 4, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_network.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_network_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "neato -Tsvg"}, + "source": "diagrams/src/graphviz_network.dot", + "license": "authored-here", + "exercises": ["edges:undirected", "arrowhead:none", "layout:neato"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_network.dot", + "directed": false, "nodes": 5, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_bidirectional.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_bidirectional_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_bidirectional.dot", + "license": "authored-here", + "notes": "dir=back draws the arrowhead at the tail, so the ground truth records the drawn direction, not the declaration order.", + "exercises": ["edge:dir-both", "edge:dir-back", "rankdir:LR"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_bidirectional.dot", + "directed": true, "nodes": 3, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_clusters.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_clusters_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_clusters.dot", + "license": "authored-here", + "notes": "The two cluster rectangles are containers, not nodes. Reporting either is a false positive, and the stripped variant removes the <title> that would have given the game away.", + "exercises": ["container:cluster", "container:nested-fill", "edge:crosses-container-boundary"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_clusters.dot", + "directed": true, "nodes": 5, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_selfloop.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_selfloop_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "neato -Tsvg"}, + "source": "diagrams/src/graphviz_selfloop.dot", + "license": "authored-here", + "notes": "Two edges cross in the middle of the drawing and share no endpoint at either end. A recogniser that joins strokes where they intersect invents a node there, an edge, or both.", + "exercises": ["edge:self-loop", "edge:self-loop-labelled", "edges:crossing-unconnected", "layout:fixed-positions"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_selfloop.dot", + "directed": true, "nodes": 4, "edges": 5, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_ortho.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_ortho_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_ortho.dot", + "license": "authored-here", + "notes": "Orthogonal elbow routing is the default in every non-Graphviz diagram tool, and an elbow leg can run alongside an unrelated node before turning.", + "exercises": ["edge:orthogonal-elbow", "edges:converging-on-one-node"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_ortho.dot", + "directed": true, "nodes": 5, "edges": 5, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_record.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_record_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_record.dot", + "license": "authored-here", + "notes": "One record is one node drawn as a single outline divided by internal rules, and edges terminate on a named port rather than the shape. Splitting a record into one node per field is the failure this looks for, so the node key is the field texts as one multi-line label.", + "exercises": ["shape:record", "node:internal-rules", "edge:port-anchored"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_record.dot", + "directed": true, "nodes": 3, "edges": 2, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_cjk.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_cjk_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_cjk.dot", + "license": "authored-here", + "notes": "No node in this drawing has a fill, so a recogniser keying on filled regions finds nothing. Labels are Japanese, Korean, Hebrew and Arabic.", + "exercises": ["node:stroke-only", "label:cjk", "label:rtl", "edge-label:non-latin"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_cjk.dot", + "directed": true, "nodes": 5, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/graphviz_large.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/graphviz_large_geometry.svg", + "producer": {"tool": "graphviz", "version": "15.1.1", "command": "dot -Tsvg"}, + "source": "diagrams/src/graphviz_large.dot", + "license": "authored-here", + "notes": "128 nodes and 141 edges, for the perf and memory profiling the review on xberg#1410 asked for. Regenerable from one rule: a complete binary tree over N0..N127 plus a dashed shortcut N_i -> N_i+16 for every i divisible by 8.", + "exercises": ["scale:128-nodes", "scale:141-edges", "edges:long-range-shortcuts"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/graphviz_large.dot", + "directed": true, "nodes": 128, "edges": 141, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/mermaid_flow.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/mermaid_flow_geometry.svg", + "producer": {"tool": "@mermaid-js/mermaid-cli", "version": "11.16.0", "command": "mmdc -i mermaid_flow.mmd -o mermaid_flow.svg -b transparent"}, + "source": "diagrams/src/mermaid_flow.mmd", + "license": "authored-here", + "notes": "Mermaid's idioms are nothing like Graphviz's: a 4.4 KB CSS <style> block, ten <marker> defs, HTML labels in a <foreignObject>, and every edge label drawn on an opaque background rect (class=\"labelBkg\") that looks exactly like a small node. It names both endpoints of every edge in id=\"L_start_auth_0\" and repeats it in data-id.", + "exercises": [ + "producer:mermaid", "label:foreignobject-html", "arrowhead:marker-def", + "edge-label:opaque-background-box", "style:css-classes", "shape:stadium", "shape:diamond" + ], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/mermaid_flow.dot", + "directed": true, "nodes": 6, "edges": 6, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/plantuml_activity.svg", + "class": "B", + "states_graph_in_metadata": true, + "geometry_only_variant": "diagrams/svg/plantuml_activity_geometry.svg", + "producer": {"tool": "plantuml", "version": "1.2026.0", "command": "java -jar plantuml.jar -tsvg"}, + "source": "diagrams/src/plantuml_activity.puml", + "license": "authored-here", + "notes": "PlantUML states each edge twice over: id=\"Read config-to-Open input\" and an XML comment <!--link Read config to Open input-->. The (*) start and end are unlabelled markers, so they are not nodes and the two edges joining them are not edges. PlantUML itself is GPL; its SVG output is not a derivative of it.", + "exercises": ["producer:plantuml", "shape:rounded-activity", "node:unlabelled-terminal", "arrowhead:polygon"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/plantuml_activity.dot", + "directed": true, "nodes": 5, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/plantuml_swimlane.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "plantuml", "version": "1.2026.0", "command": "java -jar plantuml.jar -tsvg"}, + "source": "diagrams/src/plantuml_swimlane.puml", + "license": "authored-here", + "notes": "Swimlanes: three labelled bands spanning the drawing, each holding activities that belong to it. A band is not a node. This SVG carries no id, no class and no comment anywhere, so it is honest by construction and needs no stripped variant.", + "exercises": ["producer:plantuml", "container:swimlane", "container:lane-header-label", "node:unlabelled-terminal"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/plantuml_swimlane.dot", + "directed": true, "nodes": 5, "edges": 4, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/libreoffice_connectors.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "LibreOffice Draw", "version": "26.2.5.2", "command": "soffice --headless --convert-to svg"}, + "source": "diagrams/src/libreoffice_connectors.fodg", + "license": "authored-here", + "notes": "The one dialect that states nothing: shapes are numbered id1..id9, and the classes give a shape's type (com.sun.star.drawing.ConnectorShape) without naming what it connects. Knowing a stroke is some connector still leaves the whole task. Glued connectors also start and end a few units short of the shape outline, so endpoints have to be matched by proximity.", + "exercises": [ + "producer:libreoffice", "shape:custom-shape-enhanced-geometry", "shape:round-rectangle", + "connector:glued-with-gap", "text:tspan-with-textlength", "font:embedded" + ], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/libreoffice_connectors.dot", + "directed": true, "nodes": 4, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/src/libreoffice_connectors.fodg", + "class": "A", + "states_graph_in_metadata": true, + "producer": {"tool": "hand-authored", "version": null, "command": null}, + "source": null, + "license": "authored-here", + "notes": "The same drawing as an ODF source, and the first Class A fixture in the corpus. Every draw:connector names its endpoints outright with draw:start-shape and draw:end-shape against the draw:id of a shape, so recovery is a lookup and must be exact. Rendering this file is what produced diagrams/svg/libreoffice_connectors.svg, so one ground truth covers both the stated graph and the inferred one.", + "exercises": [ + "format:odf-flat-drawing", "connector:draw-start-shape", "connector:draw-end-shape", + "connector:glue-point-index", "shape:draw-custom-shape" + ], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/libreoffice_connectors.dot", + "directed": true, "nodes": 4, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/nested_transforms.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "hand-authored", "version": null, "command": null}, + "source": null, + "license": "authored-here", + "exercises": [ + "transform:nested-groups", "transform:scale", "viewbox-viewport-mismatch", + "node:dashed-border", "edge:dashed", "edge-label" + ], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/nested_transforms.dot", + "directed": true, "nodes": 4, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/icon_nodes.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "hand-authored", "version": null, "command": null}, + "source": null, + "license": "authored-here", + "notes": "The AWS/Azure architecture house style: a node is an icon glyph with its caption underneath and no bounding outline at all, so every label sits outside every closed region and no node has a border to find.", + "exercises": ["node:icon-glyph-no-outline", "label:below-shape", "arrowhead:marker-end", "node:multi-subpath-glyph"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/icon_nodes.dot", + "directed": true, "nodes": 4, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/mixed_page.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "hand-authored", "version": null, "command": null}, + "source": null, + "license": "authored-here", + "notes": "Every other fixture is entirely a diagram or entirely not one. This page is both: a heading, two paragraphs, a four-row ruled table, and one three-node figure at the foot. Recovery has to be selective within the page rather than deciding once per file, and the table must contribute nothing. bbox is the figure's extent in the SVG user space.", + "exercises": ["page:prose-and-table-and-diagram", "negative-in-positive:ruled-table", "figure:caption"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/mixed_page.dot", + "directed": true, "nodes": 3, "edges": 2, + "page": 1, "bbox": [52, 340, 488, 392] + } + ] + }, + { + "path": "xml/org_chart.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "hand-authored", "version": null, "command": null}, + "source": null, + "license": "authored-here", + "exercises": ["label:multi-line", "nodes:isolated", "annotations-outside-shapes"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/org_chart.dot", + "directed": true, "nodes": 9, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "xml/flowchart.svg", + "class": "B", + "states_graph_in_metadata": false, + "producer": {"tool": "hand-authored", "version": null, "command": null}, + "source": null, + "license": "authored-here", + "exercises": ["arrowhead:marker-end", "annotations-outside-shapes"], + "graphs": [ + { + "id": "g0", + "ground_truth": "ground_truth/dot/flowchart.dot", + "directed": true, "nodes": 4, "edges": 3, "page": null, "bbox": null + } + ] + }, + { + "path": "diagrams/svg/negative_ruled_table.svg", + "class": "B", + "negative": true, + "reason": "A table drawn as ruling lines has the same signature as a diagram: closed rectangular regions with text inside them, joined by straight strokes running from the edge of one region to the next. This is the most dangerous negative in the set, because whatever rejects it also has to not break table detection.", + "license": "authored-here", + "ground_truth": "ground_truth/dot/negative_ruled_table.dot", + "graphs": [] + }, + { + "path": "diagrams/svg/negative_pie_chart.svg", + "class": "B", + "negative": true, + "reason": "Each label sits in its own rounded box joined to its slice by a two-segment leader line ending in a dot, which is exactly the shape of a labelled node wired up by an elbow connector. The leader lines connect nothing and the slices are not nodes.", + "license": "authored-here", + "ground_truth": "ground_truth/dot/negative_pie_chart.dot", + "graphs": [] + }, + { + "path": "diagrams/svg/negative_form.svg", + "class": "B", + "negative": true, + "reason": "A printed form: captions beside empty boxes, section rules, a row of checkboxes, a signature line. Every ingredient of a diagram is present and there is no graph, and the rules in particular run right up to the boxes, which is what an edge looks like.", + "license": "authored-here", + "ground_truth": "ground_truth/dot/negative_form.dot", + "graphs": [] + }, + { + "path": "xml/data_dashboard.svg", + "class": "B", + "negative": true, + "reason": "bar chart: closed outlines and straight strokes, but the strokes are axes and gridlines, not connectors", + "license": "authored-here", + "ground_truth": "ground_truth/dot/data_dashboard.dot", + "graphs": [] + }, + { + "path": "xml/simple_svg.svg", + "class": "B", + "negative": true, + "reason": "two unconnected shapes and one label; there is no graph to recover", + "license": "authored-here", + "ground_truth": "ground_truth/dot/simple_svg.dot", + "graphs": [] + }, + { + "path": "images/5_level_paging_and_5_level_ept_intel_revision_1_1_may_2017.svg", + "class": "B", + "negative": true, + "reason": [ + "Despite the filename this is an inferno flame graph, not a paging diagram: 77 nested", + "<rect> elements, 81 <text> labels, 76 <g> wrappers, a CSS <style> block, an ECMAScript", + "<script>, and a gradient <defs>. Adjacency here is stack containment, not connection,", + "and there is not one connector in the file. It is the hardest negative in the corpus", + "and it was already shipping -- the only thing missing was the assertion that recovery", + "returns nothing." + ], + "license": "see ATTRIBUTIONS.md", + "ground_truth": "ground_truth/dot/5_level_paging_and_5_level_ept_intel_revision_1_1_may_2017.dot", + "graphs": [] + } + ] +} diff --git a/diagrams/src/graphviz_bidirectional.dot b/diagrams/src/graphviz_bidirectional.dot new file mode 100644 index 0000000..54b6150 --- /dev/null +++ b/diagrams/src/graphviz_bidirectional.dot @@ -0,0 +1,10 @@ +digraph graphviz_bidirectional { + rankdir=LR; + node [shape=box style=filled fillcolor="#fdb462"]; + api [label="API"]; + db [label="Database"]; + cache [label="Cache"]; + api -> db [dir=both]; + api -> cache; + cache -> db [dir=back]; +} diff --git a/diagrams/src/graphviz_cjk.dot b/diagrams/src/graphviz_cjk.dot new file mode 100644 index 0000000..f52959f --- /dev/null +++ b/diagrams/src/graphviz_cjk.dot @@ -0,0 +1,17 @@ +digraph graphviz_cjk { + rankdir=LR; + // Stroke-only nodes: no fill at all, so a recogniser keying on filled regions + // sees nothing. Labels are CJK, Hebrew and Arabic, none of which are Latin-1. + node [shape=box]; + + jp [label="受注処理"]; + kr [label="검증"]; + he [label="אימות"]; + ar [label="اعتماد"]; + zh [label="完成"]; + + jp -> kr [label="次"]; + kr -> he; + he -> ar; + ar -> zh [label="סיום"]; +} diff --git a/diagrams/src/graphviz_clusters.dot b/diagrams/src/graphviz_clusters.dot new file mode 100644 index 0000000..c5d11d5 --- /dev/null +++ b/diagrams/src/graphviz_clusters.dot @@ -0,0 +1,28 @@ +digraph graphviz_clusters { + rankdir=TB; + node [shape=box style=filled fillcolor="#a6cee3"]; + + subgraph cluster_ingest { + label="Ingest"; + style=filled; + fillcolor="#eef6fb"; + recv [label="Receive"]; + parse [label="Parse"]; + recv -> parse; + } + + subgraph cluster_store { + label="Storage"; + style=filled; + fillcolor="#fdf3e7"; + node [fillcolor="#fdbf6f"]; + index [label="Index"]; + archive [label="Archive"]; + } + + emit [label="Emit" shape=ellipse fillcolor="#b2df8a"]; + + parse -> index; + parse -> archive; + index -> emit; +} diff --git a/diagrams/src/graphviz_flow.dot b/diagrams/src/graphviz_flow.dot new file mode 100644 index 0000000..f00b5e3 --- /dev/null +++ b/diagrams/src/graphviz_flow.dot @@ -0,0 +1,11 @@ +digraph graphviz_flow { + rankdir=TB; + a [label="Start" shape=box style=filled fillcolor="#a6cee3"]; + b [label="Validate" shape=diamond style=filled fillcolor="#b2df8a"]; + c [label="Process" shape=box style=filled fillcolor="#fb9a99"]; + d [label="Done" shape=ellipse style=filled fillcolor="#fdbf6f"]; + a -> b; + b -> c [label="ok"]; + b -> d [label="skip" style=dashed]; + c -> d; +} diff --git a/diagrams/src/graphviz_large.dot b/diagrams/src/graphviz_large.dot new file mode 100644 index 0000000..06aef67 --- /dev/null +++ b/diagrams/src/graphviz_large.dot @@ -0,0 +1,280 @@ +digraph graphviz_large { + // Machine-generated, and regenerable from this rule alone: 128 nodes n0..n127 + // wired as a complete binary tree (n_i -> n_2i+1, n_i -> n_2i+2 where the target + // exists), plus a shortcut edge n_i -> n_i+16 for every i divisible by 8. + // Exists for the perf and memory profiling the review on xberg#1410 asked for. + rankdir=TB; + node [shape=box style=filled fillcolor="#d9d9d9"]; + + n0 [label="N0"]; + n1 [label="N1"]; + n2 [label="N2"]; + n3 [label="N3"]; + n4 [label="N4"]; + n5 [label="N5"]; + n6 [label="N6"]; + n7 [label="N7"]; + n8 [label="N8"]; + n9 [label="N9"]; + n10 [label="N10"]; + n11 [label="N11"]; + n12 [label="N12"]; + n13 [label="N13"]; + n14 [label="N14"]; + n15 [label="N15"]; + n16 [label="N16"]; + n17 [label="N17"]; + n18 [label="N18"]; + n19 [label="N19"]; + n20 [label="N20"]; + n21 [label="N21"]; + n22 [label="N22"]; + n23 [label="N23"]; + n24 [label="N24"]; + n25 [label="N25"]; + n26 [label="N26"]; + n27 [label="N27"]; + n28 [label="N28"]; + n29 [label="N29"]; + n30 [label="N30"]; + n31 [label="N31"]; + n32 [label="N32"]; + n33 [label="N33"]; + n34 [label="N34"]; + n35 [label="N35"]; + n36 [label="N36"]; + n37 [label="N37"]; + n38 [label="N38"]; + n39 [label="N39"]; + n40 [label="N40"]; + n41 [label="N41"]; + n42 [label="N42"]; + n43 [label="N43"]; + n44 [label="N44"]; + n45 [label="N45"]; + n46 [label="N46"]; + n47 [label="N47"]; + n48 [label="N48"]; + n49 [label="N49"]; + n50 [label="N50"]; + n51 [label="N51"]; + n52 [label="N52"]; + n53 [label="N53"]; + n54 [label="N54"]; + n55 [label="N55"]; + n56 [label="N56"]; + n57 [label="N57"]; + n58 [label="N58"]; + n59 [label="N59"]; + n60 [label="N60"]; + n61 [label="N61"]; + n62 [label="N62"]; + n63 [label="N63"]; + n64 [label="N64"]; + n65 [label="N65"]; + n66 [label="N66"]; + n67 [label="N67"]; + n68 [label="N68"]; + n69 [label="N69"]; + n70 [label="N70"]; + n71 [label="N71"]; + n72 [label="N72"]; + n73 [label="N73"]; + n74 [label="N74"]; + n75 [label="N75"]; + n76 [label="N76"]; + n77 [label="N77"]; + n78 [label="N78"]; + n79 [label="N79"]; + n80 [label="N80"]; + n81 [label="N81"]; + n82 [label="N82"]; + n83 [label="N83"]; + n84 [label="N84"]; + n85 [label="N85"]; + n86 [label="N86"]; + n87 [label="N87"]; + n88 [label="N88"]; + n89 [label="N89"]; + n90 [label="N90"]; + n91 [label="N91"]; + n92 [label="N92"]; + n93 [label="N93"]; + n94 [label="N94"]; + n95 [label="N95"]; + n96 [label="N96"]; + n97 [label="N97"]; + n98 [label="N98"]; + n99 [label="N99"]; + n100 [label="N100"]; + n101 [label="N101"]; + n102 [label="N102"]; + n103 [label="N103"]; + n104 [label="N104"]; + n105 [label="N105"]; + n106 [label="N106"]; + n107 [label="N107"]; + n108 [label="N108"]; + n109 [label="N109"]; + n110 [label="N110"]; + n111 [label="N111"]; + n112 [label="N112"]; + n113 [label="N113"]; + n114 [label="N114"]; + n115 [label="N115"]; + n116 [label="N116"]; + n117 [label="N117"]; + n118 [label="N118"]; + n119 [label="N119"]; + n120 [label="N120"]; + n121 [label="N121"]; + n122 [label="N122"]; + n123 [label="N123"]; + n124 [label="N124"]; + n125 [label="N125"]; + n126 [label="N126"]; + n127 [label="N127"]; + + n0 -> n1; + n0 -> n2; + n1 -> n3; + n1 -> n4; + n2 -> n5; + n2 -> n6; + n3 -> n7; + n3 -> n8; + n4 -> n9; + n4 -> n10; + n5 -> n11; + n5 -> n12; + n6 -> n13; + n6 -> n14; + n7 -> n15; + n7 -> n16; + n8 -> n17; + n8 -> n18; + n9 -> n19; + n9 -> n20; + n10 -> n21; + n10 -> n22; + n11 -> n23; + n11 -> n24; + n12 -> n25; + n12 -> n26; + n13 -> n27; + n13 -> n28; + n14 -> n29; + n14 -> n30; + n15 -> n31; + n15 -> n32; + n16 -> n33; + n16 -> n34; + n17 -> n35; + n17 -> n36; + n18 -> n37; + n18 -> n38; + n19 -> n39; + n19 -> n40; + n20 -> n41; + n20 -> n42; + n21 -> n43; + n21 -> n44; + n22 -> n45; + n22 -> n46; + n23 -> n47; + n23 -> n48; + n24 -> n49; + n24 -> n50; + n25 -> n51; + n25 -> n52; + n26 -> n53; + n26 -> n54; + n27 -> n55; + n27 -> n56; + n28 -> n57; + n28 -> n58; + n29 -> n59; + n29 -> n60; + n30 -> n61; + n30 -> n62; + n31 -> n63; + n31 -> n64; + n32 -> n65; + n32 -> n66; + n33 -> n67; + n33 -> n68; + n34 -> n69; + n34 -> n70; + n35 -> n71; + n35 -> n72; + n36 -> n73; + n36 -> n74; + n37 -> n75; + n37 -> n76; + n38 -> n77; + n38 -> n78; + n39 -> n79; + n39 -> n80; + n40 -> n81; + n40 -> n82; + n41 -> n83; + n41 -> n84; + n42 -> n85; + n42 -> n86; + n43 -> n87; + n43 -> n88; + n44 -> n89; + n44 -> n90; + n45 -> n91; + n45 -> n92; + n46 -> n93; + n46 -> n94; + n47 -> n95; + n47 -> n96; + n48 -> n97; + n48 -> n98; + n49 -> n99; + n49 -> n100; + n50 -> n101; + n50 -> n102; + n51 -> n103; + n51 -> n104; + n52 -> n105; + n52 -> n106; + n53 -> n107; + n53 -> n108; + n54 -> n109; + n54 -> n110; + n55 -> n111; + n55 -> n112; + n56 -> n113; + n56 -> n114; + n57 -> n115; + n57 -> n116; + n58 -> n117; + n58 -> n118; + n59 -> n119; + n59 -> n120; + n60 -> n121; + n60 -> n122; + n61 -> n123; + n61 -> n124; + n62 -> n125; + n62 -> n126; + n63 -> n127; + + n0 -> n16 [style=dashed]; + n8 -> n24 [style=dashed]; + n16 -> n32 [style=dashed]; + n24 -> n40 [style=dashed]; + n32 -> n48 [style=dashed]; + n40 -> n56 [style=dashed]; + n48 -> n64 [style=dashed]; + n56 -> n72 [style=dashed]; + n64 -> n80 [style=dashed]; + n72 -> n88 [style=dashed]; + n80 -> n96 [style=dashed]; + n88 -> n104 [style=dashed]; + n96 -> n112 [style=dashed]; + n104 -> n120 [style=dashed]; +} diff --git a/diagrams/src/graphviz_network.dot b/diagrams/src/graphviz_network.dot new file mode 100644 index 0000000..aed4a52 --- /dev/null +++ b/diagrams/src/graphviz_network.dot @@ -0,0 +1,13 @@ +graph graphviz_network { + layout=neato; + node [shape=ellipse style=filled fillcolor="#cccccc"]; + gw [label="Gateway" fillcolor="#8dd3c7"]; + s1 [label="Switch A" fillcolor="#ffffb3"]; + s2 [label="Switch B" fillcolor="#ffffb3"]; + h1 [label="Host 1"]; + h2 [label="Host 2"]; + gw -- s1; + gw -- s2; + s1 -- h1; + s2 -- h2; +} diff --git a/diagrams/src/graphviz_ortho.dot b/diagrams/src/graphviz_ortho.dot new file mode 100644 index 0000000..f44a97f --- /dev/null +++ b/diagrams/src/graphviz_ortho.dot @@ -0,0 +1,19 @@ +digraph graphviz_ortho { + splines=ortho; + rankdir=TB; + nodesep=0.8; + ranksep=0.8; + node [shape=box style=filled fillcolor="#ccebc5"]; + + client [label="Client"]; + lb [label="Balancer"]; + web1 [label="Web 1"]; + web2 [label="Web 2"]; + db [label="Database"]; + + client -> lb; + lb -> web1; + lb -> web2; + web1 -> db; + web2 -> db; +} diff --git a/diagrams/src/graphviz_record.dot b/diagrams/src/graphviz_record.dot new file mode 100644 index 0000000..577c1d7 --- /dev/null +++ b/diagrams/src/graphviz_record.dot @@ -0,0 +1,11 @@ +digraph graphviz_record { + rankdir=LR; + node [shape=record style=filled fillcolor="#fbb4ae"]; + + customer [label="<hd> Customer | <f0> id | <f1> name"]; + order [label="<hd> Order | <f0> id | <f1> customer_id | <f2> total"]; + item [label="<hd> Item | <f0> order_id | <f1> sku"]; + + customer:f0 -> order:f1; + order:f0 -> item:f0; +} diff --git a/diagrams/src/graphviz_selfloop.dot b/diagrams/src/graphviz_selfloop.dot new file mode 100644 index 0000000..204abe3 --- /dev/null +++ b/diagrams/src/graphviz_selfloop.dot @@ -0,0 +1,20 @@ +digraph graphviz_selfloop { + layout=neato; + overlap=false; + node [shape=circle style=filled fillcolor="#bebada"]; + + retry [label="Retry" pos="0,2!"]; + work [label="Work" pos="3,2!"]; + left [label="Left" pos="0,0!"]; + right [label="Right" pos="3,0!"]; + + retry -> retry [label="backoff"]; + work -> work; + retry -> work; + + // These two cross in the middle of the drawing and share no endpoint at either + // end. A recogniser that joins strokes where they intersect invents a node in + // the centre, an edge, or both. + left -> work; + retry -> right; +} diff --git a/diagrams/src/graphviz_states.dot b/diagrams/src/graphviz_states.dot new file mode 100644 index 0000000..ecdaa0f --- /dev/null +++ b/diagrams/src/graphviz_states.dot @@ -0,0 +1,9 @@ +digraph graphviz_states { + rankdir=LR; + node [shape=circle style=filled fillcolor="#b3de69"]; + idle -> running [label="start"]; + running -> paused [label="pause"]; + paused -> running [label="resume"]; + running -> done [label="finish"]; + done [shape=doublecircle fillcolor="#fb8072"]; +} diff --git a/diagrams/src/libreoffice_connectors.fodg b/diagrams/src/libreoffice_connectors.fodg new file mode 100644 index 0000000..0bc5a67 --- /dev/null +++ b/diagrams/src/libreoffice_connectors.fodg @@ -0,0 +1,69 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- Flat ODF drawing, hand-authored. It is the source for diagrams/svg/libreoffice_connectors.svg + and, in its own right, a Class A fixture: every draw:connector names its endpoints outright + with draw:start-shape and draw:end-shape, so recovering the graph from ODF is a lookup, not + an inference. Rendering it through LibreOffice then yields a Class B fixture in the + LibreOffice SVG dialect, where those same endpoints survive only as geometry. --> +<office:document + xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" + xmlns:draw="urn:oasis:names:tc:opendocument:xmlns:drawing:1.0" + xmlns:svg="urn:oasis:names:tc:opendocument:xmlns:svg-compatible:1.0" + xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" + xmlns:style="urn:oasis:names:tc:opendocument:xmlns:style:1.0" + xmlns:fo="urn:oasis:names:tc:opendocument:xmlns:xsl-fo-compatible:1.0" + office:version="1.3" + office:mimetype="application/vnd.oasis.opendocument.graphics"> + <office:automatic-styles> + <style:style style:name="box" style:family="graphic"> + <style:graphic-properties draw:fill="solid" draw:fill-color="#a6cee3" draw:stroke="solid" + svg:stroke-color="#333333"/> + </style:style> + <style:style style:name="terminal" style:family="graphic"> + <style:graphic-properties draw:fill="solid" draw:fill-color="#b2df8a" draw:stroke="solid" + svg:stroke-color="#333333"/> + </style:style> + <style:style style:name="line" style:family="graphic"> + <style:graphic-properties draw:fill="none" draw:stroke="solid" svg:stroke-color="#333333" + draw:marker-end="Arrow" draw:marker-end-width="0.2cm"/> + </style:style> + </office:automatic-styles> + <office:body> + <office:drawing> + <draw:page draw:name="page1"> + <draw:custom-shape draw:id="intake" draw:style-name="terminal" + svg:x="2cm" svg:y="1cm" svg:width="4cm" svg:height="1.5cm"> + <text:p>Intake</text:p> + <draw:enhanced-geometry draw:type="round-rectangle"/> + </draw:custom-shape> + <draw:custom-shape draw:id="review" draw:style-name="box" + svg:x="2cm" svg:y="4cm" svg:width="4cm" svg:height="1.5cm"> + <text:p>Review</text:p> + <draw:enhanced-geometry draw:type="rectangle"/> + </draw:custom-shape> + <draw:custom-shape draw:id="approve" draw:style-name="box" + svg:x="8cm" svg:y="4cm" svg:width="4cm" svg:height="1.5cm"> + <text:p>Approve</text:p> + <draw:enhanced-geometry draw:type="rectangle"/> + </draw:custom-shape> + <draw:custom-shape draw:id="publish" draw:style-name="terminal" + svg:x="8cm" svg:y="1cm" svg:width="4cm" svg:height="1.5cm"> + <text:p>Publish</text:p> + <draw:enhanced-geometry draw:type="round-rectangle"/> + </draw:custom-shape> + + <draw:connector draw:style-name="line" draw:type="standard" + draw:start-shape="intake" draw:start-glue-point="2" + draw:end-shape="review" draw:end-glue-point="0" + svg:x1="4cm" svg:y1="2.5cm" svg:x2="4cm" svg:y2="4cm"/> + <draw:connector draw:style-name="line" draw:type="standard" + draw:start-shape="review" draw:start-glue-point="1" + draw:end-shape="approve" draw:end-glue-point="3" + svg:x1="6cm" svg:y1="4.75cm" svg:x2="8cm" svg:y2="4.75cm"/> + <draw:connector draw:style-name="line" draw:type="standard" + draw:start-shape="approve" draw:start-glue-point="0" + draw:end-shape="publish" draw:end-glue-point="2" + svg:x1="10cm" svg:y1="4cm" svg:x2="10cm" svg:y2="2.5cm"/> + </draw:page> + </office:drawing> + </office:body> +</office:document> diff --git a/diagrams/src/mermaid_flow.mmd b/diagrams/src/mermaid_flow.mmd new file mode 100644 index 0000000..23f0dbb --- /dev/null +++ b/diagrams/src/mermaid_flow.mmd @@ -0,0 +1,7 @@ +flowchart TD + start([Receive request]) --> auth{Authenticated?} + auth -->|yes| quota[Check quota] + auth -->|no| reject[Return 401] + quota --> handler[Run handler] + handler --> respond([Return 200]) + reject --> respond diff --git a/diagrams/src/plantuml_activity.puml b/diagrams/src/plantuml_activity.puml new file mode 100644 index 0000000..0ffab1f --- /dev/null +++ b/diagrams/src/plantuml_activity.puml @@ -0,0 +1,12 @@ +@startuml plantuml_activity +skinparam monochrome false +skinparam shadowing false + +(*) --> "Read config" +"Read config" --> "Open input" +"Open input" --> "Detect format" +"Detect format" --> "Extract text" +"Extract text" --> "Render markdown" +"Render markdown" --> (*) + +@enduml diff --git a/diagrams/src/plantuml_swimlane.puml b/diagrams/src/plantuml_swimlane.puml new file mode 100644 index 0000000..cf82030 --- /dev/null +++ b/diagrams/src/plantuml_swimlane.puml @@ -0,0 +1,16 @@ +@startuml plantuml_swimlane +' Swimlanes: the lane is a labelled band spanning the drawing, and it is not a node. A +' recogniser that treats every closed region as a node invents three of them here. +|Author| +start +:Draft article; +|Editor| +:Review draft; +|Legal| +:Clear rights; +|Editor| +:Schedule slot; +|Author| +:Publish; +stop +@enduml diff --git a/diagrams/svg/graphviz_bidirectional.svg b/diagrams/svg/graphviz_bidirectional.svg new file mode 100644 index 0000000..8b95e24 --- /dev/null +++ b/diagrams/svg/graphviz_bidirectional.svg @@ -0,0 +1,50 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" + "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by graphviz version 15.1.1 (20260805.0921) + --> +<!-- Title: graphviz_bidirectional Pages: 1 --> +<svg width="254pt" height="67pt" + viewBox="0.00 0.00 254.00 67.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> +<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 63)"> +<title>graphviz_bidirectional + + + +api + +API + + + +db + +Database + + + +api->db + + + + + + +cache + +Cache + + + +api->cache + + + + + +cache->db + + + + + diff --git a/diagrams/svg/graphviz_bidirectional_geometry.svg b/diagrams/svg/graphviz_bidirectional_geometry.svg new file mode 100644 index 0000000..6c4adf6 --- /dev/null +++ b/diagrams/svg/graphviz_bidirectional_geometry.svg @@ -0,0 +1,35 @@ + + + + +graphviz_bidirectional + + + +API + + + +Database + + + + + + + + +Cache + + + + + + + + + + + diff --git a/diagrams/svg/graphviz_cjk.svg b/diagrams/svg/graphviz_cjk.svg new file mode 100644 index 0000000..162b386 --- /dev/null +++ b/diagrams/svg/graphviz_cjk.svg @@ -0,0 +1,69 @@ + + + + + + +graphviz_cjk + + + +jp + +受注処理 + + + +kr + +검증 + + + +jp->kr + + +次 + + + +he + +אימות + + + +kr->he + + + + + +ar + +اعتماد + + + +he->ar + + + + + +zh + +完成 + + + +ar->zh + + +סיום + + + diff --git a/diagrams/svg/graphviz_cjk_geometry.svg b/diagrams/svg/graphviz_cjk_geometry.svg new file mode 100644 index 0000000..d3bc19a --- /dev/null +++ b/diagrams/svg/graphviz_cjk_geometry.svg @@ -0,0 +1,48 @@ + + + + +graphviz_cjk + + + +受注処理 + + + +검증 + + + + +次 + + + +אימות + + + + + + + +اعتماد + + + + + + + +完成 + + + + +סיום + + + diff --git a/diagrams/svg/graphviz_clusters.svg b/diagrams/svg/graphviz_clusters.svg new file mode 100644 index 0000000..450270f --- /dev/null +++ b/diagrams/svg/graphviz_clusters.svg @@ -0,0 +1,77 @@ + + + + + + +graphviz_clusters + + +cluster_ingest + +Ingest + + +cluster_store + +Storage + + + +recv + +Receive + + + +parse + +Parse + + + +recv->parse + + + + + +index + +Index + + + +parse->index + + + + + +archive + +Archive + + + +parse->archive + + + + + +emit + +Emit + + + +index->emit + + + + + diff --git a/diagrams/svg/graphviz_clusters_geometry.svg b/diagrams/svg/graphviz_clusters_geometry.svg new file mode 100644 index 0000000..33576a2 --- /dev/null +++ b/diagrams/svg/graphviz_clusters_geometry.svg @@ -0,0 +1,54 @@ + + + + +graphviz_clusters + + + +Ingest + + + +Storage + + + +Receive + + + +Parse + + + + + + + +Index + + + + + + + +Archive + + + + + + + +Emit + + + + + + + diff --git a/diagrams/svg/graphviz_flow.svg b/diagrams/svg/graphviz_flow.svg new file mode 100644 index 0000000..aea1e67 --- /dev/null +++ b/diagrams/svg/graphviz_flow.svg @@ -0,0 +1,63 @@ + + + + + + +graphviz_flow + + + +a + +Start + + + +b + +Validate + + + +a->b + + + + + +c + +Process + + + +b->c + + +ok + + + +d + +Done + + + +b->d + + +skip + + + +c->d + + + + + diff --git a/diagrams/svg/graphviz_flow_geometry.svg b/diagrams/svg/graphviz_flow_geometry.svg new file mode 100644 index 0000000..ab579db --- /dev/null +++ b/diagrams/svg/graphviz_flow_geometry.svg @@ -0,0 +1,44 @@ + + + + +graphviz_flow + + + +Start + + + +Validate + + + + + + + +Process + + + + +ok + + + +Done + + + + +skip + + + + + + + diff --git a/diagrams/svg/graphviz_large.svg b/diagrams/svg/graphviz_large.svg new file mode 100644 index 0000000..cb4c49a --- /dev/null +++ b/diagrams/svg/graphviz_large.svg @@ -0,0 +1,1627 @@ + + + + + + +graphviz_large + + + +n0 + +N0 + + + +n1 + +N1 + + + +n0->n1 + + + + + +n2 + +N2 + + + +n0->n2 + + + + + +n16 + +N16 + + + +n0->n16 + + + + + +n3 + +N3 + + + +n1->n3 + + + + + +n4 + +N4 + + + +n1->n4 + + + + + +n5 + +N5 + + + +n2->n5 + + + + + +n6 + +N6 + + + +n2->n6 + + + + + +n7 + +N7 + + + +n3->n7 + + + + + +n8 + +N8 + + + +n3->n8 + + + + + +n9 + +N9 + + + +n4->n9 + + + + + +n10 + +N10 + + + +n4->n10 + + + + + +n11 + +N11 + + + +n5->n11 + + + + + +n12 + +N12 + + + +n5->n12 + + + + + +n13 + +N13 + + + +n6->n13 + + + + + +n14 + +N14 + + + +n6->n14 + + + + + +n15 + +N15 + + + +n7->n15 + + + + + +n7->n16 + + + + + +n17 + +N17 + + + +n8->n17 + + + + + +n18 + +N18 + + + +n8->n18 + + + + + +n24 + +N24 + + + +n8->n24 + + + + + +n19 + +N19 + + + +n9->n19 + + + + + +n20 + +N20 + + + +n9->n20 + + + + + +n21 + +N21 + + + +n10->n21 + + + + + +n22 + +N22 + + + +n10->n22 + + + + + +n23 + +N23 + + + +n11->n23 + + + + + +n11->n24 + + + + + +n25 + +N25 + + + +n12->n25 + + + + + +n26 + +N26 + + + +n12->n26 + + + + + +n27 + +N27 + + + +n13->n27 + + + + + +n28 + +N28 + + + +n13->n28 + + + + + +n29 + +N29 + + + +n14->n29 + + + + + +n30 + +N30 + + + +n14->n30 + + + + + +n31 + +N31 + + + +n15->n31 + + + + + +n32 + +N32 + + + +n15->n32 + + + + + +n16->n32 + + + + + +n33 + +N33 + + + +n16->n33 + + + + + +n34 + +N34 + + + +n16->n34 + + + + + +n35 + +N35 + + + +n17->n35 + + + + + +n36 + +N36 + + + +n17->n36 + + + + + +n37 + +N37 + + + +n18->n37 + + + + + +n38 + +N38 + + + +n18->n38 + + + + + +n39 + +N39 + + + +n19->n39 + + + + + +n40 + +N40 + + + +n19->n40 + + + + + +n41 + +N41 + + + +n20->n41 + + + + + +n42 + +N42 + + + +n20->n42 + + + + + +n43 + +N43 + + + +n21->n43 + + + + + +n44 + +N44 + + + +n21->n44 + + + + + +n45 + +N45 + + + +n22->n45 + + + + + +n46 + +N46 + + + +n22->n46 + + + + + +n47 + +N47 + + + +n23->n47 + + + + + +n48 + +N48 + + + +n23->n48 + + + + + +n24->n40 + + + + + +n49 + +N49 + + + +n24->n49 + + + + + +n50 + +N50 + + + +n24->n50 + + + + + +n51 + +N51 + + + +n25->n51 + + + + + +n52 + +N52 + + + +n25->n52 + + + + + +n53 + +N53 + + + +n26->n53 + + + + + +n54 + +N54 + + + +n26->n54 + + + + + +n55 + +N55 + + + +n27->n55 + + + + + +n56 + +N56 + + + +n27->n56 + + + + + +n57 + +N57 + + + +n28->n57 + + + + + +n58 + +N58 + + + +n28->n58 + + + + + +n59 + +N59 + + + +n29->n59 + + + + + +n60 + +N60 + + + +n29->n60 + + + + + +n61 + +N61 + + + +n30->n61 + + + + + +n62 + +N62 + + + +n30->n62 + + + + + +n63 + +N63 + + + +n31->n63 + + + + + +n64 + +N64 + + + +n31->n64 + + + + + +n32->n48 + + + + + +n65 + +N65 + + + +n32->n65 + + + + + +n66 + +N66 + + + +n32->n66 + + + + + +n67 + +N67 + + + +n33->n67 + + + + + +n68 + +N68 + + + +n33->n68 + + + + + +n69 + +N69 + + + +n34->n69 + + + + + +n70 + +N70 + + + +n34->n70 + + + + + +n71 + +N71 + + + +n35->n71 + + + + + +n72 + +N72 + + + +n35->n72 + + + + + +n73 + +N73 + + + +n36->n73 + + + + + +n74 + +N74 + + + +n36->n74 + + + + + +n75 + +N75 + + + +n37->n75 + + + + + +n76 + +N76 + + + +n37->n76 + + + + + +n77 + +N77 + + + +n38->n77 + + + + + +n78 + +N78 + + + +n38->n78 + + + + + +n79 + +N79 + + + +n39->n79 + + + + + +n80 + +N80 + + + +n39->n80 + + + + + +n40->n56 + + + + + +n81 + +N81 + + + +n40->n81 + + + + + +n82 + +N82 + + + +n40->n82 + + + + + +n83 + +N83 + + + +n41->n83 + + + + + +n84 + +N84 + + + +n41->n84 + + + + + +n85 + +N85 + + + +n42->n85 + + + + + +n86 + +N86 + + + +n42->n86 + + + + + +n87 + +N87 + + + +n43->n87 + + + + + +n88 + +N88 + + + +n43->n88 + + + + + +n89 + +N89 + + + +n44->n89 + + + + + +n90 + +N90 + + + +n44->n90 + + + + + +n91 + +N91 + + + +n45->n91 + + + + + +n92 + +N92 + + + +n45->n92 + + + + + +n93 + +N93 + + + +n46->n93 + + + + + +n94 + +N94 + + + +n46->n94 + + + + + +n95 + +N95 + + + +n47->n95 + + + + + +n96 + +N96 + + + +n47->n96 + + + + + +n48->n64 + + + + + +n97 + +N97 + + + +n48->n97 + + + + + +n98 + +N98 + + + +n48->n98 + + + + + +n99 + +N99 + + + +n49->n99 + + + + + +n100 + +N100 + + + +n49->n100 + + + + + +n101 + +N101 + + + +n50->n101 + + + + + +n102 + +N102 + + + +n50->n102 + + + + + +n103 + +N103 + + + +n51->n103 + + + + + +n104 + +N104 + + + +n51->n104 + + + + + +n105 + +N105 + + + +n52->n105 + + + + + +n106 + +N106 + + + +n52->n106 + + + + + +n107 + +N107 + + + +n53->n107 + + + + + +n108 + +N108 + + + +n53->n108 + + + + + +n109 + +N109 + + + +n54->n109 + + + + + +n110 + +N110 + + + +n54->n110 + + + + + +n111 + +N111 + + + +n55->n111 + + + + + +n112 + +N112 + + + +n55->n112 + + + + + +n56->n72 + + + + + +n113 + +N113 + + + +n56->n113 + + + + + +n114 + +N114 + + + +n56->n114 + + + + + +n115 + +N115 + + + +n57->n115 + + + + + +n116 + +N116 + + + +n57->n116 + + + + + +n117 + +N117 + + + +n58->n117 + + + + + +n118 + +N118 + + + +n58->n118 + + + + + +n119 + +N119 + + + +n59->n119 + + + + + +n120 + +N120 + + + +n59->n120 + + + + + +n121 + +N121 + + + +n60->n121 + + + + + +n122 + +N122 + + + +n60->n122 + + + + + +n123 + +N123 + + + +n61->n123 + + + + + +n124 + +N124 + + + +n61->n124 + + + + + +n125 + +N125 + + + +n62->n125 + + + + + +n126 + +N126 + + + +n62->n126 + + + + + +n127 + +N127 + + + +n63->n127 + + + + + +n64->n80 + + + + + +n72->n88 + + + + + +n80->n96 + + + + + +n88->n104 + + + + + +n96->n112 + + + + + +n104->n120 + + + + + diff --git a/diagrams/svg/graphviz_large_geometry.svg b/diagrams/svg/graphviz_large_geometry.svg new file mode 100644 index 0000000..3ecb115 --- /dev/null +++ b/diagrams/svg/graphviz_large_geometry.svg @@ -0,0 +1,1086 @@ + + + + +graphviz_large + + + +N0 + + + +N1 + + + + + + + +N2 + + + + + + + +N16 + + + + + + + +N3 + + + + + + + +N4 + + + + + + + +N5 + + + + + + + +N6 + + + + + + + +N7 + + + + + + + +N8 + + + + + + + +N9 + + + + + + + +N10 + + + + + + + +N11 + + + + + + + +N12 + + + + + + + +N13 + + + + + + + +N14 + + + + + + + +N15 + + + + + + + + + + + +N17 + + + + + + + +N18 + + + + + + + +N24 + + + + + + + +N19 + + + + + + + +N20 + + + + + + + +N21 + + + + + + + +N22 + + + + + + + +N23 + + + + + + + + + + + +N25 + + + + + + + +N26 + + + + + + + +N27 + + + + + + + +N28 + + + + + + + +N29 + + + + + + + +N30 + + + + + + + +N31 + + + + + + + +N32 + + + + + + + + + + + +N33 + + + + + + + +N34 + + + + + + + +N35 + + + + + + + +N36 + + + + + + + +N37 + + + + + + + +N38 + + + + + + + +N39 + + + + + + + +N40 + + + + + + + +N41 + + + + + + + +N42 + + + + + + + +N43 + + + + + + + +N44 + + + + + + + +N45 + + + + + + + +N46 + + + + + + + +N47 + + + + + + + +N48 + + + + + + + + + + + +N49 + + + + + + + +N50 + + + + + + + +N51 + + + + + + + +N52 + + + + + + + +N53 + + + + + + + +N54 + + + + + + + +N55 + + + + + + + +N56 + + + + + + + +N57 + + + + + + + +N58 + + + + + + + +N59 + + + + + + + +N60 + + + + + + + +N61 + + + + + + + +N62 + + + + + + + +N63 + + + + + + + +N64 + + + + + + + + + + + +N65 + + + + + + + +N66 + + + + + + + +N67 + + + + + + + +N68 + + + + + + + +N69 + + + + + + + +N70 + + + + + + + +N71 + + + + + + + +N72 + + + + + + + +N73 + + + + + + + +N74 + + + + + + + +N75 + + + + + + + +N76 + + + + + + + +N77 + + + + + + + +N78 + + + + + + + +N79 + + + + + + + +N80 + + + + + + + + + + + +N81 + + + + + + + +N82 + + + + + + + +N83 + + + + + + + +N84 + + + + + + + +N85 + + + + + + + +N86 + + + + + + + +N87 + + + + + + + +N88 + + + + + + + +N89 + + + + + + + +N90 + + + + + + + +N91 + + + + + + + +N92 + + + + + + + +N93 + + + + + + + +N94 + + + + + + + +N95 + + + + + + + +N96 + + + + + + + + + + + +N97 + + + + + + + +N98 + + + + + + + +N99 + + + + + + + +N100 + + + + + + + +N101 + + + + + + + +N102 + + + + + + + +N103 + + + + + + + +N104 + + + + + + + +N105 + + + + + + + +N106 + + + + + + + +N107 + + + + + + + +N108 + + + + + + + +N109 + + + + + + + +N110 + + + + + + + +N111 + + + + + + + +N112 + + + + + + + + + + + +N113 + + + + + + + +N114 + + + + + + + +N115 + + + + + + + +N116 + + + + + + + +N117 + + + + + + + +N118 + + + + + + + +N119 + + + + + + + +N120 + + + + + + + +N121 + + + + + + + +N122 + + + + + + + +N123 + + + + + + + +N124 + + + + + + + +N125 + + + + + + + +N126 + + + + + + + +N127 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/diagrams/svg/graphviz_network.svg b/diagrams/svg/graphviz_network.svg new file mode 100644 index 0000000..4807a3d --- /dev/null +++ b/diagrams/svg/graphviz_network.svg @@ -0,0 +1,63 @@ + + + + + + +graphviz_network + + + +gw + +Gateway + + + +s1 + +Switch A + + + +gw--s1 + + + + +s2 + +Switch B + + + +gw--s2 + + + + +h1 + +Host 1 + + + +s1--h1 + + + + +h2 + +Host 2 + + + +s2--h2 + + + + diff --git a/diagrams/svg/graphviz_network_geometry.svg b/diagrams/svg/graphviz_network_geometry.svg new file mode 100644 index 0000000..b3876ae --- /dev/null +++ b/diagrams/svg/graphviz_network_geometry.svg @@ -0,0 +1,42 @@ + + + + +graphviz_network + + + +Gateway + + + +Switch A + + + + + + +Switch B + + + + + + +Host 1 + + + + + + +Host 2 + + + + + + diff --git a/diagrams/svg/graphviz_ortho.svg b/diagrams/svg/graphviz_ortho.svg new file mode 100644 index 0000000..520e099 --- /dev/null +++ b/diagrams/svg/graphviz_ortho.svg @@ -0,0 +1,73 @@ + + + + + + +graphviz_ortho + + + +client + +Client + + + +lb + +Balancer + + + +client->lb + + + + + +web1 + +Web 1 + + + +lb->web1 + + + + + +web2 + +Web 2 + + + +lb->web2 + + + + + +db + +Database + + + +web1->db + + + + + +web2->db + + + + + diff --git a/diagrams/svg/graphviz_ortho_geometry.svg b/diagrams/svg/graphviz_ortho_geometry.svg new file mode 100644 index 0000000..0ae4a6c --- /dev/null +++ b/diagrams/svg/graphviz_ortho_geometry.svg @@ -0,0 +1,50 @@ + + + + +graphviz_ortho + + + +Client + + + +Balancer + + + + + + + +Web 1 + + + + + + + +Web 2 + + + + + + + +Database + + + + + + + + + + + diff --git a/diagrams/svg/graphviz_record.svg b/diagrams/svg/graphviz_record.svg new file mode 100644 index 0000000..fa6b143 --- /dev/null +++ b/diagrams/svg/graphviz_record.svg @@ -0,0 +1,57 @@ + + + + + + +graphviz_record + + + +customer + +Customer + +id + +name + + + +order + +Order + +id + +customer_id + +total + + + +customer:f0->order:f1 + + + + + +item + +Item + +order_id + +sku + + + +order:f0->item:f0 + + + + + diff --git a/diagrams/svg/graphviz_record_geometry.svg b/diagrams/svg/graphviz_record_geometry.svg new file mode 100644 index 0000000..4364344 --- /dev/null +++ b/diagrams/svg/graphviz_record_geometry.svg @@ -0,0 +1,44 @@ + + + + +graphviz_record + + + +Customer + +id + +name + + + +Order + +id + +customer_id + +total + + + + + + + +Item + +order_id + +sku + + + + + + + diff --git a/diagrams/svg/graphviz_selfloop.svg b/diagrams/svg/graphviz_selfloop.svg new file mode 100644 index 0000000..26ec08f --- /dev/null +++ b/diagrams/svg/graphviz_selfloop.svg @@ -0,0 +1,68 @@ + + + + + + +graphviz_selfloop + + + +retry + +Retry + + + +retry->retry + + +backoff + + + +work + +Work + + + +retry->work + + + + + +right + +Right + + + +retry->right + + + + + +work->work + + + + + +left + +Left + + + +left->work + + + + + diff --git a/diagrams/svg/graphviz_selfloop_geometry.svg b/diagrams/svg/graphviz_selfloop_geometry.svg new file mode 100644 index 0000000..025b4c5 --- /dev/null +++ b/diagrams/svg/graphviz_selfloop_geometry.svg @@ -0,0 +1,47 @@ + + + + +graphviz_selfloop + + + +Retry + + + + +backoff + + + +Work + + + + + + + +Right + + + + + + + + + + + +Left + + + + + + + diff --git a/diagrams/svg/graphviz_states.svg b/diagrams/svg/graphviz_states.svg new file mode 100644 index 0000000..9146004 --- /dev/null +++ b/diagrams/svg/graphviz_states.svg @@ -0,0 +1,66 @@ + + + + + + +graphviz_states + + + +idle + +idle + + + +running + +running + + + +idle->running + + +start + + + +paused + +paused + + + +running->paused + + +pause + + + +done + + +done + + + +running->done + + +finish + + + +paused->running + + +resume + + + diff --git a/diagrams/svg/graphviz_states_geometry.svg b/diagrams/svg/graphviz_states_geometry.svg new file mode 100644 index 0000000..6f18e2b --- /dev/null +++ b/diagrams/svg/graphviz_states_geometry.svg @@ -0,0 +1,47 @@ + + + + +graphviz_states + + + +idle + + + +running + + + + +start + + + +paused + + + + +pause + + + + +done + + + + +finish + + + + +resume + + + diff --git a/diagrams/svg/icon_nodes.svg b/diagrams/svg/icon_nodes.svg new file mode 100644 index 0000000..07660a7 --- /dev/null +++ b/diagrams/svg/icon_nodes.svg @@ -0,0 +1,49 @@ + + + + Ingest pipeline + + + + + + + + + + + + + + Browser + + + + + + Queue + + + + + + + Worker + + + + + + + Bucket + + + + + + + diff --git a/diagrams/svg/libreoffice_connectors.svg b/diagrams/svg/libreoffice_connectors.svg new file mode 100644 index 0000000..4fdc55a --- /dev/null +++ b/diagrams/svg/libreoffice_connectors.svg @@ -0,0 +1,142 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Intake + + + + + + + + Review + + + + + + + + Approve + + + + + + + + Publish + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/diagrams/svg/mermaid_flow.svg b/diagrams/svg/mermaid_flow.svg new file mode 100644 index 0000000..2891538 --- /dev/null +++ b/diagrams/svg/mermaid_flow.svg @@ -0,0 +1 @@ +

yes

no

Receive request

Authenticated?

Check quota

Return 401

Run handler

Return 200

 \ No newline at end of file diff --git a/diagrams/svg/mermaid_flow_geometry.svg b/diagrams/svg/mermaid_flow_geometry.svg new file mode 100644 index 0000000..35cc19e --- /dev/null +++ b/diagrams/svg/mermaid_flow_geometry.svg @@ -0,0 +1 @@ +

yes

no

Receive request

Authenticated?

Check quota

Return 401

Run handler

Return 200

 \ No newline at end of file diff --git a/diagrams/svg/mixed_page.svg b/diagrams/svg/mixed_page.svg new file mode 100644 index 0000000..e326397 --- /dev/null +++ b/diagrams/svg/mixed_page.svg @@ -0,0 +1,71 @@ + + + + Release checklist + + + + + + + + Release checklist + + Every release goes through the same three stages. The table below records + who signs off on each one, and the diagram at the foot of the page shows the + order they run in. + + + + + + + + + + + + + Stage + Owner + Deadline + + Build + Release engineer + Monday + Sign-off + Product lead + Wednesday + Ship + On-call + Thursday + + + + A stage may only start once the one before it has been signed off: + + + + + + + + + Build + Sign-off + Ship + + + + + + + + Figure 1. Release stages, in order. + + diff --git a/diagrams/svg/negative_form.svg b/diagrams/svg/negative_form.svg new file mode 100644 index 0000000..7f5e3d6 --- /dev/null +++ b/diagrams/svg/negative_form.svg @@ -0,0 +1,40 @@ + + + + Access request form + + + Access request + + + + Full name + Team + Manager + Systems requested + Signature + + + + + + + + + + + + + + + + + Mail + Repository + Dashboards + Billing + + diff --git a/diagrams/svg/negative_pie_chart.svg b/diagrams/svg/negative_pie_chart.svg new file mode 100644 index 0000000..2377388 --- /dev/null +++ b/diagrams/svg/negative_pie_chart.svg @@ -0,0 +1,41 @@ + + + + Storage used by file type + + Storage used by file type + + + + + + + + + + + + + + + + + + + + + + + + PDF — 34% + + Everything else — 16% + + Images — 28% + + Office — 22% + + diff --git a/diagrams/svg/negative_ruled_table.svg b/diagrams/svg/negative_ruled_table.svg new file mode 100644 index 0000000..c3a6e87 --- /dev/null +++ b/diagrams/svg/negative_ruled_table.svg @@ -0,0 +1,50 @@ + + + + Quarterly revenue by region + + Quarterly revenue by region + + + + + + + + + + + + + + + Region + Q1 + Q2 + Q3 + + North + 1,204 + 1,388 + 1,502 + + South + 942 + 1,011 + 1,140 + + East + 1,655 + 1,702 + 1,690 + + West + 803 + 877 + 934 + + diff --git a/diagrams/svg/nested_transforms.svg b/diagrams/svg/nested_transforms.svg new file mode 100644 index 0000000..31a01e3 --- /dev/null +++ b/diagrams/svg/nested_transforms.svg @@ -0,0 +1,43 @@ + + + + Nested Transforms + Three-stage pipeline drawn inside translated, scaled and rotated groups + + + + + + + Ingest + + + + + Transform + + + + + Publish + + + + + + + + Quarantine + + + + on error + + + + diff --git a/diagrams/svg/plantuml_activity.svg b/diagrams/svg/plantuml_activity.svg new file mode 100644 index 0000000..d795753 --- /dev/null +++ b/diagrams/svg/plantuml_activity.svg @@ -0,0 +1 @@ +Read configOpen inputDetect formatExtract textRender markdown \ No newline at end of file diff --git a/diagrams/svg/plantuml_activity_geometry.svg b/diagrams/svg/plantuml_activity_geometry.svg new file mode 100644 index 0000000..b7667d2 --- /dev/null +++ b/diagrams/svg/plantuml_activity_geometry.svg @@ -0,0 +1 @@ +Read configOpen inputDetect formatExtract textRender markdown \ No newline at end of file diff --git a/diagrams/svg/plantuml_swimlane.svg b/diagrams/svg/plantuml_swimlane.svg new file mode 100644 index 0000000..7d295a0 --- /dev/null +++ b/diagrams/svg/plantuml_swimlane.svg @@ -0,0 +1 @@ +Draft articlePublishReview draftSchedule slotClear rightsAuthorEditorLegal \ No newline at end of file diff --git a/ground_truth/dot/5_level_paging_and_5_level_ept_intel_revision_1_1_may_2017.dot b/ground_truth/dot/5_level_paging_and_5_level_ept_intel_revision_1_1_may_2017.dot new file mode 100644 index 0000000..e69de29 diff --git a/ground_truth/dot/data_dashboard.dot b/ground_truth/dot/data_dashboard.dot new file mode 100644 index 0000000..e69de29 diff --git a/ground_truth/dot/flowchart.dot b/ground_truth/dot/flowchart.dot new file mode 100644 index 0000000..8a8f8b4 --- /dev/null +++ b/ground_truth/dot/flowchart.dot @@ -0,0 +1,9 @@ +digraph flowchart { + "Requirements" [shape=box fillcolor="#4a90d9"]; + "Design" [shape=box fillcolor="#50b848"]; + "Implementation" [shape=box fillcolor="#f5a623"]; + "Testing" [shape=box fillcolor="#d0021b"]; + "Requirements" -> "Design"; + "Design" -> "Implementation"; + "Implementation" -> "Testing"; +} diff --git a/ground_truth/dot/graphviz_bidirectional.dot b/ground_truth/dot/graphviz_bidirectional.dot new file mode 100644 index 0000000..cb95d4b --- /dev/null +++ b/ground_truth/dot/graphviz_bidirectional.dot @@ -0,0 +1,8 @@ +digraph graphviz_bidirectional { + "API" [shape=box fillcolor="#fdb462"]; + "Database" [shape=box fillcolor="#fdb462"]; + "Cache" [shape=box fillcolor="#fdb462"]; + "API" -> "Database" [dir=both]; + "API" -> "Cache"; + "Database" -> "Cache"; +} diff --git a/ground_truth/dot/graphviz_cjk.dot b/ground_truth/dot/graphviz_cjk.dot new file mode 100644 index 0000000..a2766b8 --- /dev/null +++ b/ground_truth/dot/graphviz_cjk.dot @@ -0,0 +1,13 @@ +digraph graphviz_cjk { + // Stroke-only nodes: no fill anywhere in the drawing. Labels are Japanese, Korean, + // Hebrew and Arabic, so nothing here survives a Latin-1 round trip. + "受注処理" [shape=box]; + "검증" [shape=box]; + "אימות" [shape=box]; + "اعتماد" [shape=box]; + "完成" [shape=box]; + "受注処理" -> "검증" [label="次"]; + "검증" -> "אימות"; + "אימות" -> "اعتماد"; + "اعتماد" -> "完成" [label="סיום"]; +} diff --git a/ground_truth/dot/graphviz_clusters.dot b/ground_truth/dot/graphviz_clusters.dot new file mode 100644 index 0000000..4bc88a0 --- /dev/null +++ b/ground_truth/dot/graphviz_clusters.dot @@ -0,0 +1,13 @@ +digraph graphviz_clusters { + // "Ingest" and "Storage" are cluster containers, not nodes, and are absent here + // on purpose: reporting either of them is a false positive. + "Receive" [shape=box fillcolor="#a6cee3"]; + "Parse" [shape=box fillcolor="#a6cee3"]; + "Index" [shape=box fillcolor="#fdbf6f"]; + "Archive" [shape=box fillcolor="#fdbf6f"]; + "Emit" [shape=ellipse fillcolor="#b2df8a"]; + "Receive" -> "Parse"; + "Parse" -> "Index"; + "Parse" -> "Archive"; + "Index" -> "Emit"; +} diff --git a/ground_truth/dot/graphviz_flow.dot b/ground_truth/dot/graphviz_flow.dot new file mode 100644 index 0000000..520333f --- /dev/null +++ b/ground_truth/dot/graphviz_flow.dot @@ -0,0 +1,10 @@ +digraph graphviz_flow { + "Start" [shape=box fillcolor="#a6cee3"]; + "Validate" [shape=diamond fillcolor="#b2df8a"]; + "Process" [shape=box fillcolor="#fb9a99"]; + "Done" [shape=ellipse fillcolor="#fdbf6f"]; + "Start" -> "Validate"; + "Validate" -> "Process" [label="ok"]; + "Validate" -> "Done" [label="skip" style=dashed]; + "Process" -> "Done"; +} diff --git a/ground_truth/dot/graphviz_large.dot b/ground_truth/dot/graphviz_large.dot new file mode 100644 index 0000000..b0165ac --- /dev/null +++ b/ground_truth/dot/graphviz_large.dot @@ -0,0 +1,275 @@ +digraph graphviz_large { + // 128 nodes, 141 edges. Generated from the same rule as diagrams/src/graphviz_large.dot: + // a complete binary tree over N0..N127, plus a dashed shortcut from N_i to the node + // sixteen along for every i divisible by 8. Keyed by label, which for this fixture is + // the node id with the prefix capitalised. + "N0" [shape=box fillcolor="#d9d9d9"]; + "N1" [shape=box fillcolor="#d9d9d9"]; + "N2" [shape=box fillcolor="#d9d9d9"]; + "N3" [shape=box fillcolor="#d9d9d9"]; + "N4" [shape=box fillcolor="#d9d9d9"]; + "N5" [shape=box fillcolor="#d9d9d9"]; + "N6" [shape=box fillcolor="#d9d9d9"]; + "N7" [shape=box fillcolor="#d9d9d9"]; + "N8" [shape=box fillcolor="#d9d9d9"]; + "N9" [shape=box fillcolor="#d9d9d9"]; + "N10" [shape=box fillcolor="#d9d9d9"]; + "N11" [shape=box fillcolor="#d9d9d9"]; + "N12" [shape=box fillcolor="#d9d9d9"]; + "N13" [shape=box fillcolor="#d9d9d9"]; + "N14" [shape=box fillcolor="#d9d9d9"]; + "N15" [shape=box fillcolor="#d9d9d9"]; + "N16" [shape=box fillcolor="#d9d9d9"]; + "N17" [shape=box fillcolor="#d9d9d9"]; + "N18" [shape=box fillcolor="#d9d9d9"]; + "N19" [shape=box fillcolor="#d9d9d9"]; + "N20" [shape=box fillcolor="#d9d9d9"]; + "N21" [shape=box fillcolor="#d9d9d9"]; + "N22" [shape=box fillcolor="#d9d9d9"]; + "N23" [shape=box fillcolor="#d9d9d9"]; + "N24" [shape=box fillcolor="#d9d9d9"]; + "N25" [shape=box fillcolor="#d9d9d9"]; + "N26" [shape=box fillcolor="#d9d9d9"]; + "N27" [shape=box fillcolor="#d9d9d9"]; + "N28" [shape=box fillcolor="#d9d9d9"]; + "N29" [shape=box fillcolor="#d9d9d9"]; + "N30" [shape=box fillcolor="#d9d9d9"]; + "N31" [shape=box fillcolor="#d9d9d9"]; + "N32" [shape=box fillcolor="#d9d9d9"]; + "N33" [shape=box fillcolor="#d9d9d9"]; + "N34" [shape=box fillcolor="#d9d9d9"]; + "N35" [shape=box fillcolor="#d9d9d9"]; + "N36" [shape=box fillcolor="#d9d9d9"]; + "N37" [shape=box fillcolor="#d9d9d9"]; + "N38" [shape=box fillcolor="#d9d9d9"]; + "N39" [shape=box fillcolor="#d9d9d9"]; + "N40" [shape=box fillcolor="#d9d9d9"]; + "N41" [shape=box fillcolor="#d9d9d9"]; + "N42" [shape=box fillcolor="#d9d9d9"]; + "N43" [shape=box fillcolor="#d9d9d9"]; + "N44" [shape=box fillcolor="#d9d9d9"]; + "N45" [shape=box fillcolor="#d9d9d9"]; + "N46" [shape=box fillcolor="#d9d9d9"]; + "N47" [shape=box fillcolor="#d9d9d9"]; + "N48" [shape=box fillcolor="#d9d9d9"]; + "N49" [shape=box fillcolor="#d9d9d9"]; + "N50" [shape=box fillcolor="#d9d9d9"]; + "N51" [shape=box fillcolor="#d9d9d9"]; + "N52" [shape=box fillcolor="#d9d9d9"]; + "N53" [shape=box fillcolor="#d9d9d9"]; + "N54" [shape=box fillcolor="#d9d9d9"]; + "N55" [shape=box fillcolor="#d9d9d9"]; + "N56" [shape=box fillcolor="#d9d9d9"]; + "N57" [shape=box fillcolor="#d9d9d9"]; + "N58" [shape=box fillcolor="#d9d9d9"]; + "N59" [shape=box fillcolor="#d9d9d9"]; + "N60" [shape=box fillcolor="#d9d9d9"]; + "N61" [shape=box fillcolor="#d9d9d9"]; + "N62" [shape=box fillcolor="#d9d9d9"]; + "N63" [shape=box fillcolor="#d9d9d9"]; + "N64" [shape=box fillcolor="#d9d9d9"]; + "N65" [shape=box fillcolor="#d9d9d9"]; + "N66" [shape=box fillcolor="#d9d9d9"]; + "N67" [shape=box fillcolor="#d9d9d9"]; + "N68" [shape=box fillcolor="#d9d9d9"]; + "N69" [shape=box fillcolor="#d9d9d9"]; + "N70" [shape=box fillcolor="#d9d9d9"]; + "N71" [shape=box fillcolor="#d9d9d9"]; + "N72" [shape=box fillcolor="#d9d9d9"]; + "N73" [shape=box fillcolor="#d9d9d9"]; + "N74" [shape=box fillcolor="#d9d9d9"]; + "N75" [shape=box fillcolor="#d9d9d9"]; + "N76" [shape=box fillcolor="#d9d9d9"]; + "N77" [shape=box fillcolor="#d9d9d9"]; + "N78" [shape=box fillcolor="#d9d9d9"]; + "N79" [shape=box fillcolor="#d9d9d9"]; + "N80" [shape=box fillcolor="#d9d9d9"]; + "N81" [shape=box fillcolor="#d9d9d9"]; + "N82" [shape=box fillcolor="#d9d9d9"]; + "N83" [shape=box fillcolor="#d9d9d9"]; + "N84" [shape=box fillcolor="#d9d9d9"]; + "N85" [shape=box fillcolor="#d9d9d9"]; + "N86" [shape=box fillcolor="#d9d9d9"]; + "N87" [shape=box fillcolor="#d9d9d9"]; + "N88" [shape=box fillcolor="#d9d9d9"]; + "N89" [shape=box fillcolor="#d9d9d9"]; + "N90" [shape=box fillcolor="#d9d9d9"]; + "N91" [shape=box fillcolor="#d9d9d9"]; + "N92" [shape=box fillcolor="#d9d9d9"]; + "N93" [shape=box fillcolor="#d9d9d9"]; + "N94" [shape=box fillcolor="#d9d9d9"]; + "N95" [shape=box fillcolor="#d9d9d9"]; + "N96" [shape=box fillcolor="#d9d9d9"]; + "N97" [shape=box fillcolor="#d9d9d9"]; + "N98" [shape=box fillcolor="#d9d9d9"]; + "N99" [shape=box fillcolor="#d9d9d9"]; + "N100" [shape=box fillcolor="#d9d9d9"]; + "N101" [shape=box fillcolor="#d9d9d9"]; + "N102" [shape=box fillcolor="#d9d9d9"]; + "N103" [shape=box fillcolor="#d9d9d9"]; + "N104" [shape=box fillcolor="#d9d9d9"]; + "N105" [shape=box fillcolor="#d9d9d9"]; + "N106" [shape=box fillcolor="#d9d9d9"]; + "N107" [shape=box fillcolor="#d9d9d9"]; + "N108" [shape=box fillcolor="#d9d9d9"]; + "N109" [shape=box fillcolor="#d9d9d9"]; + "N110" [shape=box fillcolor="#d9d9d9"]; + "N111" [shape=box fillcolor="#d9d9d9"]; + "N112" [shape=box fillcolor="#d9d9d9"]; + "N113" [shape=box fillcolor="#d9d9d9"]; + "N114" [shape=box fillcolor="#d9d9d9"]; + "N115" [shape=box fillcolor="#d9d9d9"]; + "N116" [shape=box fillcolor="#d9d9d9"]; + "N117" [shape=box fillcolor="#d9d9d9"]; + "N118" [shape=box fillcolor="#d9d9d9"]; + "N119" [shape=box fillcolor="#d9d9d9"]; + "N120" [shape=box fillcolor="#d9d9d9"]; + "N121" [shape=box fillcolor="#d9d9d9"]; + "N122" [shape=box fillcolor="#d9d9d9"]; + "N123" [shape=box fillcolor="#d9d9d9"]; + "N124" [shape=box fillcolor="#d9d9d9"]; + "N125" [shape=box fillcolor="#d9d9d9"]; + "N126" [shape=box fillcolor="#d9d9d9"]; + "N127" [shape=box fillcolor="#d9d9d9"]; + "N0" -> "N1"; + "N0" -> "N2"; + "N1" -> "N3"; + "N1" -> "N4"; + "N2" -> "N5"; + "N2" -> "N6"; + "N3" -> "N7"; + "N3" -> "N8"; + "N4" -> "N9"; + "N4" -> "N10"; + "N5" -> "N11"; + "N5" -> "N12"; + "N6" -> "N13"; + "N6" -> "N14"; + "N7" -> "N15"; + "N7" -> "N16"; + "N8" -> "N17"; + "N8" -> "N18"; + "N9" -> "N19"; + "N9" -> "N20"; + "N10" -> "N21"; + "N10" -> "N22"; + "N11" -> "N23"; + "N11" -> "N24"; + "N12" -> "N25"; + "N12" -> "N26"; + "N13" -> "N27"; + "N13" -> "N28"; + "N14" -> "N29"; + "N14" -> "N30"; + "N15" -> "N31"; + "N15" -> "N32"; + "N16" -> "N33"; + "N16" -> "N34"; + "N17" -> "N35"; + "N17" -> "N36"; + "N18" -> "N37"; + "N18" -> "N38"; + "N19" -> "N39"; + "N19" -> "N40"; + "N20" -> "N41"; + "N20" -> "N42"; + "N21" -> "N43"; + "N21" -> "N44"; + "N22" -> "N45"; + "N22" -> "N46"; + "N23" -> "N47"; + "N23" -> "N48"; + "N24" -> "N49"; + "N24" -> "N50"; + "N25" -> "N51"; + "N25" -> "N52"; + "N26" -> "N53"; + "N26" -> "N54"; + "N27" -> "N55"; + "N27" -> "N56"; + "N28" -> "N57"; + "N28" -> "N58"; + "N29" -> "N59"; + "N29" -> "N60"; + "N30" -> "N61"; + "N30" -> "N62"; + "N31" -> "N63"; + "N31" -> "N64"; + "N32" -> "N65"; + "N32" -> "N66"; + "N33" -> "N67"; + "N33" -> "N68"; + "N34" -> "N69"; + "N34" -> "N70"; + "N35" -> "N71"; + "N35" -> "N72"; + "N36" -> "N73"; + "N36" -> "N74"; + "N37" -> "N75"; + "N37" -> "N76"; + "N38" -> "N77"; + "N38" -> "N78"; + "N39" -> "N79"; + "N39" -> "N80"; + "N40" -> "N81"; + "N40" -> "N82"; + "N41" -> "N83"; + "N41" -> "N84"; + "N42" -> "N85"; + "N42" -> "N86"; + "N43" -> "N87"; + "N43" -> "N88"; + "N44" -> "N89"; + "N44" -> "N90"; + "N45" -> "N91"; + "N45" -> "N92"; + "N46" -> "N93"; + "N46" -> "N94"; + "N47" -> "N95"; + "N47" -> "N96"; + "N48" -> "N97"; + "N48" -> "N98"; + "N49" -> "N99"; + "N49" -> "N100"; + "N50" -> "N101"; + "N50" -> "N102"; + "N51" -> "N103"; + "N51" -> "N104"; + "N52" -> "N105"; + "N52" -> "N106"; + "N53" -> "N107"; + "N53" -> "N108"; + "N54" -> "N109"; + "N54" -> "N110"; + "N55" -> "N111"; + "N55" -> "N112"; + "N56" -> "N113"; + "N56" -> "N114"; + "N57" -> "N115"; + "N57" -> "N116"; + "N58" -> "N117"; + "N58" -> "N118"; + "N59" -> "N119"; + "N59" -> "N120"; + "N60" -> "N121"; + "N60" -> "N122"; + "N61" -> "N123"; + "N61" -> "N124"; + "N62" -> "N125"; + "N62" -> "N126"; + "N63" -> "N127"; + "N0" -> "N16" [style=dashed]; + "N8" -> "N24" [style=dashed]; + "N16" -> "N32" [style=dashed]; + "N24" -> "N40" [style=dashed]; + "N32" -> "N48" [style=dashed]; + "N40" -> "N56" [style=dashed]; + "N48" -> "N64" [style=dashed]; + "N56" -> "N72" [style=dashed]; + "N64" -> "N80" [style=dashed]; + "N72" -> "N88" [style=dashed]; + "N80" -> "N96" [style=dashed]; + "N88" -> "N104" [style=dashed]; + "N96" -> "N112" [style=dashed]; + "N104" -> "N120" [style=dashed]; +} diff --git a/ground_truth/dot/graphviz_network.dot b/ground_truth/dot/graphviz_network.dot new file mode 100644 index 0000000..c073492 --- /dev/null +++ b/ground_truth/dot/graphviz_network.dot @@ -0,0 +1,11 @@ +graph graphviz_network { + "Gateway" [shape=ellipse fillcolor="#8dd3c7"]; + "Switch A" [shape=ellipse fillcolor="#ffffb3"]; + "Switch B" [shape=ellipse fillcolor="#ffffb3"]; + "Host 1" [shape=ellipse fillcolor="#cccccc"]; + "Host 2" [shape=ellipse fillcolor="#cccccc"]; + "Gateway" -- "Switch A"; + "Gateway" -- "Switch B"; + "Switch A" -- "Host 1"; + "Switch B" -- "Host 2"; +} diff --git a/ground_truth/dot/graphviz_ortho.dot b/ground_truth/dot/graphviz_ortho.dot new file mode 100644 index 0000000..fe2af78 --- /dev/null +++ b/ground_truth/dot/graphviz_ortho.dot @@ -0,0 +1,12 @@ +digraph graphviz_ortho { + "Client" [shape=box fillcolor="#ccebc5"]; + "Balancer" [shape=box fillcolor="#ccebc5"]; + "Web 1" [shape=box fillcolor="#ccebc5"]; + "Web 2" [shape=box fillcolor="#ccebc5"]; + "Database" [shape=box fillcolor="#ccebc5"]; + "Client" -> "Balancer"; + "Balancer" -> "Web 1"; + "Balancer" -> "Web 2"; + "Web 1" -> "Database"; + "Web 2" -> "Database"; +} diff --git a/ground_truth/dot/graphviz_record.dot b/ground_truth/dot/graphviz_record.dot new file mode 100644 index 0000000..7074ca4 --- /dev/null +++ b/ground_truth/dot/graphviz_record.dot @@ -0,0 +1,11 @@ +digraph graphviz_record { + // One record is one node. Its fields are separated by rules drawn inside the single + // outline, so the node key is the field texts as one multi-line label, the same + // convention org_chart.dot uses. Splitting a record into one node per field is the + // failure this fixture looks for. + "Customer\nid\nname" [shape=record fillcolor="#fbb4ae"]; + "Order\nid\ncustomer_id\ntotal" [shape=record fillcolor="#fbb4ae"]; + "Item\norder_id\nsku" [shape=record fillcolor="#fbb4ae"]; + "Customer\nid\nname" -> "Order\nid\ncustomer_id\ntotal"; + "Order\nid\ncustomer_id\ntotal" -> "Item\norder_id\nsku"; +} diff --git a/ground_truth/dot/graphviz_selfloop.dot b/ground_truth/dot/graphviz_selfloop.dot new file mode 100644 index 0000000..19d7f6a --- /dev/null +++ b/ground_truth/dot/graphviz_selfloop.dot @@ -0,0 +1,11 @@ +digraph graphviz_selfloop { + "Retry" [shape=circle fillcolor="#bebada"]; + "Work" [shape=circle fillcolor="#bebada"]; + "Left" [shape=circle fillcolor="#bebada"]; + "Right" [shape=circle fillcolor="#bebada"]; + "Retry" -> "Retry" [label="backoff"]; + "Work" -> "Work"; + "Retry" -> "Work"; + "Left" -> "Work"; + "Retry" -> "Right"; +} diff --git a/ground_truth/dot/graphviz_states.dot b/ground_truth/dot/graphviz_states.dot new file mode 100644 index 0000000..b68677b --- /dev/null +++ b/ground_truth/dot/graphviz_states.dot @@ -0,0 +1,10 @@ +digraph graphviz_states { + "idle" [shape=ellipse fillcolor="#b3de69"]; + "running" [shape=ellipse fillcolor="#b3de69"]; + "paused" [shape=ellipse fillcolor="#b3de69"]; + "done" [shape=ellipse fillcolor="#fb8072"]; + "idle" -> "running" [label="start"]; + "running" -> "paused" [label="pause"]; + "paused" -> "running" [label="resume"]; + "running" -> "done" [label="finish"]; +} diff --git a/ground_truth/dot/icon_nodes.dot b/ground_truth/dot/icon_nodes.dot new file mode 100644 index 0000000..1ee68b0 --- /dev/null +++ b/ground_truth/dot/icon_nodes.dot @@ -0,0 +1,11 @@ +digraph icon_nodes { + // Each node is an icon glyph with its caption underneath and no bounding outline, so the + // label sits outside every closed region in the drawing. + "Browser" [shape=none]; + "Queue" [shape=none]; + "Worker" [shape=none]; + "Bucket" [shape=none]; + "Browser" -> "Queue"; + "Queue" -> "Worker"; + "Worker" -> "Bucket"; +} diff --git a/ground_truth/dot/libreoffice_connectors.dot b/ground_truth/dot/libreoffice_connectors.dot new file mode 100644 index 0000000..0036752 --- /dev/null +++ b/ground_truth/dot/libreoffice_connectors.dot @@ -0,0 +1,9 @@ +digraph libreoffice_connectors { + "Intake" [shape=box]; + "Review" [shape=box]; + "Approve" [shape=box]; + "Publish" [shape=box]; + "Intake" -> "Review"; + "Review" -> "Approve"; + "Approve" -> "Publish"; +} diff --git a/ground_truth/dot/mermaid_flow.dot b/ground_truth/dot/mermaid_flow.dot new file mode 100644 index 0000000..921ae47 --- /dev/null +++ b/ground_truth/dot/mermaid_flow.dot @@ -0,0 +1,14 @@ +digraph mermaid_flow { + "Receive request" [shape=box]; + "Authenticated?" [shape=diamond]; + "Check quota" [shape=box]; + "Return 401" [shape=box]; + "Run handler" [shape=box]; + "Return 200" [shape=box]; + "Receive request" -> "Authenticated?"; + "Authenticated?" -> "Check quota" [label="yes"]; + "Authenticated?" -> "Return 401" [label="no"]; + "Check quota" -> "Run handler"; + "Run handler" -> "Return 200"; + "Return 401" -> "Return 200"; +} diff --git a/ground_truth/dot/mixed_page.dot b/ground_truth/dot/mixed_page.dot new file mode 100644 index 0000000..e86c622 --- /dev/null +++ b/ground_truth/dot/mixed_page.dot @@ -0,0 +1,9 @@ +digraph mixed_page { + // The page also carries a heading, two paragraphs and a four-row ruled table. None of that + // is here: the ground truth is the figure at the foot of the page and nothing else. + "Build" [shape=box]; + "Sign-off" [shape=box]; + "Ship" [shape=box]; + "Build" -> "Sign-off"; + "Sign-off" -> "Ship"; +} diff --git a/ground_truth/dot/negative_form.dot b/ground_truth/dot/negative_form.dot new file mode 100644 index 0000000..e69de29 diff --git a/ground_truth/dot/negative_pie_chart.dot b/ground_truth/dot/negative_pie_chart.dot new file mode 100644 index 0000000..e69de29 diff --git a/ground_truth/dot/negative_ruled_table.dot b/ground_truth/dot/negative_ruled_table.dot new file mode 100644 index 0000000..e69de29 diff --git a/ground_truth/dot/nested_transforms.dot b/ground_truth/dot/nested_transforms.dot new file mode 100644 index 0000000..18a1822 --- /dev/null +++ b/ground_truth/dot/nested_transforms.dot @@ -0,0 +1,9 @@ +digraph nested_transforms { + "Ingest" [shape=box fillcolor="#4e79a7"]; + "Transform" [shape=box fillcolor="#f28e2b"]; + "Publish" [shape=box fillcolor="#59a14f"]; + "Quarantine" [shape=box fillcolor="#e15759" style=dashed]; + "Ingest" -> "Transform"; + "Transform" -> "Publish"; + "Transform" -> "Quarantine" [label="on error" style=dashed]; +} diff --git a/ground_truth/dot/org_chart.dot b/ground_truth/dot/org_chart.dot new file mode 100644 index 0000000..a0f0d72 --- /dev/null +++ b/ground_truth/dot/org_chart.dot @@ -0,0 +1,14 @@ +digraph org_chart { + "Jane Smith\nChief Executive Officer" [shape=box fillcolor="#2c3e50"]; + "Bob Chen\nChief Technology Officer" [shape=box fillcolor="#2980b9"]; + "Maria Garcia\nChief Financial Officer" [shape=box fillcolor="#27ae60"]; + "Alex Johnson\nChief Operating Officer" [shape=box fillcolor="#8e44ad"]; + "Engineering" [shape=box fillcolor="#3498db"]; + "Product" [shape=box fillcolor="#3498db"]; + "Finance" [shape=box fillcolor="#2ecc71"]; + "Sales" [shape=box fillcolor="#9b59b6"]; + "Operations" [shape=box fillcolor="#9b59b6"]; + "Jane Smith\nChief Executive Officer" -> "Bob Chen\nChief Technology Officer"; + "Jane Smith\nChief Executive Officer" -> "Maria Garcia\nChief Financial Officer"; + "Jane Smith\nChief Executive Officer" -> "Alex Johnson\nChief Operating Officer"; +} diff --git a/ground_truth/dot/plantuml_activity.dot b/ground_truth/dot/plantuml_activity.dot new file mode 100644 index 0000000..9b3b0bf --- /dev/null +++ b/ground_truth/dot/plantuml_activity.dot @@ -0,0 +1,14 @@ +digraph plantuml_activity { + // PlantUML draws the (*) start and end as unlabelled markers. Ground truth in this corpus + // is keyed by label, so an unlabelled marker has no key and is not a node, and the two + // edges joining them are not edges either. See diagrams/README.md. + "Read config" [shape=box]; + "Open input" [shape=box]; + "Detect format" [shape=box]; + "Extract text" [shape=box]; + "Render markdown" [shape=box]; + "Read config" -> "Open input"; + "Open input" -> "Detect format"; + "Detect format" -> "Extract text"; + "Extract text" -> "Render markdown"; +} diff --git a/ground_truth/dot/plantuml_swimlane.dot b/ground_truth/dot/plantuml_swimlane.dot new file mode 100644 index 0000000..ec00a6c --- /dev/null +++ b/ground_truth/dot/plantuml_swimlane.dot @@ -0,0 +1,14 @@ +digraph plantuml_swimlane { + // "Author", "Editor" and "Legal" are swimlane headers labelling a band that spans the whole + // drawing. A band is not a node; reporting one is a false positive. The unlabelled start and + // stop markers are out of scope for label-keyed ground truth, as in plantuml_activity. + "Draft article" [shape=box]; + "Review draft" [shape=box]; + "Clear rights" [shape=box]; + "Schedule slot" [shape=box]; + "Publish" [shape=box]; + "Draft article" -> "Review draft"; + "Review draft" -> "Clear rights"; + "Clear rights" -> "Schedule slot"; + "Schedule slot" -> "Publish"; +} diff --git a/ground_truth/dot/simple_svg.dot b/ground_truth/dot/simple_svg.dot new file mode 100644 index 0000000..e69de29 diff --git a/ground_truth/ground_truth_mapping.json b/ground_truth/ground_truth_mapping.json index d5ce093..789a5a0 100644 --- a/ground_truth/ground_truth_mapping.json +++ b/ground_truth/ground_truth_mapping.json @@ -102,6 +102,31 @@ "docx_rich_cells.docx": "test_documents/ground_truth/json/docx_rich_cells.docx.txt", "docx_simple": "test_documents/ground_truth/docx/docx_simple.txt", "docx_tables": "test_documents/ground_truth/docx/docx_tables.txt", + "dot_data_dashboard": "test_documents/ground_truth/dot/data_dashboard.dot", + "dot_flamegraph_intel_paging": "test_documents/ground_truth/dot/5_level_paging_and_5_level_ept_intel_revision_1_1_may_2017.dot", + "dot_flowchart": "test_documents/ground_truth/dot/flowchart.dot", + "dot_graphviz_bidirectional": "test_documents/ground_truth/dot/graphviz_bidirectional.dot", + "dot_graphviz_cjk": "test_documents/ground_truth/dot/graphviz_cjk.dot", + "dot_graphviz_clusters": "test_documents/ground_truth/dot/graphviz_clusters.dot", + "dot_graphviz_flow": "test_documents/ground_truth/dot/graphviz_flow.dot", + "dot_graphviz_large": "test_documents/ground_truth/dot/graphviz_large.dot", + "dot_graphviz_network": "test_documents/ground_truth/dot/graphviz_network.dot", + "dot_graphviz_ortho": "test_documents/ground_truth/dot/graphviz_ortho.dot", + "dot_graphviz_record": "test_documents/ground_truth/dot/graphviz_record.dot", + "dot_graphviz_selfloop": "test_documents/ground_truth/dot/graphviz_selfloop.dot", + "dot_graphviz_states": "test_documents/ground_truth/dot/graphviz_states.dot", + "dot_icon_nodes": "test_documents/ground_truth/dot/icon_nodes.dot", + "dot_libreoffice_connectors": "test_documents/ground_truth/dot/libreoffice_connectors.dot", + "dot_mermaid_flow": "test_documents/ground_truth/dot/mermaid_flow.dot", + "dot_mixed_page": "test_documents/ground_truth/dot/mixed_page.dot", + "dot_negative_form": "test_documents/ground_truth/dot/negative_form.dot", + "dot_negative_pie_chart": "test_documents/ground_truth/dot/negative_pie_chart.dot", + "dot_negative_ruled_table": "test_documents/ground_truth/dot/negative_ruled_table.dot", + "dot_nested_transforms": "test_documents/ground_truth/dot/nested_transforms.dot", + "dot_org_chart": "test_documents/ground_truth/dot/org_chart.dot", + "dot_plantuml_activity": "test_documents/ground_truth/dot/plantuml_activity.dot", + "dot_plantuml_swimlane": "test_documents/ground_truth/dot/plantuml_swimlane.dot", + "dot_simple_svg": "test_documents/ground_truth/dot/simple_svg.dot", "drawingml.docx": "test_documents/ground_truth/json/drawingml.docx.txt", "duck.md": "test_documents/ground_truth/md/duck.md.txt", "duplicate-paragraphs": "test_documents/ground_truth/doc/duplicate-paragraphs.txt", diff --git a/poly.toml b/poly.toml index f4bd657..e5e5e42 100644 --- a/poly.toml +++ b/poly.toml @@ -18,6 +18,7 @@ exclude = [ "csv/**", "data_formats/**", "dbf/**", + "diagrams/**", "doc/**", "docbook/**", "docx/**", diff --git a/scripts/check_diagram_ground_truth.py b/scripts/check_diagram_ground_truth.py new file mode 100644 index 0000000..d1d0c93 --- /dev/null +++ b/scripts/check_diagram_ground_truth.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python3 +"""Check every Graphviz diagram fixture's ground truth against what Graphviz actually drew. + +The ground truth under `ground_truth/dot/` is written by hand, keyed by node label, and it is +what the whole diagram corpus is measured against -- so a transcription slip in it is worse +than a bug, because it silently redefines "correct". This re-derives the graph from the +committed `.dot` source using `dot -Tplain`, which reports the labels and edges as Graphviz +itself resolved them, and diffs that against the ground truth. + +Two things the raw `-Tplain` output does not say, and this accounts for: + +* `dir=back` draws the arrowhead at the tail, so the edge reads the other way round. A + recogniser working from geometry sees only where the arrowhead is, so the ground truth + records the drawn direction and the declaration order is discarded. +* A record node's label carries port names and field separators. One record is one node, + so its key is the field texts as one multi-line label. + +Needs Graphviz on PATH, which is a fixture-regeneration dependency rather than a CI one -- it +reports that it skipped instead of failing when Graphviz is absent. + +Usage: + python3 scripts/check_diagram_ground_truth.py +""" + +from __future__ import annotations + +import re +import shutil +import subprocess +from pathlib import Path + +# Engine per fixture: the layout Graphviz was invoked with, which has to match how the +# committed SVG was rendered or the comparison is against a different drawing. +ENGINES = { + "graphviz_bidirectional": "dot", + "graphviz_cjk": "dot", + "graphviz_clusters": "dot", + "graphviz_flow": "dot", + "graphviz_large": "dot", + "graphviz_network": "neato", + "graphviz_ortho": "dot", + "graphviz_record": "dot", + "graphviz_selfloop": "neato", + "graphviz_states": "dot", +} + +ROOT = Path(__file__).resolve().parent.parent +SOURCES = ROOT / "diagrams" / "src" +GROUND_TRUTH = ROOT / "ground_truth" / "dot" + +PLAIN_TOKEN = re.compile(r'"[^"]*"|\S+') +PORT = re.compile(r"<\w+>") +COMMENT = re.compile(r"//.*") +GT_NODE = re.compile(r'^\s*"([^"]+)"\s*\[', re.M) +GT_EDGE = re.compile(r'"([^"]+)"\s*-[->]\s*"([^"]+)"') + + +def joined_record_fields(label: str) -> str: + """One record is one node, so its key is its field texts as one multi-line label.""" + if "<" not in label: + return label + return "\\n".join(PORT.sub("", field).strip() for field in label.split("|")) + + +def reversed_edges(source: Path) -> set[tuple[str, str]]: + """Edges declared with dir=back, which Graphviz draws pointing the other way.""" + text = COMMENT.sub("", source.read_text(encoding="utf-8")) + declarations = re.findall(r"(\w+)\s*->\s*(\w+)\s*\[([^\]]*)\]", text) + return {(tail, head) for tail, head, attrs in declarations if "dir=back" in attrs} + + +def drawn_graph(stem: str, engine: str) -> tuple[list[str], list[tuple[str, str]]]: + source = SOURCES / f"{stem}.dot" + plain = subprocess.run([engine, "-Tplain", str(source)], capture_output=True, text=True, check=True).stdout + flipped = reversed_edges(source) + labels: dict[str, str] = {} + nodes: list[str] = [] + edges: list[tuple[str, str]] = [] + for line in plain.splitlines(): + fields = [token[1:-1] if token.startswith('"') else token for token in PLAIN_TOKEN.findall(line)] + if fields[0] == "node": + labels[fields[1]] = joined_record_fields(fields[6]) + nodes.append(labels[fields[1]]) + elif fields[0] == "edge": + tail, head = fields[1], fields[2] + if (tail, head) in flipped: + tail, head = head, tail + edges.append((labels[tail], labels[head])) + return sorted(nodes), sorted(edges) + + +def recorded_graph(stem: str) -> tuple[list[str], list[tuple[str, str]]]: + text = COMMENT.sub("", (GROUND_TRUTH / f"{stem}.dot").read_text(encoding="utf-8")) + return sorted(GT_NODE.findall(text)), sorted(GT_EDGE.findall(text)) + + +def main() -> int: + if shutil.which("dot") is None: + print("graphviz is not installed, skipping the ground-truth cross-check") + return 0 + failures = 0 + for stem, engine in sorted(ENGINES.items()): + drawn_nodes, drawn_edges = drawn_graph(stem, engine) + recorded_nodes, recorded_edges = recorded_graph(stem) + if drawn_nodes == recorded_nodes and drawn_edges == recorded_edges: + print(f"ok {stem:24s} nodes {len(recorded_nodes):3d} edges {len(recorded_edges):3d}") + continue + failures += 1 + print(f"FAIL {stem}") + for name, drawn, recorded in ( + ("nodes", drawn_nodes, recorded_nodes), + ("edges", drawn_edges, recorded_edges), + ): + print(f" {name} drawn but not in ground truth: {sorted(set(drawn) - set(recorded))}") + print(f" {name} in ground truth but not drawn: {sorted(set(recorded) - set(drawn))}") + return 1 if failures else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/strip_svg_graph_metadata.py b/scripts/strip_svg_graph_metadata.py new file mode 100644 index 0000000..ef532de --- /dev/null +++ b/scripts/strip_svg_graph_metadata.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 +"""Remove the answer from a rendered SVG, leaving a geometry-only fixture behind. + +Every diagram tool writes the graph it just laid out back into its own output, each in its +own way: + + graphviz a the node id + a->b the whole edge + the whole edge, again + mermaid both endpoints + plantuml both endpoints + both endpoints, again + +A recogniser that reads those scores perfectly without inspecting a single coordinate, so a +fixture carrying them cannot measure geometry recovery. + +Reading them is not cheating -- when a file states its graph, using it is the correct and +exact thing to do, and that is what the Class A fixtures are for. It is simply a different +capability from inferring a graph from shapes and strokes, and measuring the two together +measures neither. So each fixture whose producer states its answer ships twice: as emitted, +and stripped. + +Two rules, and both are producer-agnostic rather than a list of the dialects seen so far: + +1. Drop `` inside an element group. The root `<title>` names the whole drawing, which + a hand-drawn diagram would plausibly carry too, so it stays. Clusters count as elements: + a cluster is a rectangle that is deliberately *not* a node, and `<title>cluster_ingest` + announces exactly that, which is the judgement the cluster fixtures exist to test. + +2. Drop every `id` that nothing in the document refers to, and every `data-id`. An id no + `url(#...)`, `href="#..."` or stylesheet selector points at has no effect on rendering -- + it is there purely to say which element this is, and that is the answer. + +3. Drop every XML comment. Graphviz and PlantUML both restate the entire edge list in + comments as well, so stripping only the titles and the ids leaves the answer sitting in + the file in plain text. The producer banner goes with them; provenance belongs in + `diagrams/manifest.json`, which is where a reader should be looking for it anyway. + +What deliberately stays is anything that gives an element's *type* without naming its +endpoints: `class="node"`, `class="flowchart-link"`, `class="com.sun.star.drawing.ConnectorShape"`. +Knowing a stroke is some connector still leaves you to work out what it connects, which is +the whole task. + +Usage: + python3 scripts/strip_svg_graph_metadata.py diagrams/svg/graphviz_flow.svg \ + diagrams/svg/graphviz_flow_geometry.svg +""" + +from __future__ import annotations + +import re +import sys +from pathlib import Path + +TITLE = re.compile(r"[ \t]*.*?\n?", re.DOTALL) +ELEMENT_GROUP = re.compile(r'()(.*?)()', re.DOTALL) + +COMMENT = re.compile(r"[ \t]*\n?", re.DOTALL) +ID_ATTRIBUTE = re.compile(r'\s+id="([^"]*)"') +DATA_ID_ATTRIBUTE = re.compile(r'\s+data-id="[^"]*"') +FRAGMENT_REFERENCE = re.compile(r'(?:url\(\s*#|(?:xlink:)?href="#)([^)"\s]+)') +STYLE_BLOCK = re.compile(r"]*>(.*?)", re.DOTALL) +STYLE_SELECTOR = re.compile(r"#([A-Za-z_][\w:.-]*)") + + +def referenced_ids(svg: str) -> set[str]: + """Every id the document itself points at, so removing the rest changes no pixels.""" + ids = set(FRAGMENT_REFERENCE.findall(svg)) + for block in STYLE_BLOCK.findall(svg): + ids.update(STYLE_SELECTOR.findall(block)) + return ids + + +def strip(svg: str) -> str: + without_titles = ELEMENT_GROUP.sub(lambda m: m.group(1) + TITLE.sub("", m.group(2)) + m.group(3), svg) + keep = referenced_ids(without_titles) + without_ids = ID_ATTRIBUTE.sub(lambda m: m.group(0) if m.group(1) in keep else "", without_titles) + return COMMENT.sub("", DATA_ID_ATTRIBUTE.sub("", without_ids)) + + +def main() -> int: + if len(sys.argv) != 3: + print(__doc__, file=sys.stderr) + return 2 + source, destination = Path(sys.argv[1]), Path(sys.argv[2]) + original = source.read_text(encoding="utf-8") + stripped = strip(original) + if stripped == original: + print(f"{source}: nothing to strip, it states no graph metadata", file=sys.stderr) + return 1 + destination.write_text(stripped, encoding="utf-8") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/test_diagram_manifest.py b/scripts/test_diagram_manifest.py new file mode 100644 index 0000000..fa24f3a --- /dev/null +++ b/scripts/test_diagram_manifest.py @@ -0,0 +1,187 @@ +"""Invariants for `diagrams/manifest.json` and the fixtures it indexes. + +The manifest is the corpus's answer key, and an answer key that disagrees with the files on +disk is worse than no answer key: it makes a scorer report numbers nobody can act on. These +run in CI, need no Graphviz, and cover the two ways it has already gone wrong once -- a +fixture path that does not exist, and a geometry-only variant that still states its answer. + +The Graphviz sources are cross-checked against what Graphviz actually drew separately, by +`scripts/check_diagram_ground_truth.py`, which needs the renderer. +""" + +from __future__ import annotations + +import json +import re +import unittest +from pathlib import Path + +from strip_svg_graph_metadata import referenced_ids, strip + +ROOT = Path(__file__).resolve().parent.parent +MANIFEST = json.loads((ROOT / "diagrams" / "manifest.json").read_text(encoding="utf-8")) +FIXTURES = MANIFEST["fixtures"] + +COMMENT = re.compile(r"//.*") +GT_NODE = re.compile(r'^\s*"([^"]+)"\s*\[', re.M) +GT_EDGE = re.compile(r'"([^"]+)"\s*-[->]\s*"([^"]+)"') +XML_COMMENT = re.compile(r"", re.DOTALL) +ATTRIBUTE = re.compile(r'[\w:.-]+="([^"]*)"') + + +def ground_truths(fixture: dict) -> list[Path]: + return [ROOT / graph["ground_truth"] for graph in fixture["graphs"]] + + +class FixtureFilesTest(unittest.TestCase): + def test_should_index_a_file_that_exists(self) -> None: + for fixture in FIXTURES: + with self.subTest(fixture["path"]): + self.assertTrue((ROOT / fixture["path"]).is_file()) + + def test_should_point_at_a_ground_truth_that_exists(self) -> None: + for fixture in FIXTURES: + for path in ground_truths(fixture): + with self.subTest(str(path)): + self.assertTrue(path.is_file()) + + def test_should_point_at_a_source_that_exists_when_it_claims_one(self) -> None: + for fixture in FIXTURES: + source = fixture.get("source") + if source: + with self.subTest(source): + self.assertTrue((ROOT / source).is_file()) + + def test_should_ship_the_geometry_variant_it_names(self) -> None: + for fixture in FIXTURES: + variant = fixture.get("geometry_only_variant") + if variant: + with self.subTest(variant): + self.assertTrue((ROOT / variant).is_file()) + + def test_should_index_every_svg_that_ships_under_diagrams(self) -> None: + indexed = {f["path"] for f in FIXTURES} + indexed |= {f["geometry_only_variant"] for f in FIXTURES if f.get("geometry_only_variant")} + on_disk = {str(path.relative_to(ROOT)) for path in (ROOT / "diagrams" / "svg").glob("*.svg")} + self.assertEqual(set(), on_disk - indexed, "fixture on disk that the manifest does not index") + + +class GroundTruthAgreementTest(unittest.TestCase): + def test_should_record_the_node_and_edge_counts_its_ground_truth_holds(self) -> None: + for fixture in FIXTURES: + for graph in fixture["graphs"]: + text = COMMENT.sub("", (ROOT / graph["ground_truth"]).read_text(encoding="utf-8")) + with self.subTest(graph["ground_truth"]): + self.assertEqual(graph["nodes"], len(GT_NODE.findall(text))) + self.assertEqual(graph["edges"], len(GT_EDGE.findall(text))) + + def test_should_write_an_undirected_ground_truth_as_a_graph_not_a_digraph(self) -> None: + for fixture in FIXTURES: + for graph in fixture["graphs"]: + raw = (ROOT / graph["ground_truth"]).read_text(encoding="utf-8") + body = COMMENT.sub("", raw) + with self.subTest(graph["ground_truth"]): + if graph["directed"]: + self.assertRegex(body, r"^digraph\b") + self.assertNotIn(" -- ", body) + else: + self.assertRegex(body, r"^graph\b") + self.assertNotIn("->", body) + + def test_should_leave_a_negative_with_no_graph_and_a_stated_reason(self) -> None: + negatives = [f for f in FIXTURES if f.get("negative")] + self.assertTrue(negatives) + for fixture in negatives: + with self.subTest(fixture["path"]): + self.assertEqual([], fixture["graphs"]) + self.assertTrue(fixture["reason"]) + # The empty file keeps "not a diagram" and "not yet annotated" apart: the + # reason in the manifest is what says which one this is. + self.assertEqual("", (ROOT / fixture["ground_truth"]).read_text(encoding="utf-8")) + + def test_should_write_a_comment_that_cannot_be_misread_as_a_node_or_an_edge(self) -> None: + # Consumers parse these files line by line and do not all strip `//` comments -- the + # scorer on xberg#1410 does not, and a prose comment mentioning `a -> b` was read as a + # real edge, which is how graphviz_large came to report one more edge than it draws. + # Cheaper to keep the comments free of DOT syntax than to rely on every reader. + for fixture in FIXTURES: + for graph in fixture["graphs"]: + path = ROOT / graph["ground_truth"] + for number, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1): + comment = line.partition("//")[2] + with self.subTest(f"{graph['ground_truth']}:{number}"): + for token in ("->", "--", "["): + self.assertNotIn(token, comment) + + +class GeometryVariantTest(unittest.TestCase): + """A geometry-only variant must not state the answer anywhere a parser can reach it.""" + + def variants(self): + for fixture in FIXTURES: + variant = fixture.get("geometry_only_variant") + if variant: + yield fixture, (ROOT / variant).read_text(encoding="utf-8") + + def test_should_be_exactly_what_stripping_its_parent_produces(self) -> None: + # The strongest statement available, and the one that does not depend on knowing how a + # given producer encodes its answer: the committed variant is the tool's output, not + # something hand-edited that merely looks stripped. + for fixture, body in self.variants(): + parent = (ROOT / fixture["path"]).read_text(encoding="utf-8") + with self.subTest(fixture["geometry_only_variant"]): + self.assertEqual(strip(parent), body) + + def test_should_have_nothing_left_to_strip(self) -> None: + for fixture, body in self.variants(): + with self.subTest(fixture["geometry_only_variant"]): + self.assertEqual(body, strip(body)) + + def test_should_carry_no_identifier_that_could_name_an_element(self) -> None: + # Mermaid puts both endpoints of every edge in id="L_start_auth_0" and data-id, using + # its own node ids rather than the labels, so a label-based check cannot see it. + for fixture, body in self.variants(): + referenced = referenced_ids(body) + declared = set(re.findall(r'\sid="([^"]*)"', body)) + with self.subTest(fixture["geometry_only_variant"]): + self.assertEqual(set(), declared - referenced) + self.assertNotIn("data-id=", body) + + def test_should_carry_no_xml_comments(self) -> None: + # Graphviz and PlantUML both restate the whole edge list in comments. + for fixture, body in self.variants(): + with self.subTest(fixture["path"]): + self.assertEqual([], XML_COMMENT.findall(body)) + + def test_should_not_name_a_ground_truth_node_in_any_attribute(self) -> None: + for fixture, body in self.variants(): + labels = set() + for graph in fixture["graphs"]: + text = COMMENT.sub("", (ROOT / graph["ground_truth"]).read_text(encoding="utf-8")) + labels.update(GT_NODE.findall(text)) + values = ATTRIBUTE.findall(body) + for label in labels: + with self.subTest(fixture=fixture["path"], label=label): + self.assertFalse( + [v for v in values if label.lower() in v.lower()], + "a node label leaked into an attribute value", + ) + + def test_should_keep_every_node_label_as_rendered_text(self) -> None: + # Stripping must remove metadata only. If a label vanished from the drawing itself the + # variant would be measuring a different graph. + for fixture, body in self.variants(): + if fixture["path"].endswith("mermaid_flow.svg"): + continue # mermaid wraps labels in HTML, which splits them across elements + for graph in fixture["graphs"]: + text = COMMENT.sub("", (ROOT / graph["ground_truth"]).read_text(encoding="utf-8")) + for label in GT_NODE.findall(text): + # A multi-line label is one node but several elements, so each line + # is looked for on its own. + for word in label.replace("\\n", " ").split(): + with self.subTest(fixture=fixture["path"], word=word): + self.assertIn(word, body) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_strip_svg_graph_metadata.py b/scripts/test_strip_svg_graph_metadata.py new file mode 100644 index 0000000..8a6dbe7 --- /dev/null +++ b/scripts/test_strip_svg_graph_metadata.py @@ -0,0 +1,119 @@ +"""Tests for `strip_svg_graph_metadata.py`. + +The script exists to stop a fixture from stating its own answer, so what these assert is that +the answer really is gone, that the drawing still renders the same, and that an element's type +survives -- knowing a stroke is a connector is not knowing what it connects. +""" + +from __future__ import annotations + +import unittest + +from strip_svg_graph_metadata import strip + +GRAPHVIZ_NODE = ( + '\na\n\n\n' +) +GRAPHVIZ_EDGE = ( + '\n' + "a->b\n" + '\n' + "\n" +) +GRAPHVIZ_CLUSTER = ( + '\n' + "cluster_ingest\n" + '\n' + "\n" +) +GRAPHVIZ_ROOT = '\nwhole_graph\n' + +MERMAID_EDGE = ( + '' +) +PLANTUML_EDGE = ( + '' + '' +) + + +class TitleStrippingTest(unittest.TestCase): + def test_should_remove_the_node_id_from_a_node_group(self) -> None: + self.assertNotIn("", strip(GRAPHVIZ_NODE)) + + def test_should_remove_the_edge_endpoints_from_an_edge_group(self) -> None: + stripped = strip(GRAPHVIZ_EDGE) + self.assertNotIn("a->b", stripped) + self.assertNotIn("<title>", stripped) + + def test_should_remove_the_cluster_name_so_a_container_is_not_labelled_as_one(self) -> None: + self.assertNotIn("cluster_ingest", strip(GRAPHVIZ_CLUSTER)) + + def test_should_keep_the_root_title_that_names_the_whole_drawing(self) -> None: + self.assertIn("<title>whole_graph", strip(GRAPHVIZ_ROOT + GRAPHVIZ_NODE)) + + +class IdentifierStrippingTest(unittest.TestCase): + def test_should_remove_a_mermaid_edge_id_naming_both_endpoints(self) -> None: + stripped = strip(MERMAID_EDGE) + self.assertNotIn("L_start_auth_0", stripped) + self.assertNotIn("data-id", stripped) + + def test_should_remove_a_plantuml_edge_id_naming_both_endpoints(self) -> None: + self.assertNotIn("Read config-to-Open input", strip(PLANTUML_EDGE)) + + def test_should_keep_an_id_the_document_points_at_with_url(self) -> None: + marker = '' + self.assertEqual(marker, strip(marker)) + + def test_should_keep_an_id_the_document_points_at_with_href(self) -> None: + used = '' + self.assertEqual(used, strip(used)) + + def test_should_keep_an_id_a_stylesheet_selector_points_at(self) -> None: + styled = '' + self.assertIn('id="my-svg"', strip(styled)) + + def test_should_keep_an_opaque_id_that_names_nothing_but_is_referenced(self) -> None: + # LibreOffice numbers its shapes id1..idN and refers to them from clip paths. + libreoffice = '' + self.assertEqual(libreoffice, strip(libreoffice)) + + +class CommentStrippingTest(unittest.TestCase): + def test_should_remove_a_graphviz_comment_restating_an_edge(self) -> None: + self.assertNotIn("a->b", strip("\n" + GRAPHVIZ_EDGE)) + + def test_should_remove_a_plantuml_comment_restating_an_edge(self) -> None: + commented = "" + PLANTUML_EDGE + self.assertNotIn("Read config", strip(commented)) + + def test_should_remove_the_producer_banner_because_provenance_lives_in_the_manifest(self) -> None: + banner = "\n" + self.assertEqual("", strip(banner)) + + +class PreservationTest(unittest.TestCase): + def test_should_preserve_every_byte_of_geometry(self) -> None: + stripped = strip(GRAPHVIZ_NODE + GRAPHVIZ_EDGE + GRAPHVIZ_CLUSTER) + self.assertIn('points="0,0 1,1"', stripped) + self.assertIn('d="M63,-234.8C63,-227.16 63,-208.24"', stripped) + self.assertIn('points="8,-148 8,-297 84,-297"', stripped) + + def test_should_keep_the_class_that_gives_an_element_its_type(self) -> None: + for original in (GRAPHVIZ_NODE, GRAPHVIZ_EDGE, MERMAID_EDGE, PLANTUML_EDGE): + stripped = strip(original) + self.assertIn("class=", stripped) + + def test_should_keep_the_text_a_node_is_labelled_with(self) -> None: + labelled = 'aStart' + self.assertIn("Start", strip(labelled)) + + def test_should_leave_a_hand_authored_svg_without_graph_metadata_untouched(self) -> None: + hand_authored = 'Start' + self.assertEqual(hand_authored, strip(hand_authored)) + + +if __name__ == "__main__": + unittest.main()