From 71d2a7c11dc2ebf6c41e06f7f911895d7be26746 Mon Sep 17 00:00:00 2001 From: v0lumehi Date: Tue, 27 Oct 2026 12:18:29 +0100 Subject: [PATCH 1/3] README's games gallery is written from docs/images/gallery.tsv The gallery is a table with three games to a row, and new games go at its end. Every game after the first moved cell shifts, so two pull requests that each add a game always conflicted in README.md, and the resolution was redoing the rows by hand. docs/images/gallery.tsv now holds the gallery, one row per game in the order it shows them: image, name, note (the IPA size and a remark) and optionally the title ID that links the name to the game's note. dm/tools/docs_gallery.py writes README.md's table from it between two comment markers; the table it writes is byte for byte the one README had (the only change to README.md is the two markers). .gitattributes merges the TSV with the union driver, so two new rows merge without a conflict; README.md can still conflict in the gallery, and running docs_gallery.py on it, conflict markers and all, writes the right table. Checked in a clone: two branches that each add a game at the end rebase onto each other with the TSV merged cleanly and README.md conflicting; docs_gallery.py then writes both games and test_docs passes. test_docs fails when README's gallery is not the one the table gives, or a game is in the table twice. AGENTS.md's rule for the gallery names the table. --- .gitattributes | 3 ++ AGENTS.md | 2 +- README.md | 2 + dm/tools/docs_gallery.py | 58 ++++++++++++++++++++++++++ dm/tools/tests/test_docs.py | 12 +++++- docs/images/gallery.tsv | 82 +++++++++++++++++++++++++++++++++++++ 6 files changed, 157 insertions(+), 2 deletions(-) create mode 100644 .gitattributes create mode 100644 dm/tools/docs_gallery.py create mode 100644 docs/images/gallery.tsv diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000000..8fa2fa7865 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,3 @@ +# The games gallery's table (dm/tools/docs_gallery.py writes README.md's gallery from it): a new game is a row at its end. README.md itself is not merged this way; after a +# conflict in its gallery, run docs_gallery.py. test_docs fails on a game in the table twice +docs/images/gallery.tsv merge=union diff --git a/AGENTS.md b/AGENTS.md index 09c9908849..289ed9e7f0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,7 +12,7 @@ | Another Switch title | the NSP script (below), then [docs/agents](docs/agents/README.md) | | Packing, building, prewarm lists, power | [docs/development.md](docs/development.md) | | Checking a change before a pull request | [06-dev-loop.md](docs/agents/06-dev-loop.md) | -- Where a new finding goes, once: a fact about one game (its passes, shader keys, measurements, why its profile needs a key or a cut, its status) in that game's note, in the section it belongs to and marked current or resolved; a general mechanism (a host or tool change that holds for other games) as an entry in its `docs/reference/.md` section (Symptom, Cause, Fix, Found on with the game's note linked), then `python3 dm/tools/docs_symptoms.py` to update the symptom index, and a line under the game note's "Found on this game". Link to a finding instead of repeating it. Adding a game takes its note and its row in the games table. Once Mac gameplay testing passes, add it to README's regular games gallery with a screenshot and IPA size; Mac testing is sufficient for inclusion. Add caption notes only for noteworthy features or limitations (such as a 60 fps upgrade or controller requirement); otherwise show only the size. Keep testing milestones, platform and remaining validation in the game note. This file changes only when a rule does; it states no game count (the games table is the count). `dm/tools/tests/test_docs.py` checks the links and anchors, the notes and their table rows, the symptom index, and the length of this file (1,500 words) and of each guide in `docs/agents` (6,000). +- Where a new finding goes, once: a fact about one game (its passes, shader keys, measurements, why its profile needs a key or a cut, its status) in that game's note, in the section it belongs to and marked current or resolved; a general mechanism (a host or tool change that holds for other games) as an entry in its `docs/reference/.md` section (Symptom, Cause, Fix, Found on with the game's note linked), then `python3 dm/tools/docs_symptoms.py` to update the symptom index, and a line under the game note's "Found on this game". Link to a finding instead of repeating it. Adding a game takes its note and its row in the games table. Once Mac gameplay testing passes, add it to README's regular games gallery with a screenshot and IPA size (`docs/images/gallery.tsv`, then `dm/tools/docs_gallery.py`); Mac testing is sufficient for inclusion. Add caption notes only for noteworthy features or limitations (such as a 60 fps upgrade or controller requirement); otherwise show only the size. Keep testing milestones, platform and remaining validation in the game note. This file changes only when a rule does; it states no game count (the games table is the count). `dm/tools/tests/test_docs.py` checks the links and anchors, the notes and their table rows, the symptom index, and the length of this file (1,500 words) and of each guide in `docs/agents` (6,000). - Each game's constants and shader tables form a game profile in `dm/games//`, where `` is the base game's title ID: `game.tsv` plus the tables (and `patches.py` when the rewriter has patches for its builds). Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)). - Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it. - Find a bug's root cause and fix it there; do not just work around it. Before changing code, find why it happens: the faulting instruction and its decompilation, the value that went wrong and who wrote it, the Horizon / NVN / Vulkan / Metal semantics the host or the translator gets wrong. Then fix that place. A change that only hides the symptom (a stub, a skipped call, a clamp, a retry, a sleep or a longer timeout, a cache flush, a profile key that turns a feature off) is a workaround: use one only when the cause is known and lies where it cannot be fixed (the game's own code, which a rewriter cut changes deliberately, or a platform limit), or when the user asks for a stopgap. Then say so in the code comment and in the game note or reference entry: the cause, or what is still unknown, marked open. diff --git a/README.md b/README.md index 8a4a3f8b5a..87bdae2fe6 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ Needs iOS / iPadOS 26 or later on an iPhone 15 Pro or newer (A17 Pro chip or lat Under each game: the IPA's size, and "60 fps with 1.2.0" where the game runs at 60 fps (30 on the Switch) with that update. + @@ -166,6 +167,7 @@ Under each game: the IPA's size, and "60 fps with 1.2.0" where the game runs at
Mario Kart 8 Deluxe
Mario Kart 8 Deluxe
~11.9 GB · no Mii heads		FINAL FANTASY CRYSTAL CHRONICLES Remastered Edition
FINAL FANTASY CRYSTAL CHRONICLES Remastered Edition
+ ## Pack an IPA diff --git a/dm/tools/docs_gallery.py b/dm/tools/docs_gallery.py new file mode 100644 index 0000000000..c66ea51b20 --- /dev/null +++ b/dm/tools/docs_gallery.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python3 +# Writes the games gallery in README.md (between its gallery markers) from docs/images/gallery.tsv: one cell per row of the table, three to a row, in the table's order. +# The table is the source: a new game is a row in it (usually at the end), so two pull requests that each add a game change one line each instead of moving every cell after +# the first one; .gitattributes merges the table with the union driver. Columns: image (a file in docs/images), name (the caption and the image's alt text), note (the small +# line under it: the IPA's size, then " · " and a remark if any) and, optionally, the base game's title ID, which links the name to the game's note (dm/games//README.md). +# Run it after changing the table; --check only reports whether README.md is current (dm/tools/tests/test_docs.py runs it). +# Usage: docs_gallery.py [--check] +import pathlib, re, sys + +REPO = pathlib.Path(__file__).resolve().parents[2] +TABLE = REPO / 'docs' / 'images' / 'gallery.tsv' +OUT = REPO / 'README.md' +BEGIN = '' +END = '' +PER_ROW = 3 + + +def rows(): + out = [] + for n, line in enumerate(TABLE.read_text(encoding='utf-8').splitlines(), 1): + if not line.strip() or line.startswith('#'): continue + c = line.split('\t') + if len(c) not in (3, 4): sys.exit(f'{TABLE.name}:{n}: 3 or 4 columns expected, {len(c)} found') + image, name, note, tid = c + [''] * (4 - len(c)) + if not (REPO / 'docs' / 'images' / image).is_file(): sys.exit(f'{TABLE.name}:{n}: no image docs/images/{image}') + if any(ch in name + note for ch in '"<>&'): sys.exit(f'{TABLE.name}:{n}: no " < > & in a name or note (they go into HTML as they are)') + if tid and not re.fullmatch(r'[0-9A-F]{16}', tid): sys.exit(f'{TABLE.name}:{n}: a title ID is 16 hex digits: {tid}') + out.append((image, name, note, tid)) + return out + + +def cell(image, name, note, tid): + caption = f'{name}' if tid else name + return f'{name}
{caption}
{note}' + + +def gallery(): + r = rows() + lines = [''] + for i in range(0, len(r), PER_ROW): + row = [cell(*x) for x in r[i:i + PER_ROW]] + lines += [''] + row + [''] * (PER_ROW - len(row)) + [''] # a short last row is filled with empty cells, as README had it + return '\n'.join(lines + ['
']) + + +def readme(): + text = OUT.read_text(encoding='utf-8') + if text.count(BEGIN) != 1 or text.count(END) != 1: sys.exit(f'README.md: the gallery markers ({BEGIN} … {END}) must appear once each') + head, rest = text.split(BEGIN) + return head + BEGIN + '\n' + gallery() + '\n' + END + rest.split(END, 1)[1] + + +if __name__ == '__main__': + text = readme() + if '--check' in sys.argv[1:]: + if OUT.read_text(encoding='utf-8') != text: sys.exit('README.md: the gallery is out of date: run dm/tools/docs_gallery.py') + else: + OUT.write_text(text, encoding='utf-8') diff --git a/dm/tools/tests/test_docs.py b/dm/tools/tests/test_docs.py index 4fa5625980..df35cc803f 100644 --- a/dm/tools/tests/test_docs.py +++ b/dm/tools/tests/test_docs.py @@ -1,7 +1,7 @@ """The repository's own Markdown stays navigable: every relative link and #anchor resolves (GitHub's heading anchors), every game profile has its note (dm/games//README.md) and a row in the games table of dm/games/README.md, the symptom index (docs/agents/symptoms.md) is the one dm/tools/docs_symptoms.py writes from docs/reference, and the entry points stay short: AGENTS.md at most 1,500 words, each numbered guide in docs/agents at most 6,000 (a new finding goes into a game note or a -reference, not into these). Third-party trees are left out. No game files needed.""" +reference, not into these), and README's games gallery is the one dm/tools/docs_gallery.py writes from docs/images/gallery.tsv. Third-party trees are left out. No game files needed.""" from pathlib import Path import re import subprocess @@ -12,6 +12,7 @@ REPO = Path(__file__).resolve().parents[3] sys.path.insert(0, str(REPO / 'dm/tools')) import docs_symptoms # noqa: E402 +import docs_gallery # noqa: E402 SKIP = ('third_party', 'node_modules') AGENTS_WORDS, GUIDE_WORDS = 1500, 6000 @@ -84,6 +85,15 @@ def test_index_is_current(self): self.assertEqual(docs_symptoms.OUT.read_text(), docs_symptoms.index(), 'docs/agents/symptoms.md is out of date: run dm/tools/docs_symptoms.py') +class Gallery(unittest.TestCase): + def test_gallery_is_current(self): + self.assertEqual(docs_gallery.OUT.read_text(encoding='utf-8'), docs_gallery.readme(), 'README.md: the gallery is out of date: run dm/tools/docs_gallery.py') + + def test_each_game_once(self): # a union merge of two rows for the same game keeps both + names = [r[1] for r in docs_gallery.rows()] + self.assertEqual(sorted({n for n in names if names.count(n) > 1}), [], 'docs/images/gallery.tsv: a game is in the table twice') + + class Length(unittest.TestCase): def test_entry_points_stay_short(self): words = lambda p: len(p.read_text().split()) diff --git a/docs/images/gallery.tsv b/docs/images/gallery.tsv new file mode 100644 index 0000000000..5fb0e5d79f --- /dev/null +++ b/docs/images/gallery.tsv @@ -0,0 +1,82 @@ +# The games gallery in README.md, in the order it shows them (dm/tools/docs_gallery.py writes it from this table). A new game: its screenshot in docs/images and a row here, +# usually at the end. Columns: image, name (caption and alt text), note (the IPA size, then " · " and a remark if any), and optionally the title ID that links the name to its note. +mario-kart-8-deluxe.jpg Mario Kart 8 Deluxe ~11.9 GB · no Mii heads +mario-vs-donkey-kong.jpg Mario vs. Donkey Kong ~1.6 GB +paper-mario-the-thousand-year-door.jpg Paper Mario: The Thousand-Year Door ~5 GB · 60 fps with 1.0.1 +super-mario-3d-world.jpg Super Mario 3D World + Bowser's Fury ~3 GB +super-mario-bros-wonder.jpg Super Mario Bros. Wonder ~4 GB +super-mario-galaxy.jpg Super Mario Galaxy ~3 GB +super-mario-galaxy-2.jpg Super Mario Galaxy 2 ~3 GB +super-mario-odyssey.jpg Super Mario Odyssey ~6 GB +zelda-breath-of-the-wild.jpg The Legend of Zelda: Breath of the Wild ~16 GB · 60 fps with 1.9.0 +zelda-links-awakening.jpg The Legend of Zelda: Link's Awakening ~6 GB +zelda-tears-of-the-kingdom.jpg The Legend of Zelda: Tears of the Kingdom ~17 GB · 60 fps with 1.4.3 +13-sentinels.jpg 13 Sentinels: Aegis Rim ~5 GB +absolum.jpg Absolum ~2 GB +ace-attorney-investigations.jpg Ace Attorney Investigations Collection ~7 GB +advance-wars-1-2-re-boot-camp.jpg Advance Wars 1+2: Re-Boot Camp ~7.1 GB +ai-the-somnium-files.jpg AI: THE SOMNIUM FILES ~8.3 GB +ai-the-somnium-files-nirvana-initiative.jpg AI: THE SOMNIUM FILES - nirvanA Initiative ~11.6 GB +animal-crossing-new-horizons.jpg Animal Crossing: New Horizons ~10.5 GB +animal-well.png ANIMAL WELL ~54 MB +astral-chain.png ASTRAL CHAIN ~9.7 GB · 60 fps target with 1.0.1 01007300020FA000 +balatro.jpg Balatro ~82 MB · played by touch +ball-x-pit.jpg BALL x PIT ~1.1 GB +bayonetta.jpg Bayonetta ~8.8 GB +bayonetta-2.jpg Bayonetta 2 ~12.5 GB +bayonetta-3.jpg Bayonetta 3 ~14.2 GB +big-brain-academy-brain-vs-brain.jpg Big Brain Academy: Brain vs. Brain ~0.8 GB +brain-memory-training-academy.jpg Brain Memory Training Academy ~0.1 GB +captain-toad-treasure-tracker.jpg Captain Toad: Treasure Tracker ~2 GB +catherine-full-body.jpg Catherine: Full Body ~14 GB +celeste.jpg Celeste ~0.9 GB +cuphead.jpg Cuphead ~5 GB +dark-souls-remastered-creation.jpg DARK SOULS: REMASTERED ~4 GB · 60 fps with 1.0.3 +digimon-story-cyber-sleuth.jpg Digimon Story Cyber Sleuth: Complete Edition ~5 GB +donkey-kong-country-tropical-freeze.jpg Donkey Kong Country: Tropical Freeze ~7.3 GB +dragon-quest-vii-reimagined.jpg DRAGON QUEST VII Reimagined ~5.9 GB +dragon-quest-xi-s.jpg DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition ~15 GB · 60 fps with 1.0.4 +final-fantasy-resonance.jpg FINAL FANTASY RESONANCE ~10 GB · 60 fps with 1.0.1 +final-fantasy-tactics.jpg FINAL FANTASY TACTICS - The Ivalice Chronicles ~11 GB +fire-emblem-engage.jpg Fire Emblem Engage ~15 GB · 60 fps with 2.0.0 +fire-emblem-three-houses.jpg Fire Emblem: Three Houses ~13 GB · 60 fps with 1.2.0 +ghost-trick-phantom-detective.jpg Ghost Trick: Phantom Detective ~3 GB +the-great-ace-attorney-chronicles.jpg The Great Ace Attorney Chronicles ~4 GB +hades.jpg Hades ~4 GB +hades-ii.jpg Hades II ~4 GB +hollow-knight-silksong.jpg Hollow Knight: Silksong ~6.4 GB +hyrule-warriors-age-of-calamity.jpg Hyrule Warriors: Age of Calamity ~15 GB · 60 fps with 1.3.0 +kirby-and-the-forgotten-land.jpg Kirby and the Forgotten Land ~6.4 GB +luigis-mansion-3.jpg Luigi's Mansion 3 ~7.5 GB · 60 fps with 1.4.0 +metroid-dread.jpg Metroid Dread ~5 GB +monster-hunter-rise.jpg Monster Hunter Rise ~22 GB · 60 fps with 16.0.2 +monster-train-2.jpg Monster Train 2 ~1.6 GB · 60 fps with 1.1.1 +octopath-traveler.jpg OCTOPATH TRAVELER ~3.6 GB +octopath-traveler-0.jpg Octopath Traveler 0 ~4 GB · 60 fps with 1.0.5 +octopath-traveler-ii.jpg Octopath Traveler II ~5.9 GB · 60 fps with 1.1.1 +patricks-parabox.jpg Patrick's Parabox ~0.2 GB +persona-5-royal.jpg Persona 5 Royal ~14.9 GB +pikmin-4.jpg Pikmin 4 ~10.8 GB +pokemon-brilliant-diamond.jpg Pokémon Brilliant Diamond ~4.6 GB · 60 fps with 1.3.0 +pokemon-legends-z-a.jpg Pokémon Legends: Z-A ~6 GB +pokemon-scarlet.png Pokémon Scarlet ~7.0 GB +pokemon-sword.jpg Pokémon Sword ~12.4 GB +pokemon-shield.jpg Pokémon Shield ~12.4 GB +portal.jpg Portal ~2.6 GB +raging-loop.jpg Raging Loop ~0.9 GB +rhythm-heaven-groove.jpg Rhythm Heaven Groove ~3 GB +romance-of-the-three-kingdoms-8-remake.jpg ROMANCE OF THE THREE KINGDOMS 8 REMAKE ~16.8 GB · controller only +sea-of-stars.jpg Sea of Stars ~5.1 GB +shin-megami-tensei-v-vengeance.jpg SHIN MEGAMI TENSEI V: Vengeance ~14.6 GB · 60 fps with 1.0.3 +stardew-valley.jpg Stardew Valley ~0.6 GB +steins-gate-reboot.jpg STEINS;GATE RE:BOOT ~6 GB +super-smash-bros-ultimate.jpg Super Smash Bros. Ultimate ~17.5 GB · offline +taiko-risshiden-v-dx.jpg Taiko Risshiden V DX ~3.3 GB · Japanese · played by touch +unicorn-overlord.jpg Unicorn Overlord ~4 GB +yoshis-crafted-world.jpg Yoshi's Crafted World ~6.1 GB 01006000040C2000 +nier-automata.png NieR:Automata The End of YoRHa Edition ~10.9 GB · 60 fps with 1.0.3 +zelda-echoes-of-wisdom.jpg The Legend of Zelda: Echoes of Wisdom ~7.8 GB +donkey-kong-country-returns-hd.png Donkey Kong Country Returns HD ~11.6 GB +mario-rabbids-sparks-of-hope.jpg Mario + Rabbids Sparks of Hope ~9.8 GB +xenoblade-chronicles-2.jpg Xenoblade Chronicles 2 ~17.1 GB +xenoblade-chronicles-3.jpg Xenoblade Chronicles 3 ~19.2 GB From a0f0b3e45b0403d4d1fc130db599b9bc66918abc Mon Sep 17 00:00:00 2001 From: seedds Date: Tue, 27 Oct 2026 22:38:45 +0800 Subject: [PATCH 2/3] Reject conflicting gallery identities after union merges The union driver can retain both rows when one branch edits a game's caption or screenshot while another updates its IPA size. Checking only captions lets renamed games appear twice with every docs check passing. Validate unique images, captions and non-empty title IDs in the gallery reader before regenerating README, and exercise the actual union merge for each identity in the regression test. Validation: eight docs tests pass on Python 3.14; the three identity regressions fail against the original generator; concurrent append and README conflict recovery passes; both generated docs checks pass; the pinned lint ratchet has zero new or stale findings. This only changes documentation generation and its tests; no game runtime paths are touched. Co-authored-by: v0lumehi --- dm/tools/docs_gallery.py | 7 +++++++ dm/tools/tests/test_docs.py | 22 +++++++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/dm/tools/docs_gallery.py b/dm/tools/docs_gallery.py index c66ea51b20..53db0dea09 100644 --- a/dm/tools/docs_gallery.py +++ b/dm/tools/docs_gallery.py @@ -17,6 +17,7 @@ def rows(): out = [] + seen = {'image': {}, 'name': {}, 'title ID': {}} for n, line in enumerate(TABLE.read_text(encoding='utf-8').splitlines(), 1): if not line.strip() or line.startswith('#'): continue c = line.split('\t') @@ -25,6 +26,12 @@ def rows(): if not (REPO / 'docs' / 'images' / image).is_file(): sys.exit(f'{TABLE.name}:{n}: no image docs/images/{image}') if any(ch in name + note for ch in '"<>&'): sys.exit(f'{TABLE.name}:{n}: no " < > & in a name or note (they go into HTML as they are)') if tid and not re.fullmatch(r'[0-9A-F]{16}', tid): sys.exit(f'{TABLE.name}:{n}: a title ID is 16 hex digits: {tid}') + # A union merge can retain two different captions for the same game, so check its image and title ID too. + for field, value in (('image', image), ('name', name), ('title ID', tid)): + if not value: continue + if value in seen[field]: + sys.exit(f'{TABLE.name}:{n}: duplicate {field} {value!r} (first at line {seen[field][value]}): merge the rows by hand') + seen[field][value] = n out.append((image, name, note, tid)) return out diff --git a/dm/tools/tests/test_docs.py b/dm/tools/tests/test_docs.py index df35cc803f..b738d0cece 100644 --- a/dm/tools/tests/test_docs.py +++ b/dm/tools/tests/test_docs.py @@ -6,7 +6,9 @@ import re import subprocess import sys +import tempfile import unittest +from unittest.mock import patch from urllib.parse import unquote REPO = Path(__file__).resolve().parents[3] @@ -89,9 +91,23 @@ class Gallery(unittest.TestCase): def test_gallery_is_current(self): self.assertEqual(docs_gallery.OUT.read_text(encoding='utf-8'), docs_gallery.readme(), 'README.md: the gallery is out of date: run dm/tools/docs_gallery.py') - def test_each_game_once(self): # a union merge of two rows for the same game keeps both - names = [r[1] for r in docs_gallery.rows()] - self.assertEqual(sorted({n for n in names if names.count(n) > 1}), [], 'docs/images/gallery.tsv: a game is in the table twice') + def test_union_merge_rejects_conflicting_game_rows(self): + first, second = docs_gallery.rows()[:2] + # Concurrent caption/screenshot and size edits must require a manual resolution even when the names differ. + for field, changes in (('image', {1: 'Corrected caption'}), ('name', {0: second[0]}), + ('title ID', {0: second[0], 1: 'Corrected caption'})): + with self.subTest(field=field), tempfile.TemporaryDirectory() as folder: + base = [first[0], first[1], first[2], '0100000000000000' if field == 'title ID' else ''] + ours, theirs = base.copy(), base.copy() + for i, value in changes.items(): ours[i] = value + theirs[2] = '~2 GB' + paths = [Path(folder) / f'{side}.tsv' for side in ('ours', 'base', 'theirs')] + for path, row in zip(paths, (ours, base, theirs)): path.write_text('\t'.join(row) + '\n', encoding='utf-8') + result = subprocess.run(['git', 'merge-file', '--union', '-p', *map(str, paths)], check=True, capture_output=True, text=True) + merged = Path(folder) / 'merged.tsv' + merged.write_text(result.stdout, encoding='utf-8') + with patch.object(docs_gallery, 'TABLE', merged), self.assertRaisesRegex(SystemExit, f'duplicate {field}'): + docs_gallery.readme() class Length(unittest.TestCase): From 3a78737de78bfad944c82fce544acd3d60b67d4e Mon Sep 17 00:00:00 2001 From: seedds Date: Wed, 28 Oct 2026 01:32:02 +0800 Subject: [PATCH 3/3] Preserve current gallery content and keep merge rules scoped Carry Final Fantasy Crystal Chronicles into the gallery source after rebasing onto main. Allow an unknown IPA size without an empty subcaption, escape plain-text captions and notes, and reject reversed gallery markers with a clear error before writing README. Keep union merging on the gallery TSV only. The separate reference-doc union proposal can silently combine contradictory handwritten prose that its duplicate-bullet check does not detect, so it is no longer a dependency of this PR. Document the gallery editing and conflict-recovery workflow. Check concurrent additions with a real Git merge and regeneration, retain the conflicting-identity regressions, and cover plain text and missing sizes. Ten docs tests pass; the full tool suite passes 384 tests with 16 skipped outside the sandbox. Pinned lint reports 405 baseline findings, zero new and zero stale. README matches main byte for byte except for its two generator markers. No game runtime code changes. Co-authored-by: v0lumehi --- CONTRIBUTING.md | 1 + dm/tools/docs_gallery.py | 13 ++++---- dm/tools/tests/test_docs.py | 59 +++++++++++++++++++++++++++++++++++++ docs/agents/06-dev-loop.md | 2 +- docs/images/gallery.tsv | 3 +- 5 files changed, 71 insertions(+), 7 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b6be155cf5..3c9c36623a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,6 +22,7 @@ Never commit or attach game files, keys, title keys or anything extracted from a - **One change per PR, off `main`.** Rebase on `main` rather than merging it in. - **Commits.** Each commit message says what changed and why, in plain English sentences, as the history does: the symptom, the cause, the fix, and what was measured or checked. - **Where findings go.** A fact about one game goes in its note (`dm/games//README.md`). A general mechanism gets an entry in `docs/reference/.md` (Symptom, Cause, Fix, Found on), followed by `python3 dm/tools/docs_symptoms.py`. A new game takes its profile, its note and its row in the games table ([dm/games/README.md](dm/games/README.md)). +- **The README gallery.** After Mac gameplay testing, add its screenshot to `docs/images/` and its row to [gallery.tsv](docs/images/gallery.tsv), then run `python3 dm/tools/docs_gallery.py`. The table preserves your chosen order; notes hold the IPA size and noteworthy features or limitations. Leave the note empty until the IPA size is measured. Concurrent additions merge in the TSV; regenerate README to resolve a conflict inside its gallery. Conflicting rows for one game need a manual resolution before regeneration. - **The shader chain.** After changing one of its tools, run `python3 dm/gfx/tools/dmsh_chain.py --header` and commit `dm/gfx/nvn/dmsh_chain.h` with the change. Every game's shaders are then rebuilt, so say which games you checked. - **Tests.** CI runs the host tests, the CPU-only graphics tests, the tool tests and the builds ([Continuous integration](docs/development.md#continuous-integration)). GitHub's runners have no Metal 4, so run the GPU tests on an Apple-silicon Mac: `make -C dm/gfx/tests GAME= run`. - **The PR text.** Fill in the template: what and why, what was checked (tests, games, devices), and which docs changed. diff --git a/dm/tools/docs_gallery.py b/dm/tools/docs_gallery.py index 53db0dea09..4944caa2f9 100644 --- a/dm/tools/docs_gallery.py +++ b/dm/tools/docs_gallery.py @@ -2,10 +2,10 @@ # Writes the games gallery in README.md (between its gallery markers) from docs/images/gallery.tsv: one cell per row of the table, three to a row, in the table's order. # The table is the source: a new game is a row in it (usually at the end), so two pull requests that each add a game change one line each instead of moving every cell after # the first one; .gitattributes merges the table with the union driver. Columns: image (a file in docs/images), name (the caption and the image's alt text), note (the small -# line under it: the IPA's size, then " · " and a remark if any) and, optionally, the base game's title ID, which links the name to the game's note (dm/games//README.md). +# line under it: the IPA's size, then " · " and a remark if any; omit when the size is unknown) and, optionally, the base game's title ID, which links to its game note. # Run it after changing the table; --check only reports whether README.md is current (dm/tools/tests/test_docs.py runs it). # Usage: docs_gallery.py [--check] -import pathlib, re, sys +import html, pathlib, re, sys REPO = pathlib.Path(__file__).resolve().parents[2] TABLE = REPO / 'docs' / 'images' / 'gallery.tsv' @@ -21,10 +21,9 @@ def rows(): for n, line in enumerate(TABLE.read_text(encoding='utf-8').splitlines(), 1): if not line.strip() or line.startswith('#'): continue c = line.split('\t') - if len(c) not in (3, 4): sys.exit(f'{TABLE.name}:{n}: 3 or 4 columns expected, {len(c)} found') + if len(c) not in (2, 3, 4): sys.exit(f'{TABLE.name}:{n}: 2 to 4 columns expected, {len(c)} found') image, name, note, tid = c + [''] * (4 - len(c)) if not (REPO / 'docs' / 'images' / image).is_file(): sys.exit(f'{TABLE.name}:{n}: no image docs/images/{image}') - if any(ch in name + note for ch in '"<>&'): sys.exit(f'{TABLE.name}:{n}: no " < > & in a name or note (they go into HTML as they are)') if tid and not re.fullmatch(r'[0-9A-F]{16}', tid): sys.exit(f'{TABLE.name}:{n}: a title ID is 16 hex digits: {tid}') # A union merge can retain two different captions for the same game, so check its image and title ID too. for field, value in (('image', image), ('name', name), ('title ID', tid)): @@ -37,8 +36,11 @@ def rows(): def cell(image, name, note, tid): + # Attributes use double quotes; apostrophes in existing captions need no escaping. + image, name, note = (html.escape(s, quote=False).replace('"', '"') for s in (image, name, note)) caption = f'{name}' if tid else name - return f'{name}
{caption}
{note}' + detail = f'
{note}' if note else '' # a game whose IPA size is still open has only its caption + return f'{name}
{caption}{detail}' def gallery(): @@ -54,6 +56,7 @@ def readme(): text = OUT.read_text(encoding='utf-8') if text.count(BEGIN) != 1 or text.count(END) != 1: sys.exit(f'README.md: the gallery markers ({BEGIN} … {END}) must appear once each') head, rest = text.split(BEGIN) + if END not in rest: sys.exit('README.md: the gallery end marker must follow the begin marker') return head + BEGIN + '\n' + gallery() + '\n' + END + rest.split(END, 1)[1] diff --git a/dm/tools/tests/test_docs.py b/dm/tools/tests/test_docs.py index b738d0cece..84adf9d483 100644 --- a/dm/tools/tests/test_docs.py +++ b/dm/tools/tests/test_docs.py @@ -91,6 +91,65 @@ class Gallery(unittest.TestCase): def test_gallery_is_current(self): self.assertEqual(docs_gallery.OUT.read_text(encoding='utf-8'), docs_gallery.readme(), 'README.md: the gallery is out of date: run dm/tools/docs_gallery.py') + def test_plain_text_captions_and_unknown_size(self): + image = docs_gallery.rows()[0][0] + with tempfile.TemporaryDirectory() as folder: + table = Path(folder) / 'gallery.tsv' + table.write_text(f'{image}\tA & B: "Adventure" <2>\n', encoding='utf-8') + with patch.object(docs_gallery, 'TABLE', table): + text = docs_gallery.gallery() + self.assertIn('alt="A & B: "Adventure" <2>"', text) + self.assertIn('
A & B: "Adventure" <2>', text) + self.assertNotIn('', text) + self.assertIn('~2 GB & more', docs_gallery.cell(image, 'Game', '~2 GB & more', '')) + + def test_reversed_markers_fail_before_writing(self): + with tempfile.TemporaryDirectory() as folder: + out = Path(folder) / 'README.md' + text = f'{docs_gallery.END}\nKeep this text\n{docs_gallery.BEGIN}\n' + out.write_text(text, encoding='utf-8') + with patch.object(docs_gallery, 'OUT', out), self.assertRaisesRegex(SystemExit, 'end marker must follow'): + docs_gallery.readme() + self.assertEqual(out.read_text(encoding='utf-8'), text) + + def test_concurrent_additions_merge_and_regenerate(self): + games = docs_gallery.rows()[:5] + with tempfile.TemporaryDirectory() as folder: + root = Path(folder) + table, out = root / 'docs/images/gallery.tsv', root / 'README.md' + table.parent.mkdir(parents=True) + (root / '.gitattributes').write_bytes((REPO / '.gitattributes').read_bytes()) + + def git(*args, check=True): + return subprocess.run(['git', '-c', 'user.name=seedds', '-c', 'user.email=seedds@gmail.com', + '-c', 'commit.gpgSign=false', '-c', 'core.hooksPath=/dev/null', *args], + cwd=root, check=check, capture_output=True, text=True) + + def write(rows): + table.write_text(''.join('\t'.join(row).rstrip('\t') + '\n' for row in rows), encoding='utf-8') + out.write_text(f'Before\n{docs_gallery.BEGIN}\n{docs_gallery.END}\nAfter\n', encoding='utf-8') + with patch.object(docs_gallery, 'TABLE', table), patch.object(docs_gallery, 'OUT', out): + out.write_text(docs_gallery.readme(), encoding='utf-8') + git('add', '.') + git('commit', '-m', 'Gallery fixture') + + git('init', '-b', 'base') + write(games[:3]) + git('switch', '-c', 'ours') + write(games[:4]) + git('switch', '-c', 'theirs', 'base') + write(games[:3] + games[4:]) + self.assertEqual(git('merge', '--no-commit', '--no-ff', 'ours', check=False).returncode, 1) + self.assertIn('<<<<<<<', out.read_text(encoding='utf-8')) + self.assertEqual(git('diff', '--name-only', '--diff-filter=U').stdout.strip(), 'README.md') + with patch.object(docs_gallery, 'TABLE', table), patch.object(docs_gallery, 'OUT', out): + self.assertCountEqual(docs_gallery.rows(), games) + text = docs_gallery.readme() + self.assertNotIn('<<<<<<<', text) + self.assertTrue(text.startswith(f'Before\n{docs_gallery.BEGIN}\n')) + self.assertTrue(text.endswith(f'\n{docs_gallery.END}\nAfter\n')) + self.assertEqual(text.count('/reports/`): one By what the change touches (the largest that applies): -- **Docs only:** `python3 -m unittest dm/tools/tests/test_docs.py`; after a change to `docs/reference`, `python3 dm/tools/docs_symptoms.py`. +- **Docs only:** `python3 -m unittest dm/tools/tests/test_docs.py`; after a change to `docs/reference`, `python3 dm/tools/docs_symptoms.py`; after editing `docs/images/gallery.tsv`, `python3 dm/tools/docs_gallery.py` (also resolves conflicts inside README's generated gallery). - **One game's profile or note** (`dm/games//`): the Python tool tests; a full shader build of that game after a change to its shader inputs; the game on the Mac (or the iPhone) along its route, with `scripts/bench.py` numbers when the change is about speed; the finding in the game's note ([AGENTS.md](../../AGENTS.md)). - **The host, the chain or the tools** (`dm/host`, `dm/gfx`, `dm/tools`, `dm/gfx/tools`, `dm/ios`, `scripts`): everything above, `make -C dm/gfx/tests GAME= run`, `dmsh_chain.py --header` after a chain change, `python3 scripts/regress.py impact main..HEAD` to see which games it reaches, then `python3 scripts/regress.py check --games ` (`--quick` while iterating, the full run before the pull request), and its `report.md` pasted into the pull request with a word on every `CHANGED`, `NEW-FAIL` or `REGRESSION`. Choose the games, not every local one: the five the change is most likely to reach or break (those `impact` names, then games that cover the APIs, engines and paths it touches). Take more than five only when the change truly needs them (a broad chain or host change no five games cover), and fewer when its reach is narrow and clear (a fix on one game's path, a tool only some games use). Name the games the report covers and why you chose them: the others are unchecked. diff --git a/docs/images/gallery.tsv b/docs/images/gallery.tsv index 5fb0e5d79f..17371c0852 100644 --- a/docs/images/gallery.tsv +++ b/docs/images/gallery.tsv @@ -1,5 +1,5 @@ # The games gallery in README.md, in the order it shows them (dm/tools/docs_gallery.py writes it from this table). A new game: its screenshot in docs/images and a row here, -# usually at the end. Columns: image, name (caption and alt text), note (the IPA size, then " · " and a remark if any), and optionally the title ID that links the name to its note. +# usually at the end. Columns: image, name (caption and alt text), optional note (IPA size, then " · " and a remark), and optionally the title ID that links the name to its note. mario-kart-8-deluxe.jpg Mario Kart 8 Deluxe ~11.9 GB · no Mii heads mario-vs-donkey-kong.jpg Mario vs. Donkey Kong ~1.6 GB paper-mario-the-thousand-year-door.jpg Paper Mario: The Thousand-Year Door ~5 GB · 60 fps with 1.0.1 @@ -80,3 +80,4 @@ donkey-kong-country-returns-hd.png Donkey Kong Country Returns HD ~11.6 GB mario-rabbids-sparks-of-hope.jpg Mario + Rabbids Sparks of Hope ~9.8 GB xenoblade-chronicles-2.jpg Xenoblade Chronicles 2 ~17.1 GB xenoblade-chronicles-3.jpg Xenoblade Chronicles 3 ~19.2 GB +final-fantasy-crystal-chronicles.jpg FINAL FANTASY CRYSTAL CHRONICLES Remastered Edition