Skip to content

feat: supplement JSON docs with XML documentation from Godot source#1

Merged
deevus merged 18 commits intomainfrom
feature/xml-doc-supplementation
Mar 21, 2026
Merged

feat: supplement JSON docs with XML documentation from Godot source#1
deevus merged 18 commits intomainfrom
feature/xml-doc-supplementation

Conversation

@deevus
Copy link
Copy Markdown
Contributor

@deevus deevus commented Mar 21, 2026

Summary

  • Automatically fetches Godot XML documentation from the source tree (via GitHub tarball), streaming HTTP → gzip → tar extraction without writing the full archive to disk
  • Parses XML docs with zig-xml to extract tutorials, descriptions, methods, properties, signals, and constants
  • Merges XML data into the JSON-based DocDatabase: fills missing descriptions, adds tutorial links, and creates entries for XML-only classes
  • Adds GDOC_NO_XML env var to disable XML supplementation (set automatically during zig build test)

How it works

  1. On first run (cache empty), after generating the JSON dump, runs godot --version to get the version/commit hash
  2. Downloads the source tarball from GitHub (refs/tags/{version}-stable.tar.gz, with commit hash fallback)
  3. Streams through gzip decompression and tar extraction, filtering for */doc/classes/*.xml and */modules/*/doc_classes/*.xml
  4. Writes ~800 XML files (~5 MB) to ~/.cache/gdoc/xml_docs/ with a .complete version marker
  5. Parses each XML file and merges supplemental data into the DocDatabase before generating the markdown cache
  6. $DOCS_URL in tutorial links is expanded to https://docs.godotengine.org/en/stable

Godot must be installed. If godot --version fails or the download fails, XML supplementation is silently skipped and gdoc works with JSON-only data. --clear-cache clears everything including XML docs.

New files

  • src/source_fetch.zig — Version parsing, tarball URL building, HTTP download, streaming tar extraction, cache markers
  • src/XmlDocParser.zig — Godot XML doc parser using zig-xml pull parser

Test plan

  • zig build test passes (~0.6s, no network I/O)
  • gdoc --clear-cache && gdoc Node2D shows tutorials section from XML docs
  • gdoc Node2D.position shows property documentation
  • GDOC_NO_XML=1 gdoc --clear-cache && gdoc Node2D works without tutorials (JSON-only)

deevus added 18 commits March 21, 2026 12:04
- Fix std.compress.gzip -> std.compress.flate reference
- Add fallback for when Godot is not installed
- Use tag-based tarball URL with hash fallback
- Add error handling section with graceful degradation
- Include modules/*/doc_classes/*.xml in tar filter
- Add .complete marker for staleness/partial download detection
- Specify Tutorial struct type and $DOCS_URL expansion
- Clarify merge happens during cache generation, not display time
- Note --godot-extension-api bypasses XML supplementation
10-task plan covering: zig-xml dependency, version parsing,
tarball download/extraction, XML parsing, Entry tutorials field,
cache integration, XML merge into DocDatabase, and e2e verification.
Add GDOC_NO_XML environment variable to disable XML doc supplementation.
Set automatically in build.zig for test runs to avoid network I/O
during zig build test.
@deevus deevus merged commit 43fd8aa into main Mar 21, 2026
1 of 2 checks passed
@deevus deevus deleted the feature/xml-doc-supplementation branch March 21, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant