Skip to content

The first headless import no longer crashes, and the suite stops reporting green runs it did not earn - #297

Merged
Naruto merged 5 commits into
developfrom
fix/headless-suite-first-scan-crash
Aug 28, 2026
Merged

The first headless import no longer crashes, and the suite stops reporting green runs it did not earn#297
Naruto merged 5 commits into
developfrom
fix/headless-suite-first-scan-crash

Conversation

@Naruto

@Naruto Naruto commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

The first-scan crash is diagnosed and designed out, and three ways the suite
could report a pass it had not earned are fixed. Two of the three were found
while verifying the first.

The crash is Godot's, not godot-cpp's

EditorHelp::_gen_extensions_docs() dereferences the static DocTools
without a null check. The chain:

  1. An extension discovered mid-scan emits GDExtensionManager::extensions_reloaded
  2. EditorNode::_gdextensions_reloaded() calls EditorHelp::generate_doc(true, false)
  3. The worker thread's last act is to queue _gen_extensions_docs as a deferred call
  4. --import quits before the message queue is flushed
  5. EditorNode::~EditorNode() runs EditorHelp::cleanup_doc(), which joins that
    thread and then frees the DocTools
  6. The flush at the end of Main::cleanup() finally runs the deferred call

Not a race: every other thread is parked in condition_variable::wait when it
happens. The 0x8 in the fault address is class_list, the HashMap at offset
8 of a null DocTools.

run-tests.* now names the extension in .godot/extension_list.cfg before
starting Godot, so it loads at startup, extensions_reloaded never fires and
nothing is queued. The retry is gone: with the file seeded the import either
works or has failed for some other reason.

The earlier explanation -- godot-cpp master paired with Godot 4.7 -- was the
last suspect standing, not the culprit. godot-cpp is not in the stack at all.

An aborted case read as a pass

GDScript abandons a single function on a bad call and carries straight on, so a
case that died left no assertion, no skip and no failure behind and the suite
printed PASS. A case that records nothing at all is now a failure.

It caught its own first defect immediately: set_part_color_override was being
called with five arguments against a four-argument binding, so that case had
been asserting nothing.

only= was a hang

_only_filter() returned an untyped Array from a function declared
Array[String]. That aborts _init, the tree idles until --quit-after fires
about a minute later, and the run ends with no marker and a status of 0.

Verification

From a deleted test_gdextension/.godot, on Godot 4.7.2-stable:

== RESULT: 35 cases, 101 assertions, 0 failed, 0 skipped ==
==== SUITE FINISHED ====

Exit 0, no handle_crash. Assertions are up from 97 because the override case
now actually runs. The guard was confirmed by breaking that case again on
purpose: FAIL, exit 1, the case named in the failure list.

The diagnosis itself came from a dev_build=yes Godot, which is the only way
to get a symbolicated stack here -- the official build is signed with hardened
runtime so lldb cannot attach, and a stock local build is stripped.

Naruto added 5 commits August 28, 2026 16:36
The crash is Godot's own, and now that it is understood it can be avoided
outright rather than retried around.

EditorHelp::_gen_extensions_docs() dereferences the static DocTools without
a null check. Discovering an extension mid-scan emits extensions_reloaded,
which sends EditorNode off to regenerate the class reference on a worker
thread; that thread's last act is to queue _gen_extensions_docs as a
deferred call. --import quits before the message queue is flushed, so the
call lands on the flush at the end of Main::cleanup() -- by which time
EditorHelp::cleanup_doc() has freed the DocTools and set it to null.

Naming the extension in .godot/extension_list.cfg before Godot starts means
it is loaded at startup instead, extensions_reloaded never fires, and
nothing is queued. Godot rewrites that file itself, so seeding it decides
only the run in which it did not exist yet.

The retry is gone with it. It was a second run standing in for an
explanation; with the file seeded the import either works or has failed for
some other reason, and a crash is a crash.

Earlier commits blamed the pairing of godot-cpp master with Godot 4.7. That
was wrong -- godot-cpp is not in the stack at all.
GDScript answers a bad call -- the wrong argument count, a method that is
not there -- by printing SCRIPT ERROR and abandoning that one function, not
by stopping the run. The case then left no assertion, no skip and no failure
behind, and a suite whose failure count had not grown printed PASS. The
suite has been reporting a case it never ran as green.

A case that records nothing at all is now a failure, named as such, with a
pointer to the SCRIPT ERROR above it. The check is three counters read
before and after the call, so it costs nothing and cannot be forgotten by a
suite author.

only= took the runner down rather than filtering. _only_filter() returned an
untyped Array from a function declared Array[String], which is a runtime
error that aborts _init -- the tree then idles until --quit-after fires,
about a minute, and the run ends with no marker and a status of 0. It reads
as a hang. Array.assign() is the conversion that was wanted.

The comments explaining --quit-after said a script error inside a case
aborts _init. It does not, as above; corrected in all three files.
test_colour_and_cell_overrides_are_accepted_for_a_real_part passed a stray
1.0 between the blend operation and the priority, so every call in it failed
at the first line and the case asserted nothing. The bound signature is
(part_name, color, blend_op, priority) -- the rate the extra argument looks
like is the colour's own alpha.

Found by the check in the previous commit, which is what it is for.
The headless suite section blamed godot-cpp master paired with Godot 4.7,
because that was the last axis left after six other suspects were ruled out.
It was the wrong one: godot-cpp does not appear in the stack. The paragraph
now names the function, the deferred call and the flush that runs it too
late, and describes the seeding that avoids it instead of the retry that
used to absorb it.

Also states the counting rule the runner now enforces: a case that recorded
nothing at all is a failure, not a pass.
"Condition !actions.custom_samplers.has(...) is true. Continuing." shows up
once per player and reads like a defect. It is not one.

RendererDummy::MaterialStorage builds its shader compiler with a
default-constructed DefaultIdentifierActions, so custom_samplers is empty,
while the real canvas renderer fills it with TEXTURE and its siblings.
ss_blur.fs passes the builtin TEXTURE into ss_input_texture(), so headless
cannot resolve a sampler for it and says so through ERR_CONTINUE --
compilation still returns OK. The same suite under a real renderer prints
none of them, with the same 35 cases and 101 assertions.
@Naruto
Naruto merged commit 66446a4 into develop Aug 28, 2026
0 of 2 checks passed
@Naruto
Naruto deleted the fix/headless-suite-first-scan-crash branch August 28, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant