Skip to content

Revisit Julia signal preservation with stable Julia 1.14 #610

Closed as not planned

Description

@jtravs

What happens

SEP 48 §5.4 proposes enabling Julia's signal handlers for threaded GC, then restoring the application's actual Python SIGINT and SIGTERM handlers. On macOS arm64 with Julia 1.13.3 and JuliaCall/PythonCall 0.9.37, restoring an ignored or custom SIGTERM handler does not prevent Julia from terminating the process.

This was reproduced on an Apple M5 Max on 2026-11-10, independently by two agents, while qualifying proposal #609 at 5fb24689c4874c374bad621e208c42b1b7175355. The minimal reproducer needs no solprop imports, explicit GC or numerical work. Both variants exit on signal 15 (shell status 143), despite Python reporting the restored handler. A separate custom-SIGINT probe passed. Restoring SIGTERM after the first threaded work also failed.

Why it matters

A custom or ignored SIGTERM can terminate the entire application and prevent checkpoint/output service, including while no Julia session is advancing. Closing a session does not remove this process-lifetime limitation.

On 2026-11-10, John Travers explicitly chose to keep Julia 1.13.3 and JuliaCall/PythonCall 0.9.37 in process, accepting this limitation for the optional Tier 3 backend. Proposal #609 has merged. The implementation preserves actual Python SIGINT handling and cooperative atomic stop tokens, documents fatal SIGTERM behavior, and retains explicit subprocess qualification. This issue no longer blocks proposal #609 or SEP 48's Final status under that accepted policy. It remains open for a later improvement; SEP 48's implementation must not close it.

Additional implementation evidence and accepted policy

During the accepted main-thread-only runtime matrix on macOS arm64 (Julia 1.13.3, JuliaCall/PythonCall 0.9.37), work/ignored/1 began fatal SIGTERM shutdown but remained alive for 180 seconds until an external supervisor sent SIGKILL. The original test remains failing and its evidence is retained. A bounded control reproduced a first-SIGTERM stall with solprop's Python owner-cleanup hook unregistered; a second SIGTERM terminated it promptly. Five companion controls exited on the first signal. These samples establish an intermittent failure, not a measured incidence rate or proof of an ownership-hook regression.

The pinned Julia fatal-signal path explicitly anticipates that asynchronous fatal exit may stall and escalates repeated exit requests. A second SIGTERM is not a substitute for an external hard deadline.

On 2026-11-10, the maintainer explicitly accepted this broader limitation: supervisors that require bounded termination must enforce an external SIGKILL deadline. Cooperative cancellation and actual SIGINT handling retain their strict contracts. The maintainer then directed the numerical implementation to proceed using the experiment's core, deferring further runtime investigation until after implementation. ADR-078 and SEP 48 Part II record this change of policy and work order. Existing failures must remain visible; no failed first-TERM-exit test is claimed as passed. The revised supervisor qualification must distinguish prompt exit from forced termination and verify cleanup.

The revised main-thread macOS matrix passed all 36 non-SIGTERM cases with zero skips, including cancellation during threaded GC, ownership cleanup, SIGINT, spawning, native coexistence and Metal loading. Linux runtime qualification is incomplete; one isolated SIGTERM case passed, while the complete older-policy lane has failures awaiting later triage.

Reproduction

Prepare a Julia environment with PythonCall 0.9.37 and Python with juliacall 0.9.37. Set PYTHON_JULIACALL_EXE, PYTHON_JULIACALL_PROJECT and JULIA_DEPOT_PATH to that prepared installation, then run this script in fresh processes, first without arguments and then with custom:

import os
import signal
import sys
import time

os.environ["PYTHON_JULIACALL_THREADS"] = "2"
os.environ["PYTHON_JULIACALL_HANDLE_SIGNALS"] = "yes"
received = []


def handler(number, frame):
    received.append(number)


custom = sys.argv[1:] == ["custom"]
expected = handler if custom else signal.SIG_IGN
signal.signal(signal.SIGTERM, expected)
import juliacall
signal.signal(signal.SIGTERM, expected)
assert signal.getsignal(signal.SIGTERM) is expected
print("before SIGTERM; custom =", custom, flush=True)
os.kill(os.getpid(), signal.SIGTERM)
time.sleep(0.5)
assert received == ([signal.SIGTERM] if custom else [])
print("survived SIGTERM", flush=True)

Both actual runs printed the first checkpoint and terminated before the second, with subprocess.returncode == -15. Each had a 30-second timeout and exited promptly. A separate two-thread initialization probe with PYTHON_JULIACALL_HANDLE_SIGNALS=no returned from import but crashed with SIGBUS during shutdown; disabling handlers is not a qualified workaround.

Context

Follow-up when stable Julia 1.14 is available

  • Check compatible JuliaCall/PythonCall, CUDA, AMDGPU and Metal releases before changing the runtime pin; re-resolve the joint dependency graph and rerun the device qualification.
  • Evaluate the supported signal-interception mechanism in the released runtime. A newer cancellation API alone does not establish ownership of operating-system SIGTERM.
  • Preserve the application's actual default, custom and ignored SIGINT/SIGTERM dispositions, including changes after Julia initialization, without duplicate forwarding or asynchronous Julia interruption.
  • Qualify idle Python, GIL-free advancement, startup/shutdown, threaded GC and foreign-thread atomic cancellation in subprocesses on macOS arm64 and Linux x86-64.
  • Remove the intermittent fatal-shutdown stall, or prove a supported bounded termination contract; retain explicit supervisor escalation evidence while the limitation applies.
  • Independently review the lifecycle and update the manual, runtime tests and architectural decision if the limitation can be removed.

Done when

A supported runtime and compatible package stack demonstrably preserve the application's signal policy on both qualified CPU platforms, with no unintended process termination, crash or hang. Until then, the explicit SIGTERM limitation remains documented; no upgrade is assumed to fix it, and disabling Julia's signal handling is not an accepted workaround.

Activity

  1. added
    bugSomething isn't working
    priority: highWrong results, wrong public statements, or a user's first contact
    on Nov 10, 2026
  2. changed the title [-]Resolve Julia's SIGTERM ownership before freezing SEP 48's runtime policy[/-] [+]Revisit Julia signal preservation with stable Julia 1.14[/+] on Nov 10, 2026
  3. jtravs commented on Nov 12, 2026

    @jtravs
    ContributorAuthor

    Closing: the maintainer withdrew SEP 48 (the Julia backend) on 2026-11-11. C++/Kokkos stays the engine, and solprop has exactly two backends, native and numpy_ref (ADR-077, recorded in #763). This issue concerns only the Julia backend, its package, bridge or CI, so it no longer applies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: highWrong results, wrong public statements, or a user's first contact

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions