Skip to content

[Based on #2090] runtime: introduce G/M/P pthread backend#2166

Open
cpunion wants to merge 4 commits into
xgo-dev:mainfrom
cpunion:codex/runtime-gmp-pthread-backend
Open

[Based on #2090] runtime: introduce G/M/P pthread backend#2166
cpunion wants to merge 4 commits into
xgo-dev:mainfrom
cpunion:codex/runtime-gmp-pthread-backend

Conversation

@cpunion

@cpunion cpunion commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Depends on #2090

Problem

The compiler currently lowers every go statement directly to pthread setup. That couples goroutine code generation to one OS backend and provides no runtime-owned G/M/P boundary for a future scheduler.

Change

  • extend runtime: store goroutine state behind pthread-key getg #2090's pthread-key g into minimal Go-compatible G, M, and P objects;
  • retain the current 1:1 model: one detached pthread, one G, one M, and one P per goroutine;
  • make the compiler emit only a startup record and call runtime.NewProc;
  • move pthread creation, optional stack-size setup, startup, and cleanup behind newproc1, newm, mstart, and mexit;
  • route normal return and runtime.Goexit through the same cleanup path;
  • keep the G/M/P context in one GC root, released by mexit for runtime-created goroutines and by the pthread destructor for bootstrap/external threads;
  • publish P status atomically on threaded targets while retaining direct bare-metal access without compiler-rt atomic dependencies.

This establishes the replaceable native backend boundary proposed in #2152. It does not add M:N scheduling, preemption, growable stacks, LLVM coroutines, or runtime assembly reuse.

The independent diff is +622/-181 across 15 files. It contains no #2079 locality compiler/runtime files.

Validation

Tests were serialized with GOMAXPROCS=2, GOMEMLIMIT=8GiB, and -p=1.

  • macOS arm64, Go 1.26.5: complete ./ssa, runtime compilation, and real-LLGo G/M/P tests pass.
  • Ubuntu arm64, Go 1.26.5, LLVM 19.1.7: compiler/runtime and real-LLGo tests pass in an 8 GiB, 2 CPU, no-swap, 512-PID container.
  • Ownership tests cover parent IDs, unique G/M/P IDs, running states, bidirectional links, getg isolation, and defer isolation.
  • Existing goroutine and select IR fixtures pass.
  • No test is skipped or ignored.

Reusing pinned GOROOT runtime and assembly code remains tracked separately in #2165.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: runtime G/M/P pthread backend

Overall this is a solid, well-structured change. The runtime side is clean and, if anything, a small improvement: go lowering now emits a single runtime.NewProc call, and newproc1 does one combined AllocRoot for the G/M/P context. The runtimeContext lifecycle (alloc → mexitFreeRoot), the mutually-exclusive arg-free paths, and the EnterLocalContext/LeaveLocalContext pairing are internally consistent. The switch of the FuncForPC/symtab caches to atomic pointer publication is the correct call now that goroutines run on real threads. No blocking memory-safety defect found.

The actionable items below are all compile-time performance and maintainability concerns — none block correctness. Inline comments carry the specifics; a couple of non-inline notes follow.

Non-inline notes (low severity / nits):

  • runtime/internal/runtime/os_pthread.go header still reads Copyright (c) 2024 (carried over from the z_thread.go rename); every other new/rewritten file in this PR uses 2026.
  • runtime/internal/runtime/os_pthread.go newosproc ignores the return of attr.Destroy(). Low risk and conventional (the create-critical returns — Init/SetDetached/SetStackSize/Create — are all checked), noting only for completeness.
  • internal/build/main_module.go defineLibraryRuntimeInit: the doc block describes only the global_ctors registration but the function now also conditionally emits the enterLibraryLocalContext call when Prog.NeedsLocalContext() is true. Worth extending the comment, since that call is what lets exported Go calls on a C constructor thread inherit a context.

Comment thread ssa/locality.go Outdated
Comment thread ssa/locality.go Outdated
Comment thread runtime/internal/runtime/local_context.go Outdated
Comment thread cl/locality_lower.go Outdated
Comment thread runtime/internal/runtime/proc.go Outdated
Comment thread runtime/internal/runtime/local_context.go Outdated
@cpunion
cpunion force-pushed the codex/runtime-gmp-pthread-backend branch from 2cd48fd to 47035fc Compare July 23, 2026 10:49
@cpunion

cpunion commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the non-inline review notes as well:

  • updated os_pthread.go copyright to 2026;
  • made every pthread attribute Destroy result explicitly ignored, with a comment explaining why a post-Create destroy failure cannot be surfaced as thread-creation failure;
  • extended defineLibraryRuntimeInit documentation in parent runtime: store goroutine state behind pthread-key getg #2090 to include the conditional process-lifetime locality context.

Validation on the final head:

  • macOS/arm64 and Linux/amd64 runtime execution tests pass, including a 2 MB pthread stack;
  • WASI, JS/Wasm, and ESP32-C3 cross-builds pass;
  • full repository coverage run passes at 75.61% (16,539 / 21,873 statements; ssa 92.0%, cl 97.0%).

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cpunion
cpunion force-pushed the codex/runtime-gmp-pthread-backend branch from 47035fc to 0c415b2 Compare July 24, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant