Skip to content

coro: add the stackless preemptive runtime core - #2158

Closed
cpunion wants to merge 324 commits into
xgo-dev:mainfrom
cpunion:llvm-coro
Closed

coro: add the stackless preemptive runtime core#2158
cpunion wants to merge 324 commits into
xgo-dev:mainfrom
cpunion:llvm-coro

Conversation

@cpunion

@cpunion cpunion commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

This is the integrated compiler/runtime core for #1546. It keeps ordinary Go source and standard-library calls synchronous in syntax, infers suspension transitively, and lowers the selected functions to LLVM stackless coroutines.

The implementation deliberately does not generate two complete bodies for every function. A source function has one primary body; an additional callable/dispatch representation is emitted only when a value can cross an open dynamic boundary such as any, an interface, a function value, reflection, or foreign code.

What this adds

  • A whole-program effect/demand plan with canonical function identity, frozen per-site lowering facts, cache digests, archive validation, and fail-closed dynamic-call handling.
  • A structured LLVM coroutine builder and physical ABI for LLVM 19–22, including frame allocation/root retention, child await, preemption safepoints, explicit panic outcomes, defer/recover cleanup, and dynamic callable dispatch.
  • Compiler-injected callable metadata for C/assembly/host functions. Blocking behavior is never recovered from a raw function address; exact wrappers and call sites carry content-addressed contracts into the plan.
  • One runtime scheduling model built around G, P, executor ownership, OperationID, ParkState, WaitSetRecord, result leases, select winner claims, cancellation, detach, and producer quiescence.
  • A shared event-source model for timer, poll, worker, channel, semaphore, and notify waits, including lossless idle admission and bounded run slices.
  • A native two-domain executor fleet with exact routed wakeups, a bounded worker pool for blocking syscalls/files, poll/deadline integration for sockets, and strong shutdown/join semantics.
  • Target-neutral host-pull adapters and cross-target compilation coverage for JS/WASM, WASI, embedded, and bare-metal environments without requiring libuv or BDWGC in the scheduler core.
  • Go 1.26 synchronous-source acceptance probes that compile, link, and run through LLGo for time.Sleep, time.Timer/Ticker, direct file syscalls, os.File, and loopback TCP with deadlines.

Architectural cutover

This branch has completed the planned hard cutover for the core implementation:

  • no staged EnableCoro* feature surface;
  • no legacy logical wait registration or Timer/Poll dual modes;
  • one static source dispatcher and one native fleet path;
  • one frozen physical plan for helper discovery, preflight, emission, and cache identity;
  • zero-tolerance architecture gates prevent the removed protocols from returning.

The design and invariants are documented in:

  • doc/llvm-coro-runtime-design.md
  • doc/coro-ir-design.md
  • doc/coro-async-core-contract.md
  • doc/coro-callable-contract.md

Validation

Validated locally with Go 1.26.5, LLVM 19.1.7, Darwin/arm64:

  • go test ./ssa -count=1 — pass (109 s)
  • go test ./cl -count=1 -timeout=20m — pass (1106 s)
  • go test ./internal/build -count=1 -timeout=20m — pass (582 s), including the five standard-library acceptance programs
  • runtime core, allocator, doorbell, fleet, channel, keyed wait, and timer tests under -race -shuffle=on
  • JS/WASM runtime adapter and channel execution
  • JS/WASM, WASIp1, Linux ARM/RISC-V, and Cortex-M/bare-metal cross-compilation
  • all coroutine architecture/hard-cutover gates
  • all repository coroutine vet lanes

The dedicated Coroutine workflow adds LLVM 19, 20, 21, and 22 jobs, plus integration and target lanes. The regular upstream workflows are re-enabled for this integrated PR.

Scope boundaries

This PR lands the reusable core and proven vertical slices; it does not claim that every target or every GOROOT test is complete. Follow-up work remains for dynamic P sizing/work stealing, full timer GC/synctest/asynctimerchan semantics, broader FD/process/signal coverage, the ordinary JS/WASI command reactor pump, RTOS/bare-metal HAL adapters, precise moving-GC integration, and the remaining open dynamic FFI/tooling surface.

LLVM dependency

go.mod temporarily replaces github.com/xgo-dev/llvm with github.com/cpunion/llvm@bf5fb88be315 for the LLVM 22 bindings used by the matrix. The corresponding upstream proposal was xgo-dev/llvm#43; this replacement can be removed once equivalent bindings are available upstream.

Part of #1546.

cpunion added 30 commits July 16, 2026 17:16
feat(coro): add runnable stackless scheduler and panic prototype
runtime(coro): add stable wait registration and cancellation
cpunion added 20 commits July 22, 2026 12:49
Make one ProgramIR call-site plan authoritative for intrinsic semantics, no-init/patch/intrinsic elision, and worker certificates. Route analysis, preflight, and physical emission through the frozen plan, retire builder scratch after freeze, and fail closed on actual opcode/recipe/elision mismatches. Extend exact architecture gates and remove the helper planner LLSSA runtime-type dependency exposed by integration CI.
coro: integrate stackless async runtime and scheduler prototype
# Conflicts:
#	cl/compile.go
#	go.sum
#	internal/build/build.go
#	internal/build/collect.go
#	internal/build/fingerprint.go
#	internal/build/main_module.go
#	internal/crosscompile/crosscompile_test.go
#	internal/plan9asm/indexbyte_return_test.go
#	runtime/internal/lib/reflect/type.go
#	runtime/internal/lib/runtime/_wrap/runtime.c
#	ssa/abitype.go
#	ssa/stmt_builder.go
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cpunion

cpunion commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

Keeping this experimental coroutine line in cpunion/llgo for now. The upstream CI result has been recorded; development and validation will continue in the fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant