Proposal: Go-style funcvals with a C-compatible closure-context ABI

[cpunion]

Summary

LLGo should replace the current explicit closure-environment argument and __llgo_stub convention with a Go-style, one-pointer function-value representation and a target-neutral closure-context (CTXT) calling convention.

The design keeps LLGo's defining interoperability property: ordinary Go code entries use the target C ABI and ordinary Go functions can be passed directly as C callbacks. Stateful callables—closures and bound methods—carry their environment in a Go-style funcval; calls transport the funcval pointer through an LLVM-modelled hidden context parameter. Source-visible arguments and returns continue to use the target C ABI.

Dynamic calls should continue to use stock libffi as the platform ABI classifier. LLGo should implement an internal ffi_call_llgo path with a compiler-generated, per-signature invocation thunk on top of stock ffi_call; it should not require a patched libffi.

ABI mode 2 (allfunc) is the only mode that fully satisfies the C-compatible-function premise and should become the single supported production ABI. Modes 0 and 1 should remain temporarily as development and migration modes, with as much source-level closure compatibility as practical, but should not multiply the long-term ABI or DWARF support matrix.

This proposal turns the alternatives discussed in #1497 into an implementation direction. It also defines the interaction with the cross-platform debugging work in #2164.

Goals

• Represent a Go func value as one pointer to a Go-style function-value object.
• Keep ordinary Go function entries and raw C function pointers directly C-callable.
• Remove the explicit environment argument from source-visible/machine-visible argument registers on supported native targets.
• Remove the need for __llgo_stub wrappers for ordinary functions.
• Support closures, nested closures, bound concrete methods, bound interface methods, defer, goroutine entry, reflection, and C callbacks under one model.
• Keep libffi responsible for dynamic C ABI classification without forking or rebuilding libffi with reserved registers.
• Make the hidden context visible to LLVM so register allocation, optimization, unwind generation, and LTO preserve it correctly.
• Keep ABI lowering, optimization level, and DWARF emission logically independent.
• Reduce the tested ABI/optimization/DWARF cross-product to a small set of explicit profiles.

Non-goals

• Binary compatibility with the official Go internal ABI. LLGo remains C-ABI based.
• Making a stateful closure directly convertible to a C callback when the C API provides neither a user-data pointer nor a trampoline facility.
• Reimplementing libffi's per-platform aggregate and return-value classification.
• Reusing the official reflect.makeFuncStub assembly before LLGo has an equivalent and proven C-ABI contract.
• Stabilizing ABI modes 0 and 1 as externally linkable artifact formats.

Callable representations

The representation is selected statically; arbitrary pointers are never inspected at runtime to guess their kind.

Callable kind	Machine value	Call rule
Raw C function pointer	code address	direct target-C-ABI call; no context required
Ordinary Go function pointer	code address	same as a raw C function pointer
Ordinary function converted to Go func	pointer to a read-only funcval{code} cell	load code, set CTXT to the cell, call; target ignores CTXT
Go closure	pointer to funcval{code, captured data...}	load code, set CTXT to the funcval, call
Bound method	pointer to funcval{method-value stub, receiver/interface data...}	stub reads receiver data through CTXT, then directly calls the method
reflect.MakeFunc value	Go funcval whose entry is a C-ABI libffi closure or a mode-specific bridge	ordinary Go function-value call

A nil Go function value is a nil funcval pointer. The first word of every non-nil funcval is the code entry; captured values follow inline with compiler/runtime type metadata sufficient for GC scanning. This is closer to the official Go layout than LLGo's current two-word {code, env pointer} language value and leaves room for variable-sized environments.

The compiler should expose explicit operations internally rather than overload an untyped pointer:

• callRaw(code, args...)
• callFuncval(fv, args...)
• funcvalCode(fv) and funcvalContext(fv) for callback adapters

C callbacks

An ordinary Go function is already a target-C-ABI code pointer and may be registered directly as a C callback.

For a C callback API with a separate void *user_data, LLGo can generate an ordinary C-ABI adapter:

callbackAdapter(cArgs..., userData):
    fv = userData
    code = fv.code
    return call code with CTXT=fv and cArgs...

The callback pointer and userData stay separate, which matches common C APIs and requires no executable trampoline allocation.

For an API without user_data, a stateful closure requires an allocated trampoline such as a libffi closure. Ordinary functions still need no wrapper. Lifetime, executable-memory policy, and callback release remain explicit runtime concerns.

CTXT transport

The frontend should model closure context as a hidden LLVM IR parameter. It is hidden only at the machine C ABI: LLVM parameter attributes place it in a dedicated non-argument register, so visible C arguments retain their normal registers and stack locations.

The callee uses the formal CTXT SSA value directly and materializes any needed environment address before making unrelated calls. No global pendingFuncval state is needed, so nested calls, reentrancy, threads, and signals do not share a context slot.

Initial LLVM 19 mapping

The mapping is an internal compiler contract keyed by LLVM major version and target triple, not a permanent public register ABI.

Target	LLVM mechanism	LLVM 19 register	Notes
amd64 SysV, Darwin, Windows	nest	R10	caller-saved; visible C arguments unchanged
386 cdecl	nest	ECX	must not be combined with a calling convention that uses ECX for a visible argument
Linux arm64 where X18 is available	nest	X18	lower call overhead, but target policy must exclude platforms reserving X18
Darwin/Windows/Android arm64	swiftself	X20	LLVM-modelled and portable across these LLVM 19 triples; callee-saved save/restore cost
ARM AAPCS soft-float	nest	R12	validate per target ABI
ARM AAPCS-VFP	swiftself	R10	LLVM 19 does not provide the required nest lowering here
riscv32/riscv64	nest	T2/X7	caller-saved; visible C arguments unchanged
wasm and targets without a safe attribute mapping	explicit closure-only context parameter and typed adapter	n/a	do not pretend that Wasm has a native context register

nest is preferred where LLVM implements it on a safe caller-saved register. swiftself is the fallback where it provides a reliable non-argument register; its callee-saved convention adds prologue/epilogue work around indirect closure calls.

The implementation must never depend only on inline assembly that reads or writes a physical register. The parameter attribute must occur on both the definition and the call edge so LLVM models the register as a live-in/implicit operand. internal/cabi must preserve and remap nest/swiftself when it inserts sret, removes zero-sized parameters, or splits/coerces aggregates.

LLVM has changed some architecture mappings between releases—for example, AArch64 nest differs between LLVM 19 and newer releases. Code-generation tests must pin the expected mapping for the LLVM version shipped by LLGo. Runtime code must use the compiler's semantic CTXT abstraction rather than hard-code a register name.

Wasm fallback

Wasm has typed function references and no LLVM nest/swiftself register lowering. A closure target may therefore use an explicit context parameter behind a generated typed adapter while ordinary Go and C functions retain their normal Wasm C signature. This is target-specific lowering of the same funcval semantics, not a second language-level function-value layout.

The Wasm path must be tested with both single-threaded command-line/WASI execution and Emscripten/browser builds. A plain mutable global is not an acceptable long-term context transport for threaded or reentrant execution. Dynamic reflection on targets without libffi will require generated typed table-call thunks or an equivalent target runtime facility; that work can land separately from the native CTXT implementation.

Stock-libffi integration

libffi should remain LLGo's ABI oracle for runtime-known calls and callbacks, but it should not own CTXT transport.

For each compiled function signature, type metadata should make an invocation thunk available conceptually equivalent to:

invokeThunk(fv, args...):
    code = fv.code
    return call code with CTXT=fv and args...

ffi_call_llgo is an LLGo runtime operation built on public stock-libffi APIs:

ffi_call_llgo(type, fv, result, args):
    cif = C ABI signature of invokeThunk(type)
    stock ffi_call(cif, invokeThunk(type), result, [fv, args...])

libffi classifies and materializes the thunk's ordinary C ABI. The compiler and internal/cabi lower the typed thunk-to-target edge, including hidden CTXT, sret, byval, homogeneous aggregates, split values, and return coercions. Context does not have to survive an internal libffi call.

Raw C/ordinary-function reflection calls may use stock ffi_call directly. reflect.MakeFunc can continue to use stock ffi_closure; in canonical ABI mode 2 its generated code entry already has the C ABI and ignores CTXT. Runtime-created function types and mode 0/1 fallback behavior need dedicated tests before removing the existing path.

This avoids three fragile dependencies:

1. libffi's optional ffi_call_go, which is absent on important targets such as Darwin/Windows arm64 and Wasm;
2. coupling libffi's selected static-chain register to the exact LLVM version used by LLGo;
3. compiling a custom libffi that reserves LLGo's CTXT register in every caller.

Local feasibility prototypes with LLVM 19 have validated:

• ffi_call_go plus nest on Linux amd64 and arm64 for scalars, small aggregate returns, and sret returns;
• stock ffi_call into a typed C-ABI thunk, followed by a swiftself CTXT call, on Darwin arm64 for the same return classes.

The second result is the proposed portable foundation; the first is useful only as an optional target optimization.

ABI mode compatibility and migration

The current modes describe how internal/cabi rewrites LLVM signatures:

Mode	Current meaning	Compatibility under this proposal	Long-term status
0 / none	no C ABI rewrite	keep the one-pointer funcval layout and Go-to-Go closure/method semantics where possible; complex aggregate entries are not generally C-compatible	deprecated development mode
1 / cfunc	rewrite direct C symbols/calls only	keep the same funcval layout; generated C entry/callback thunks must be explicitly C-lowered; ordinary Go functions with aggregate signatures are not universally direct C callbacks	transitional diagnostic mode
2 / allfunc	rewrite every eligible function and call edge	full design: ordinary Go entries, callbacks, funcval targets, reflection thunks, and C calls share the target C ABI	canonical production mode

Modes 0 and 1 cannot fully meet the premise that every ordinary Go function has the C ABI. Compatibility should therefore mean:

• identical source-level behavior and function-value layout when practical;
• direct calls, closures, nested closures, method values, defer, and goroutine entry covered in all three modes during migration;
• scalar reflection/callback smoke coverage in modes 0 and 1;
• full aggregate, reflect.Call, reflect.MakeFunc, and C callback conformance required in mode 2;
• a clear unsupported result rather than silent ABI mismatch if a legacy mode cannot bridge a runtime-created signature safely.

Packages or bitcode compiled with different ABI modes must not be mixed. The existing build fingerprint already records ABI mode and should continue to prevent cache reuse across modes.

After the CTXT migration and a deprecation period, remove the user-selectable modes 0 and 1 (the -abi flag is already development-only). Keep narrow transformer tests that compare pre-CABI IR and full-CABI IR without exposing multiple production ABIs.

Alternatives

Approach	C ABI fidelity	Portability	Runtime cost	Optimizer safety	Maintenance	Decision
Current explicit environment parameter and __llgo_stub	visible signature differs; wrappers required	high	extra argument/wrapper	straightforward	duplicate entries and special FFI signatures	replace
Raw reserved register through inline asm	visible C ABI preserved	per-arch	low	fragile unless the register is a formal live-in	per-arch constraints	reject as primary mechanism
LLVM nest/swiftself plus typed thunk	visible C ABI preserved	native per-target mapping; explicit Wasm fallback	direct calls low; reflected closure call adds one thunk	modelled by LLVM	compiler mapping and tests	proposed
libffi ffi_call_go	visible C ABI preserved where implemented	missing/inconsistent on key targets	low	depends on LLVM/libffi agreeing on a register	toolchain-version coupling	optional optimization only
Patched libffi or libffi built with a reserved register	can be made correct	requires custom build everywhere	low	potentially sound	permanent fork/distribution burden	reject
TLS/global pendingFuncval	visible C ABI preserved	broad	store/load and TLS access	state is outside the call edge	reentrancy, signal, and thread hazards	emergency fallback only, not native design
Official-Go-style handwritten makeFuncStub ABI assembly	excellent for Go's internal ABI	assembly per architecture	low	proven for Go ABI	must reproduce every C ABI classification	defer; does not replace libffi today
Always allocate a libffi closure for a stateful funcval	C-callable code pointer	limited by executable-memory/libffi support	allocation and trampoline call	sound	lifetime/W^X complexity	use only when a C API has no user_data

DWARF and optimization

Today, enabling LLGo DWARF changes more than metadata retention: emitDebugInfo enables ssa.GlobalDebug, disables the LLVM pass pipeline, and disables C ABI lowering optimizations. This coupling contributes to failures such as #2118, #2121, and #2122. A debug flag must not select a different semantic ABI implementation.

The CTXT design adds further debug requirements:

• hidden CTXT parameters must be marked artificial and must not appear as source Go parameters;
• captured variables and bound receivers need locations derived from the funcval object;
• C ABI rewriting must remap parameter attributes and dbg.value/dbg.declare records after sret, byval, removal, splitting, and coercion;
• optimized builds need valid location lists after SROA, inlining, tail calls, and thunk folding;
• FFI invocation thunks need valid CFI/unwind data and should be marked artificial so debuggers can hide them without breaking stack walking;
• LTO must retain verifier-valid call locations and must not devirtualize a raw C/ordinary code pointer into an IR-incompatible CTXT call;
• -w=false must control DWARF emission/retention only, not passOpt, cabiOptimize, or ABI mode.

There are already 3 ABI modes × 6 optimization levels × DWARF on/off = 36 combinations before adding LTO mode, build mode, target, and architecture. Full debugger conformance for that cross-product is neither necessary nor sustainable.

Recommended supported profiles

User-facing flags remain independent, but CI and compatibility promises should normalize around these profiles:

Profile	ABI	Optimization	DWARF	LTO	Required guarantee
Native release	2	O2	current artifact policy; currently omitted by default	independently selected	full semantic/C ABI correctness
Size/embedded release	2	Oz (or the target default)	omitted or host-side per #2164	independently selected	full semantic/C ABI correctness
Debug	2	O0	enabled	off	source lines, parameters, locals, captures, receivers, unwind, reflection
Optimized debug	2	O2	enabled	off initially	same semantics; useful optimized locations after #2118/#2121/#2122 are fixed
Legacy compatibility smoke	0 and 1	O0 and O2	omitted	off	migration-only closure/direct-call behavior; not a public artifact ABI

The two primary public profiles are therefore ABI 2 + O2 release and ABI 2 + O0 + DWARF debug. ABI 2 + O2 + DWARF is the next explicit compatibility target so LLGo can eventually match Go's normal optimized-debug behavior. O1/O3/Os remain valid compiler choices and must never miscompile, but do not each receive a separate full debugger-conformance matrix. Targeted pairwise and IR-verifier tests are sufficient unless a regression requires promotion.

A Go-compatible debug invocation such as -gcflags=all=-N -l -ldflags=-w=false should resolve to the Debug profile. -w=false alone must not imply O0.

Implementation plan

1. Freeze the contract with tests

   • Add IR/codegen tests for every target CTXT attribute mapping.
   • Verify visible scalar, floating-point, small aggregate, and sret arguments match C ABI locations.
   • Add full-LTO tests for C/ordinary function pointers called through a Go funcval.

2. Introduce target-neutral CTXT lowering

   • Add a semantic closure-context parameter/call operation in SSA.
   • Teach internal/cabi to preserve/remap nest/swiftself and all existing parameter metadata.
   • Use an explicit closure-only context signature on Wasm/unsupported targets.

3. Migrate to one-pointer funcvals

   • Use {code, captured data...} objects for closures and bound methods.
   • Use static {code} cells for ordinary functions used as Go values.
   • Migrate nested closures, interface/concrete method values, defer, goroutine entry, panic/recover paths, and GC scanning.
   • Remove obsolete __llgo_stub entries and the explicit environment argument on supported native targets.

4. Add stock-libffi bridges

   • Implement ffi_call_llgo over stock ffi_call and per-signature typed invocation thunks.
   • Migrate reflect.Value.Call and validate reflect.MakeFunc.
   • Add generated C callback adapters for APIs with user_data; retain libffi closures for the no-user-data case.

5. Decouple and validate DWARF

   • Make DWARF emission independent of pass and C ABI optimization selection.
   • Establish the Debug profile first, then Optimized debug.
   • Mark CTXT/thunk metadata artificial while retaining captured-variable locations and unwind correctness.

6. Retire legacy ABI modes

   • Keep migration smoke tests while the implementation lands.
   • Deprecate and then remove development -abi=0/1 selection once mode 2 covers all supported targets.

The representation, CTXT lowering, libffi bridge, DWARF decoupling, and Wasm fallback should land as separate reviewable PRs even if they are tracked by this single proposal.

Acceptance criteria

• Ordinary Go functions are usable as raw C callbacks without a wrapper in ABI mode 2.
• Closures and bound methods are one-pointer funcvals and preserve captures/receivers under recursion, nesting, concurrency, and reentrancy.
• A user_data C callback adapter invokes a funcval correctly without modifying libffi.
• reflect.Value.Call and reflect.MakeFunc pass scalar, float, small-aggregate, large-aggregate/sret, variadic, and multi-result tests.
• Executable, c-archive, and c-shared tests execute callbacks and inspect returned information rather than only checking that symbols link.
• Native amd64/arm64 tests pass on Darwin and Linux; compile/codegen coverage protects 386, ARM, RISC-V, Windows, and Wasm target mappings.
• ABI mode 2 passes O0/O2/Oz semantic tests; modes 0/1 pass the documented migration subset.
• Debug profile tests can inspect arguments, captures, and bound receivers and unwind through adapters.
• Enabling or omitting DWARF does not change generated program behavior or C ABI lowering.
• The implementation uses an unmodified, system-distributed libffi on supported native platforms.