[Sanne-ai-bot]

Closes #2129

What

isx ssh-proxy could stop relaying a connection for 30 s in CI, which failed the strict ssh checks of test-ssh-host-keys.sh.
The cause is a lost wakeup in the native image: one of the JDK's virtual-thread pollers can park for good while sun.nio.ch.Poller is still being initialized, and every socket it serves then stays unread until something closes it.
Both binaries now make the JDK wake a virtual thread blocked on a socket from dedicated platform threads, not from virtual ones.
VirtualThreadPollers.usePlatformThreads() sets jdk.pollerMode=1 (SYSTEM_THREADS), first thing in IncusSpawn.run and ProxyMain.run; a mode already set with -D is kept.

Design

Why it stalled. In the failing run (job), Incus connected all four exec sockets by 10:30:31.357, and the guest's sshd was listening.
isx then read nothing for exactly 30 s, until the stdin socket's 30 s handshake watchdog (UnixSocketTransport.Watchdog) closed it at 10:31:01.357.
Its close unparks the reader from the closing thread (Poller.stopPoll), without a sub-poller, so the poller's readiness wakeup was what never came.

The cause is GraalVM's native image, not the JDK's pollers. On Linux, JDK 22+ defaults to VTHREAD_POLLERS, where readiness reaches a reader only through a virtual sub-poller that the platform MasterPoller unparks.

1. The first virtual thread to wait on a socket runs Poller's static initializer, which starts the sub-pollers before it returns.
2. A sub-poller starts on another carrier, registers with the master poller, then meets a class-initialization check for Poller and parks until the initializer is done.
3. In a native image that wait is a ReentrantLock condition (ClassInitializationInfo.initCondition), which parks through LockSupport and so uses the virtual thread's one park permit. HotSpot waits inside the VM and leaves the permit alone.
4. The initializer ends and unparks the sub-poller. The first thread registers its socket, and the master poller unparks the sub-poller too. When the sub-poller has not run in between, the permit is already set and the second unpark adds nothing.
5. The sub-poller resumes once, spends the permit on the class-initialization wait, and parks in Poller.poll for good: the master poller has taken it off its list and its one-shot epoll registration is spent.

Why CI. The window is the time the sub-poller waits for a CPU between the two unparks, so few or busy cores make it likely. Native isx ssh-proxy in the default mode, against the fake Incus, on a 32-CPU box:

Poller mode	CPUs allowed (taskset)	Runs	Stalls
default	32	15,000	0
default	2	15,000	12
default	1	10,000	121
platform (this PR)	1	30,000	0

A HotSpot JVM told it has one processor did not stall in 3,000 runs.

How it was found. At the stall, the stdin forwarder and its sub-poller are both parked in Poller.poll, the sub-poller has no permit and is off the master poller's list, and the scheduler has nothing queued.
A native build with tracing patched into Poller and VirtualThread then showed the order: the initializing thread's unpark, the master poller's unpark finding the permit set, one resume, and the park that never ends.
Two things are not established: which class-initialization check the sub-poller waits at (most likely the lambda call in subPollerLoop), and what delivered the banner in CI 5 s after the close.
JDK-8371045 is a different failure, where every carrier is pinned by class initialization.

Why the fix holds. Mode 1's Read-Poller and Write-Poller are platform threads that block in epoll_wait and unpark the reader themselves. No virtual thread polls, so none holds a registration while it waits for Poller's initializer. A woken reader still needs a carrier, in either mode.

Chosen: set the property at both entry points, before any socket work.
Alternatives: -Djdk.pollerMode=1 in the JVM launchers only, which does not reach the native binaries, since a native image does not read a runtime -D. Fewer virtual threads in the exec relay, a much larger change that leaves the proxy exposed.
Why: one line per binary covers the JVM and native builds alike. What it gives up is the scheduler-integrated polling, which neither binary needs: the proxy's traffic runs on Netty, and the CLI relays one connection at a time.

The stall reproduces outside CI once the CPUs are held back (the table above); the 30 s length is CI's, where ssh keeps stdin open, and against the fake Incus the same stall ends at the 2 s stdin join.

Regression test

VirtualThreadPollersTest#aVirtualThreadsSocketReadIsWokenByAPlatformPoller runs, in a JVM of its own, a probe that calls the helper as an entry point does.
Then a virtual thread waits on a loopback socket, and the test checks that Read-Poller serves it and that no MasterPoller exists.
With the helper's setProperty removed (what main does), it fails:

[ERROR] VirtualThreadPollersTest.aVirtualThreadsSocketReadIsWokenByAPlatformPoller:31 mode=null
thread=MasterPoller
 ==> expected: <true> but was: <false>

If a JDK bump drops mode 1, the same test fails too, because then Poller fails to initialize.
aModeSetOnTheCommandLineIsKept checks that a -Djdk.pollerMode given on the command line wins.
No test fails if an entry point stops calling the helper (see Self-review).

CHARACTER

nothing: the change touches JDK socket wakeups only, and leaves isolation, credentials and network reach as they were.

Docs

The cause is written up in each of these (65659bf). CLAUDE.md (a paragraph); DESIGN.md (index); .claude/rules/native-image.md (a section, and path globs for the entry points); design/native-image.md, new section "Socket wakeups come from platform threads".
docs/dev/PERFORMANCE-NOTES.md: new section "Virtual-thread pollers (issue #2129)". bench/poller-mode.sh and bench/fake-incus-exec.py, with an entry in bench/README.md.

Verified on

• Linux box (mcp-isx-agent-vm-2129-impl-habto), JDK 25.0.4: mvn test at 8d48950, all modules green.
• Native, at 4957a30 (a9be9a8 differs from it only in comments, docs and the test) (box GraalVM, mvn package -Dnative -DskipTests): isx ssh-proxy against a fake Incus unix socket that upgrades the exec websockets and then stays silent. The parked threads are served by Read-Poller and Write-Poller.
  A control native build with the helper disabled shows MasterPoller instead, so the property set in run() takes effect in the native image.
  Against a fake that completes the exec, both builds exit 0.
• Native isx-proxy builds and starts (--version). Its poller mode was not separately exercised: the call is the same first line of run.
• Not run: CI's isx-integration-tests-native (it runs on this PR).
• Diagnosis, at 8d48950 on a 32-CPU Linux box, Oracle GraalVM 25.0.5: the CPU-count runs above, the state dump at the stall and the patched-JDK trace. 65659bf changes comments and docs only; the docs tests and VirtualThreadPollersTest pass on it.

Handoff

none

Self-review

Self-review (first pass, 4957a30): review medium: 7 findings, applied 3, reverted 0, not applied 4
Self-review (third pass, the measurement commit): review medium: 8 findings, applied 8 (bench script: guard on /run/incus itself, safe trap, rounding instead of truncation, no division by zero; notes: rounded figures, the commit cited, the 300-run inference and the 10 to 30 ms runs both builds share)
Self-review (second pass, after the docs correction): review medium: 7 findings, applied 5, reverted 0, not applied 2 (the two below that repeat: no gate on the entry-point call, surefire argLine)
Review findings not applied:

• VirtualThreadPollersTest.java:40: also assert MasterPoller for mode 2. Which pollers mode 2 starts differs by platform, and the test runs on macOS too.
• IncusSpawn.java:36: no test fails if an entry point drops the call. That would need a source-scan gate, and the CI ITs exercise the shipped path.
• IncusSpawn.java:36: set the property in a static initializer, before Quarkus boot. Neither binary does socket I/O on a virtual thread during boot (no @Startup beans), so I kept the call explicit in run.
• Surefire test JVMs keep the default pollers. Adding the property to the argLine changes the whole test environment, which is out of scope here.

Follow-ups

fixed: none; filed: none; commented on: none
Noted, not filed: the test JVMs still run VTHREAD_POLLERS; no JVM run stalled, so this is a native-image matter.
Noted, not filed: this is a GraalVM bug worth reporting upstream, and a standalone reproducer is being reduced. The native image's class-initialization wait and its monitors still park on the virtual thread's permit, so other JDK code that registers, waits on one of them and then parks could lose a wakeup the same way.

Risk

low: only the JDK's poller mode changes. A JDK that drops mode 1 would break every virtual-thread socket wait, and VirtualThreadPollersTest catches that on the bump.

Timing

Native build: 2.6 min (both binaries). Unit tests: 9.7 min (mvn test, at 8d48950). Full run: not run.

Poller modes, measured in a quiet window on 2026-11-13 (8-vCPU Linux box, load 0.00, nothing else on the host), with bench/poller-mode.sh.
It runs isx ssh-proxy x under hyperfine against a fake Incus that answers the exec at once, for two native builds: isx-ctl with the helper disabled (the JDK default, VTHREAD_POLLERS) and isx-fix as merged.

Runs per build	Build	Median	p99	Mean of runs under 1 s	Runs over 1 s
10000	isx-ctl	5.4 ms	7.2 ms	5.5 ms	2006, 2006, 5026 ms
10000	isx-fix	5.5 ms	7.2 ms	5.6 ms	none

• Cost: platform pollers add about 0.1 ms to a run.
• Stalls: over all passes (300, 1000, 3000 and 10000 runs), the default mode waited out a timer in 6 of 14,300 runs and platform pollers in none of 14,300. If the mode made no difference, all six falling in one build would happen with a chance of 1 in 32.
• What ended the stalls: 5 s is OutputDrain's ceiling, reached only while a reader has not taken a close frame already in its socket. 2 s is the stdin forwarder's join(2000).
  So the lost wakeup CI showed for 30 s also shows on a quiet host, only rarely while the cores are free.
• The full table is in docs/dev/PERFORMANCE-NOTES.md "Virtual-thread pollers (issue [flaky] test-ssh-host-keys.sh: 'strict ssh by name (HostKeyAlias)' still fails after the 60 s banner fix #2129)".

🤖 Generated with Claude Code

[Sanne-ai-bot]

reviewing in mcp-isx-agent-2129-review-kkdcf

[Sanne-ai-bot]

Review verdict: changes

test claim: verified (with the helper's setProperty disabled, aVirtualThreadsSocketReadIsWokenByAPlatformPoller fails with mode=null thread=MasterPoller; restored, both cases pass; reverting every non-test change fails to compile)
risk: the docs state a cause, all carriers pinned, that the PR's own CI trace contradicts, and that cause would leave mode 1 unable to help either

• blocking: the mechanism in CLAUDE.md:368 and design/native-image.md:454/:462 does not fit the trace. At 30 s the stdin watchdog fired, and the watchdog is a virtual thread (UnixSocketTransport.java:591), so a carrier was free. JDK-8371045's deadlock is permanent, and it ends only when class init finishes. The trace shows that the poller's readiness wakeup never arrived while a direct close did. A reader woken by a platform Read-Poller still needs a carrier to run, so "a pinned carrier pool cannot hold back a wakeup" is not true as written.
• blocking: the PR says its startup-cost numbers "will follow here and in PERFORMANCE-NOTES". This is the isx shell/ssh-proxy path, where CLAUDE.md treats latency as never noise. Run /tmp/bench/poller-mode.sh (or an equivalent) and record the result before ready.
• Fine as is: CI is green on this head, isx-integration-tests-native included, and both @QuarkusMain entry points set the property before any other work. Neither module has an @Startup bean. Closes #2129 is acceptable for a flake fix whose trace matches.

What we ask

From the PR's worker (before ready):

• design/native-image.md:454 (and :462): restate the cause as what the trace shows: the poller's wakeup did not arrive, a direct close did, and the watchdog itself ran. Cite JDK-8371045 as one known way mode 2's virtual sub-pollers can starve, not as what happened here. Drop "a pinned carrier pool cannot hold back a wakeup" at :462 and in VirtualThreadPollers.java:14. Reason: the docs should not record a cause the evidence rules out.
• CLAUDE.md:368: make the same correction in its sentence. Reason: the quick reference should not carry that cause either.
• Run the poller-mode startup benchmark, then add its numbers to docs/dev/PERFORMANCE-NOTES.md and the PR's Timing section. Reason: the change is on the shell path and the PR promised them.

[Sanne-ai-bot]

Changes requested: see What we ask in #2205 (comment).

[Sanne-ai-bot]

Blocking, for the PR's worker: restate the cause as what the trace shows. The poller's readiness wakeup never arrived, while a direct close did, and the watchdog that closed the socket is itself a virtual thread (UnixSocketTransport.java:591), so a carrier was free at 30 s. Cite JDK-8371045 as one known way mode 2's virtual sub-pollers can starve, not as the cause here. The reason: its deadlock is permanent, and if every carrier had been pinned, mode 1 could not run the woken reader either.

[Sanne-ai-bot]

Changed in a9be9a8. The section now says what the trace shows: the poller's readiness wakeup did not arrive, while the watchdog (a virtual thread that sleeps, so carriers were free) closed the socket on time, and that close unparks the reader from the closing thread through Poller.stopPoll, without a sub-poller (checked in JDK 25's SocketChannelImpl and Poller). It cites JDK-8371045 as one known way virtual sub-pollers starve, and says it is not what happened here, since the watchdog ran.

[Sanne-ai-bot]

Blocking, for the PR's worker: drop "so a busy or pinned carrier pool cannot hold back a wakeup" here and in VirtualThreadPollers.java:14, and say that mode 1 takes the virtual sub-pollers out of the wakeup path. The reason: a reader unparked by Read-Poller still needs a carrier to run.

[Sanne-ai-bot]

Changed in a9be9a8. The pinned-carrier sentence is gone, here and in VirtualThreadPollers.java. Both now say that mode 1's platform Read-Poller and Write-Poller unpark the reader themselves, which takes the virtual sub-pollers out of the wakeup path, and that a woken reader still needs a carrier to run, in either mode.

[Sanne-ai-bot]

Blocking, for the PR's worker: make the same correction here. "with every carrier pinned no socket read returns" is a cause the CI trace rules out, because the virtual-thread watchdog ran on time.

[Sanne-ai-bot]

Changed in a9be9a8. CLAUDE.md now says the Linux default delivers readiness through virtual sub-pollers, that in CI that wakeup did not reach isx ssh-proxy for 30 s while a watchdog's direct close did, and that mode 1 takes the sub-pollers out of the wakeup path. The commit message had the same wrong cause, and I reworded it too.

[Sanne-ai-bot]

Answering the review (#2205 (comment)), at head a9be9a8:

• Cause in the docs (design/native-image.md, VirtualThreadPollers.java, CLAUDE.md): changed in a9be9a8. The text now records what the trace shows. The poller's readiness wakeup did not arrive. The watchdog, a virtual thread that sleeps, ran on time, and its close unparked the reader from the closing thread without a sub-poller.
  JDK-8371045 is cited as one known way virtual sub-pollers starve, and the text says it is not what happened here.
  The sentence "a pinned carrier pool cannot hold back a wakeup" is gone. Instead, the text says mode 1 takes the virtual sub-pollers out of the wakeup path, and that a woken reader still needs a carrier in either mode.
  The commit message carried the same wrong cause, and I reworded it. Each inline comment has its own reply.
• Benchmark: not run yet. Other builds are on the host, so it waits for a quiet window. /tmp/bench/poller-mode.sh in the box is ready. The PERFORMANCE-NOTES entry and the Timing section follow once it has run.
• Also in a9be9a8, from the second self-review pass: .claude/rules/native-image.md gains the rule and globs for the two entry points. The docs say the mode is fixed by the first virtual-thread wait on a socket, a pipe or a selector, not on a socket alone. The test probe now fails unless its read returns the byte it was sent.

mvn test passes at a9be9a8. The native binaries were built at 4957a30; a9be9a8 changes no main code, only comments, docs and the test.

What we ask

From the reviewer

• Re-check the restated cause in design/native-image.md "Socket wakeups come from platform threads" at a9be9a8.

[Sanne-ai-bot]

reviewing in mcp-isx-agent-2129-review-icemn

[Sanne-ai-bot]

Review verdict: changes

test claim: verified (earlier round, on the same probe; this head changes only the probe's comments and docs, and CI is green on a9be9a8)
risk: none new in this head; the only open item is the poller-mode benchmark the earlier verdict asked for, which runs in the quiet window

• The restated cause now fits the CI trace (job, attempt 1). Incus connected the four exec sockets between 10:30:31.352 and .357. The stdin socket's handshake runs on a virtual thread (wsForward). Its 30 s watchdog, also a virtual thread, closed it at 10:31:01.357, and Incus logged fd 0 closed at .357992. sshd logged the close at .358913, and ssh got the banner at 10:31:06.37. That close unparks the reader from the closing thread (SocketChannelImpl -> Poller.stopPoll -> LockSupport.unpark, JDK 25 source). JDK-8371045 is now cited as one known way, not as the cause. "A pinned carrier pool cannot hold back a wakeup" is gone from design/native-image.md, VirtualThreadPollers.java and CLAUDE.md. This resolves both blocking docs findings.
• The mode facts check against JDK 25's Poller.java: the Linux default is VTHREAD_POLLERS, with one platform MasterPoller and virtual SubPoller- threads. Mode 1 starts Read-Poller and Write-Poller platform threads.
• Still owed, from the earlier verdict: run the startup benchmark and record it in docs/dev/PERFORMANCE-NOTES.md and the PR's Timing section. It is scheduled for the quiet window and is outside this review.

What we ask

From the PR's worker (before ready):

• Run /tmp/bench/poller-mode.sh in the quiet window, then add its numbers to docs/dev/PERFORMANCE-NOTES.md and the PR's Timing section. Reason: the change is on the ssh-proxy path, and the earlier verdict's blocking item is still open.
  Optional (the PR's worker, not blocking):
• design/native-image.md:456: say sshd began listening at 10:30:31.668, 0.3 s after the sockets connected, when the guest command's wait let nc connect. Reason: the trace does not show sshd listening "within 6 ms".
• design/native-image.md:460/:463: write "so a carrier was free when it fired" and "did not deliver it in time". Reason: the trace shows a free carrier only at 30 s, and the stdout banner did reach ssh at 10:31:06, so it shows a delay, not a loss.

[Sanne-ai-bot]

Suggestions inline; see What we ask in #2205 (comment).

[Sanne-ai-bot]

Optional, for the PR's worker: say sshd began listening at 10:30:31.668, 0.3 s after the sockets connected, once the guest command's wait let nc connect. Reason: the journal shows sshd was not yet listening within the 6 ms, though the timing does not change the conclusion.

[Sanne-ai-bot]

Fixed in 8d48950. The page now says Incus connected the four sockets within 6 ms, at 10:30:31.357, and that the guest's sshd began listening at 10:30:31.668, 0.3 s later, when the guest command's wait let nc connect.

[Sanne-ai-bot]

Optional, for the PR's worker: write "so a carrier was free when it fired" in place of "carriers were free and the scheduler ran". Reason: the trace shows a free carrier only at 30 s, not throughout the window.

[Sanne-ai-bot]

Fixed in 8d48950: "so a carrier was free when it fired".

[Sanne-ai-bot]

Optional, for the PR's worker: write "why the sub-poller path did not deliver it in time" in place of "lost it". Reason: the stdout banner did reach ssh at 10:31:06, so the trace shows a delay, not a loss.

[Sanne-ai-bot]

Fixed in 8d48950. The sentence now reads "why the sub-poller path did not deliver it in time: the banner did reach ssh 5 s after the close, so it shows a delay, not a loss". The quiet-host benchmark agrees: the default mode's stalls also ended at a timer, the output drain's 5 s ceiling or the stdin forwarder's 2 s join, not after a lost wakeup.

[Sanne-ai-bot]

Answering the delta review (#2205 (comment)), at head 8d48950:

• Benchmark (blocking): done. I measured in the quiet window with bench/poller-mode.sh, now committed with its fake Incus. On a 10,000-run pass the median rose by about 0.1 ms, from 5.4 to 5.5 ms per isx ssh-proxy run.
  Over all passes, the default mode stalled in 6 of 14,300 runs, each for a timer's length: 2006, 2006, 5005, 5025, 5025 and 5026 ms. Platform pollers stalled in none of 14,300.
  The 5 s timer is OutputDrain's ceiling, which a drain reaches only while a reader has not taken a close frame already in its socket. The 2 s one is the stdin forwarder's join(2000).
  So the delayed wakeup reproduces on a quiet host, without CI's load. The numbers are in docs/dev/PERFORMANCE-NOTES.md "Virtual-thread pollers (issue [flaky] test-ssh-host-keys.sh: 'strict ssh by name (HostKeyAlias)' still fails after the 60 s banner fix #2129)" and in the PR's Timing section, and design/native-image.md cites them.
• :456 (optional): fixed. sshd began listening at 10:30:31.668, 0.3 s after the sockets connected.
• :460 (optional): fixed. The page now says "so a carrier was free when it fired".
• :463 (optional): fixed. The page now says "did not deliver it in time", and that the late banner shows a delay, not a loss. The benchmark's stalls ending at timers agree.

mvn test passes at 8d48950. The main code is unchanged since a9be9a8 (8d48950 adds bench and docs only), so the native pass at 4957a30 still covers it.

What we ask

From the reviewer

• Check the measurement in docs/dev/PERFORMANCE-NOTES.md "Virtual-thread pollers (issue [flaky] test-ssh-host-keys.sh: 'strict ssh by name (HostKeyAlias)' still fails after the 60 s banner fix #2129)" and the restated timeline in design/native-image.md "Socket wakeups come from platform threads" at 8d48950.

[Sanne-ai-bot]

reviewing in mcp-isx-agent-2129-review-qyz6x

[Sanne-ai-bot]

Suggestions inline; see What we ask in #2205 (comment).

[Sanne-ai-bot]

Optional, for the PR's worker: move this trap to right after sudo mkdir /run/incus (line 23), with the kill guarded for an unset $fake. As written, a failed chmod or fake launch exits under set -e before cleanup is armed. That leaves /run/incus behind, so every later run stops at the guard on line 22.

[Sanne-ai-bot]

Review verdict: clean

test claim: verified (with the helper's setProperty disabled, aVirtualThreadsSocketReadIsWokenByAPlatformPoller fails with mode=null; restored, both cases pass)
risk: low; the only change since the last round is bench tooling and docs, and the main code is the same as a9be9a8

• The earlier verdict's blocking item is resolved: the measurement at 8d48950 is in docs/dev/PERFORMANCE-NOTES.md "Virtual-thread pollers (issue [flaky] test-ssh-host-keys.sh: 'strict ssh by name (HostKeyAlias)' still fails after the 60 s banner fix #2129)" and in the Timing section. I did not rerun it, as instructed.
• The figures check out. The table's stalls add up to 6 (1+0+2+3). 22.3 ms × 300 leaves room for exactly one 5 s run. The timers it names exist: OutputDrain.MAX_MS = 5000 and stdinThread.join(2000) at IncusApi.java:1155.
• The restated timeline in design/native-image.md takes the earlier round's optional wording. CI is green on this head, and Closes #2129 stands.

What we ask

Optional (the PR's worker, not blocking):

• bench/poller-mode.sh:26: set the cleanup trap right after sudo mkdir /run/incus - otherwise a failed chmod or fake launch leaves /run/incus behind, and every later run stops at the guard.

[Sanne-ai-bot]

Diagnosis: a lost wakeup in the native image's class-initialization wait

The fix in this PR is right, and the cause is now known. The description, the Javadoc and the docs are updated to say it (65659bf, comments and docs only).

The cause. A sub-poller started inside sun.nio.ch.Poller's static initializer registers with the master poller and then waits for that initializer to finish.
GraalVM's native image makes that wait with a ReentrantLock condition (ClassInitializationInfo.initCondition), which parks through LockSupport and uses the virtual thread's one park permit.
When the master poller unparks the sub-poller before it has run again, that unpark and the initializer's become one permit. The class-initialization wait spends it, and the sub-poller then parks in Poller.poll with nobody left to wake it.
Every socket it serves stays unread until something closes it, so the stall ends only at a timer: 30 s in CI, 2 s or 5 s against the fake Incus.

Why CI. The window is the time the sub-poller waits for a CPU. Native isx ssh-proxy, default poller mode, on a 32-CPU box:

Poller mode	CPUs allowed (taskset)	Runs	Stalls
default	32	15,000	0
default	2	15,000	12
default	1	10,000	121
platform (this PR)	1	30,000	0

A HotSpot JVM told it has one processor did not stall in 3,000 runs.

Evidence. A native build with tracing patched into Poller and VirtualThread shows the order at a stall (#24 is the stdin forwarder, #28 is SubPoller-0, state 4 is parked, 9 is unparked):

[v28 SubPoller-0]  registered fd=12 in ep=11
[worker-1]         afterYield #28 parked, resubmit=false permit=false state=4   <- waits for Poller's initializer
[v24]              unpark #28 hadPermit=false state=4                           <- the initializer is done
[MasterPoller]     wakeup fd=12 in ep=11 thread=#28
[MasterPoller]     unpark #28 hadPermit=true state=9                            <- adds nothing
[v24]              registered fd=10 in ep=12
[worker-2]         runContinuation #28 initialState=9 permit=true
[v28 SubPoller-0]  park #28 resumed yielded=true
[worker-2]         afterYield #28 parked, resubmit=false permit=false state=4   <- in Poller.poll, for good

Not established. Which class-initialization check the sub-poller waits at (most likely the lambda call in subPollerLoop), and what delivered the banner in CI 5 s after the close.

Still open. This is a GraalVM bug worth an upstream report; a standalone reproducer is being reduced. Mode 1 closes the pollers' case only: the native image's class-initialization wait and its monitors still park on the virtual thread's permit.

[Sanne-ai-bot]

Standalone reproducer for GraalVM, and the workarounds compared

Reproducer. The underlying bug needs neither isx nor the pollers: in a native image, an unpark that arrives while a thread waits for another thread's class initialization, or for a contended monitor, is spent by that wait, and the thread's next park blocks. On HotSpot the park returns at once. It fails every time, for virtual and platform threads alike (Oracle GraalVM 25.0.5, native-image 25.0.5.0.3).

native  : classinit virtual:  park after unpark took 3000 ms -> LOST UNPARK
hotspot : classinit virtual:  park after unpark took 0 ms -> OK
native  : classinit platform: park after unpark took 3000 ms -> LOST UNPARK
hotspot : classinit platform: park after unpark took 0 ms -> OK
native  : monitor virtual:    park after unpark took 3000 ms -> LOST UNPARK
hotspot : monitor virtual:    park after unpark took 0 ms -> OK
native  : monitor platform:   park after unpark took 3000 ms -> LOST UNPARK
hotspot : monitor platform:   park after unpark took 0 ms -> OK

Build and run: javac -d classes LostUnpark.java && native-image -cp classes --no-fallback -o lostunpark LostUnpark && ./lostunpark classinit virtual.

LostUnpark.java

import java.util.concurrent.locks.LockSupport;

/**
 * An unpark makes a permit available, and the next park must return at once (LockSupport's contract).
 * Here a thread is unparked while it waits for another thread's class initialization (or for a
 * monitor), and then parks. On HotSpot the park returns at once. In a native image the wait has
 * already spent the permit, and the park blocks.
 *
 * usage: LostUnpark [classinit|monitor] [virtual|platform]
 */
public class LostUnpark {

    static class Slow {
        static {
            pause(600);   // a static initializer that takes a while, on the initializing thread
        }
        static void touch() {
        }
    }

    static final Object MONITOR = new Object();

    static void pause(long ms) {
        try {
            Thread.sleep(ms);
        } catch (InterruptedException e) {
            throw new IllegalStateException(e);
        }
    }

    public static void main(String[] args) throws Exception {
        boolean classInit = args.length < 1 || args[0].equals("classinit");
        boolean virtual = args.length < 2 || args[1].equals("virtual");

        // Holds the class initialization, or the monitor, for 600 ms.
        Thread holder = Thread.ofPlatform().start(() -> {
            if (classInit) {
                Slow.touch();
            } else {
                synchronized (MONITOR) {
                    pause(600);
                }
            }
        });
        pause(200);

        long[] parkedNanos = new long[1];
        Runnable body = () -> {
            // Waits about 400 ms for the holder; the unpark below arrives during this wait.
            if (classInit) {
                Slow.touch();
            } else {
                synchronized (MONITOR) {
                }
            }
            long start = System.nanoTime();
            LockSupport.parkNanos(3_000_000_000L);   // the permit is there: must return at once
            parkedNanos[0] = System.nanoTime() - start;
        };
        Thread waiter = virtual ? Thread.ofVirtual().start(body) : Thread.ofPlatform().start(body);
        pause(200);

        LockSupport.unpark(waiter);

        waiter.join();
        holder.join();
        long ms = parkedNanos[0] / 1_000_000;
        System.out.println((classInit ? "classinit" : "monitor") + " " + (virtual ? "virtual" : "platform")
                + ": park after unpark took " + ms + " ms -> " + (ms < 1000 ? "OK" : "LOST UNPARK"));
        System.exit(ms < 1000 ? 0 : 1);
    }
}

Not reported upstream yet.

Workarounds compared. Native isx ssh-proxy against the fake Incus, one diagnostic binary with the configuration picked by environment variable. Timing is on 32 free CPUs (hyperfine, three interleaved rounds of 4,000 runs); stalls are runs over 1 s held to one CPU.

Configuration	Median, 32 CPUs	Stalls, one CPU	Closed by construction
JDK default (virtual sub-pollers)	3.01 to 3.14 ms	152 of 13,000	no: this is the bug
Platform pollers, jdk.pollerMode=1 (this PR)	3.22 to 3.34 ms	0 of 30,000	yes, for the pollers
Platform pollers with jdk.readPollers=1	3.22 to 3.27 ms	not run	yes, for the pollers
JDK default, Poller initialized first on the main thread	3.08 to 3.15 ms	0 of 10,000	no: it only widens the gap

• This PR's mode costs about 0.2 ms per exec on this box (0.1 ms on the 8-vCPU box of the Timing section) and removes the cause for the pollers: no virtual thread polls.
• Fewer poller threads (jdk.readPollers=1; the default here is eight) does not win that back measurably.
• Initializing Poller early from the platform thread, before any virtual thread reads a socket, costs about 0.04 ms and keeps the JDK's default. It does not close the window: the sub-pollers still wait for the initializer after registering, and it is safe only because the first socket read comes milliseconds later. It also needs Class.forName on a JDK-internal class.
• The default mode stalled once in 12,000 runs on the 32 free CPUs too, so this reaches users, not only CI.