how to config when you have multiple GPUs ?

[Andrewp9]

hello everyone,

I have two GPU's. in the json for openCL I have:
"producerOpenCL": [
{
"platformIndex": 0,
"deviceType": -1,
"deviceIndex": 0,
...
I've played around with the parameters trying to get it to run on my second GPU without any luck. My preference would be to use both.
what configuration would I need to use both?

[daveshenal]

producerOpenCL is a JSON array. Eeach element is one device's own config (you can see this in your own snippet, since it opens with [). To use both GPUs, don't edit the existing entry... add a second object alongside it:

"producerOpenCL": [
  {
    "platformIndex": 0,
    "deviceType": -1,
    "deviceIndex": 0,
    ...
  },
  {
    "platformIndex": 0,
    "deviceType": -1,
    "deviceIndex": 1,
    ...
  }
]

Each entry runs as its own independent producer, so both GPUs work in parallel.

The likely reason your earlier attempts didn't work: deviceIndex is scoped per platform, not global across all GPUs in the system. If both your GPUs are the same vendor (e.g. two NVIDIA cards), they're almost always on the same platformIndex, so 0 and 1 is correct. But if they're different vendors (e.g. NVIDIA + an Intel iGPU), they're usually on two different platforms... so the second entry needs a different platformIndex (e.g. 1), not the same platformIndex with deviceIndex: 1. Guessing which combo is right without checking is exactly the kind of thing that silently fails.

To find the correct indices for your system: run clinfo -l (install clinfo if you don't have it) - it lists every OpenCL platform and the devices under it, in the same enumeration order the app uses. Match your second GPU's platform/device position from that output into your JSON. BitcoinAddressFinder's own startup log should also print the OpenCL devices it detects, which is worth checking too.

deviceType: -1 just means "any device type". leave that as is on both entries, it's not the issue.

[bernardladenthin]

The producerOpenCL field is a JSON array, so you can add one entry per GPU. Each entry targets a specific device via platformIndex + deviceIndex, and each must reference its own keyProducerId (key producers cannot be shared between producers).

Step 1 — find your platform/device indices

Run the tool with {"command": "OpenCLInfo"} first. It prints every OpenCL platform and device with their indices.

Step 2 — define one key producer per GPU

Each ProducerOpenCL needs its own dedicated key producer so both GPUs draw from independent random streams.

Step 3 — add a second entry to producerOpenCL

{
  "command": "Find",
  "finder": {
    "keyProducerJavaRandom": [
      {
        "keyProducerId": "gpu0_keys",
        "randomAlgorithm": "SECURE_RANDOM",
        "customSeed": 0,
        "privateKeyMaxNumBits": 256
      },
      {
        "keyProducerId": "gpu1_keys",
        "randomAlgorithm": "SECURE_RANDOM",
        "customSeed": 0,
        "privateKeyMaxNumBits": 256
      }
    ],
    "consumerJava": {
      "lmdbConfigurationReadOnly": {
        "lmdbDirectory": "lmdb",
        "useProxyOptimal": true,
        "logStatsOnInit": true,
        "logStatsOnClose": false,
        "addressLookupBackend": "LMDB_ONLY",
        "disableAddressLookup": false
      },
      "printStatisticsEveryNSeconds": 10,
      "threads": 8,
      "queuePollTimeoutMillis": 50,
      "queueSize": 8,
      "awaitQueueEmptySeconds": 60,
      "runtimePublicKeyCalculationCheck": false,
      "enableVanity": false,
      "vanityPattern": "1[Ee][Mm][Ii][Ll].*"
    },
    "awaitTerminateSeconds": 31536000000,
    "producerJava": [],
    "producerJavaSecretsFiles": [],
    "producerOpenCL": [
      {
        "platformIndex": 0,
        "deviceType": -1,
        "deviceIndex": 0,
        "maxResultReaderThreads": 4,
        "keyProducerId": "gpu0_keys",
        "batchSizeInBits": 18,
        "keysPerWorkItem": 16,
        "batchUsePrivateKeyIncrement": true,
        "logSecretBase": false,
        "runOnce": false,
        "shutdownTimeoutSeconds": 300,
        "enableGpuFilter": true,
        "transferAll": false,
        "useSafeGcdInverse": true
      },
      {
        "platformIndex": 0,
        "deviceType": -1,
        "deviceIndex": 1,
        "maxResultReaderThreads": 4,
        "keyProducerId": "gpu1_keys",
        "batchSizeInBits": 18,
        "keysPerWorkItem": 16,
        "batchUsePrivateKeyIncrement": true,
        "logSecretBase": false,
        "runOnce": false,
        "shutdownTimeoutSeconds": 300,
        "enableGpuFilter": true,
        "transferAll": false,
        "useSafeGcdInverse": true
      }
    ]
  }
}

Key points:

• deviceIndex: 0 and deviceIndex: 1 select the two GPUs. If your GPUs are on different OpenCI platforms (e.g. one NVIDIA, one AMD), you may need platformIndex: 0 for one and platformIndex: 1 for the other — OpenCLInfo will tell you.
• Each GPU entry needs its own keyProducerId — they must be unique across the whole config.
• queueSize was bumped to 8 to give two producers more buffer space; tune this based on your system's RAM.
• You can also mix CPU producers alongside the two GPU producers the same way config_Find_1OpenCLDeviceAnd2CPUProducer.json does.

[bernardladenthin]

@Andrewp9 Thanks for running both — that is exactly the data that was missing, and it already changed two things I told you earlier.

What your runs measured

	RTX 500 Ada Laptop	Intel Arc Pro (integrated)
Winner	batchSizeInBits=23, keysPerWorkItem=256	batchSizeInBits=20, keysPerWorkItem=256
Peak measured	94.79 M keys/s	17.88 M keys/s
Verification cost	12.61 µs	11.55 µs
Filter recommendation	FUSE_16	FUSE_16

Correction to what I said earlier: I estimated the Arc at ~5 % of your total and said it would stay "far behind". Measured alone it does 17.9 M/s, so the real solo ratio is 5.3:1, not 20:1. I was reading a number that had another effect baked into it — see below.

Three things your logs show

1. Both sweeps stopped too early — these are lower bounds, not peaks

keysPerWorkItem=256 is the last value in the default candidate list, and it won on both devices. On the RTX at batchSizeInBits=23 the column is still climbing at the edge:

keysPerWorkItem:      1        4       16       64      256
                  2.94 M  11.32 M  36.49 M  64.59 M  94.79 M   <- still rising, sweep ended here

Your own earlier dual-GPU run backs this up: the RTX at keysPerWorkItem=512, batchSizeInBits=24 produced roughly 110 M/s, above the 94.79 M/s the tuner just called the "winner". So the true optimum is above 256 and neither run found it.

2. Your two GPUs are not independent — the Arc loses ~3× when the RTX runs

Same device, same settings (batchSizeInBits=21, keysPerWorkItem=256):

• Alone, in this tuning run: 16.25 M/s
• Alongside the RTX, in your dual Find run: 5.45 M/s

That is a 2.98× loss, and it is why my earlier estimate was so far off. The Arc Pro is an integrated GPU — clinfo reports Unified memory for Host and Device: Yes and 50.73 GiB of "global memory", which is a share of your system RAM, not dedicated VRAM. When the RTX saturates host memory bandwidth, it takes that bandwidth directly from the Arc. The RTX has its own VRAM and does not suffer the reverse effect.

Practical consequence: tune each device alone, but expect the integrated one to underperform its solo number in a dual run. That is a hardware property, not a configuration mistake. This was not documented anywhere; it is going into the docs now.

3. The 0.00 candidates/s arms are not failures

On the Arc, batchSizeInBits 22/23/24 with keysPerWorkItem=1 read 0.00. Nothing broke — one batch at those settings takes about 40 s, and the measurement window is 20 s, so not a single batch completed inside it. Only arms annotated FAILED: are real driver rejections. The report should say so instead of printing a bare 0.00, and I am fixing that.

---

What I would like you to run

One re-run per GPU with a wider keysPerWorkItem list. Everything else stays as you had it.

Step 1 — edit your two tuning configs

In each of your tuning configs, change this line:

"keysPerWorkItemCandidates": [1, 4, 16, 64, 256],

to:

"keysPerWorkItemCandidates": [16, 64, 256, 512, 1024, 2048],

Dropping 1 and 4 costs nothing — they are far too small to ever win on a GPU — and pays for the four larger values without making the run longer than it already was.

While you are in there, one more improvement: set

"targetDatabaseEntries": 140964881,

You used the default 132288304, but your Find log shows your actual database is 140 964 881 addresses. The sweep sizes its synthetic filter from this number, so matching your real database makes the result apply to the run you actually do.

Keep one device per config file, as you already have:

• config_TuneConfiguration_nvidia.json → "platformIndex": 0, "deviceIndex": 0
• config_TuneConfiguration_arc.json → "platformIndex": 1, "deviceIndex": 0

(The sweep only ever tunes the first producerOpenCL entry, which is why they have to be separate files.)

Step 2 — run each one, capturing the log

Use run_TuneConfiguration.bat — not a bare java -jar. The launcher passes the --add-opens flags the JVM needs before the LMDB lookup path will work; without them you get InaccessibleObjectException as soon as a database is configured.

Open run_TuneConfiguration.bat in a text editor and make two changes:

1. On the second-to-last line, point it at the config you want to run:

   config_TuneConfiguration_nvidia.json

2. On the last line, remove the leading rem so the redirect is active, and give the log a name that says which GPU it is:

   >> log_TuneConfiguration_nvidia.txt 2>&1

   (2>&1 sends error output to the same file; without it, any error goes to the console and is lost from the log.)

Then run it from a command prompt in that directory:

run_TuneConfiguration.bat

When it finishes, edit the same two lines to say arc instead of nvidia and run it again.

Please leave the machine otherwise idle during each run — and in particular do not run the two GPUs at the same time. The second finding above is exactly what happens when you do, and it would contaminate the numbers.

What this creates

File	Where it comes from
log_TuneConfiguration_nvidia.txt	your redirect — contains the full run and the report
log_TuneConfiguration_arc.txt	same, for the second GPU
nothing else	the tuner writes no files of its own; the recommended config is printed inside the log

Expect roughly 20 minutes per GPU: 7 batch sizes × 6 keysPerWorkItem values = 42 arms × 25 s, plus a one-off filter build of about a minute at 141 M entries.

Reading the report in 1.7.0: it arrives as one very long line with | where the line breaks should be. Paste it into Notepad++ and replace | with \n (Search mode: Extended) — same trick @JamesKrolak described in #250. The next release emits it one line per record, so this stops being necessary.

---

What I need to publish the numbers

Measurements are stored per machine, because results depend on hardware — so a number without its machine cannot be interpreted. Two things, then:

1. The two log files. Attach them here as you did this time.

2. Your machine's details. Either way is fine:

• If you have a git clone of the repo and Python 3 installed (standard library only, nothing to pip install), run:

  python docs/measurements/register_machine.py --dry-run

  --dry-run prints the entry without writing anything. Paste the output here.
• Or just tell me: CPU model, total RAM, OS version, JDK version, and the drive the LMDB sits on (SSD/NVMe model, or "spinning disk"). That is all the registry stores.

Both your GPUs would be genuinely new coverage — everything measured so far has been high-end desktop hardware, so a laptop NVIDIA and an Intel integrated GPU fill a real gap, especially now that we know integrated GPUs behave differently under load.

I am also adding a step-by-step guide covering all of this (docs/tuning-your-gpu.md) so the next person does not have to work it out from a discussion thread.

[bernardladenthin]

@Andrewp9 Sorry, it seems I missed your last comment. My latest instructions above should help. 😄

[Andrewp9]

I'm running a Lenovo Thinkpad P14s Gen 5 an ultra 7 155H @ 1400 Mhz, 96 GB ram, 4TB SSD on an M2 card (crucial P3 plus), Win 11, Java 8 update 491. ha ha when I realized I did not follow the original instructions I deleted the logs, reran... now here we are. :) Now I'm getting 124M/s generated running both with 24/2048.

[bernardladenthin]

No apology needed — the confusion was mine. I posted those instructions as a new reply instead of answering your comment directly, so you had no reason to see them before you had already run the sweeps. Thanks for redoing them.

And you nailed it: both new logs have keysPerWorkItemCandidates: [16, 64, 256, 512, 1024, 2048] and targetDatabaseEntries: 140964881. Exactly right.

The result was worth the re-run

	RTX 500 Ada	Intel Arc Pro (integrated)
Old sweep (stopped at 256)	94.79 M/s @ 23 / 256	17.88 M/s @ 20 / 256
New sweep (up to 2048)	110.73 M/s @ 24 / 2048	36.48 M/s @ 23 / 2048
Gain	+17 %	+104 %

The Arc more than doubled. That also settles the question of how your two GPUs really compare: solo, it is a 3:1 split, not the 20:1 I first estimated from your dual run — that early number had the memory-bandwidth contention baked into it, and the 5.3:1 from the previous sweep was still held down by the 256 ceiling.

Your verification cost also came out lower this time (10.69 µs / 10.11 µs versus 12.61 / 11.55), which is the page cache being warmer. FUSE_16 remains the recommendation on both.

Your GPUs are now in different places

The RTX has converged. At batchSizeInBits=24 the last step is worth almost nothing (1024 → 2048 is +1.6 %), and at 23 it actually gets worse (107.7 → 104.8 M/s). You are at the top of the surface; going wider will not buy anything meaningful.

The Arc has not. At batchSizeInBits=23 it is still climbing hard at the edge:

keysPerWorkItem:     256      512     1024     2048
                  15.1 M   20.1 M   25.2 M   36.5 M   <- sweep ended here, still rising

So if you feel like one more run — entirely optional, and only for the Arc — set

"keysPerWorkItemCandidates": [1024, 2048, 4096, 8192],

That is 28 arms, roughly 12 minutes. One honest caveat: the Arc's numbers are noticeably noisier than the RTX's (at batchSizeInBits=22 the same 1024 → 2048 step gains only 3 %, and at 24 it drops), which is what a GPU sharing system RAM with the host tends to look like. So treat a further gain as plausible rather than certain. If you would rather stop here, 36.5 M/s is already a solid, usable result — just put batchSizeInBits=23, keysPerWorkItem=2048 into the Arc's entry in your Find config.

---

What I would still appreciate

Thank you for the hardware list — that is most of it. To publish these numbers I would really like the output of the registration script rather than a hand-written list, for one specific reason: measurements are stored per machine, and a number whose machine is described inconsistently cannot be compared against anything later.

A concrete example from your own message: you wrote Java 8 update 491, but BitcoinAddressFinder requires Java 21+, and your logs contain warnings (--enable-native-access, terminally-deprecated sun.misc.Unsafe) that only a JDK 24 or newer produces. So the Java 8 you are seeing is almost certainly a separate installation, not the one that ran these sweeps. The script reads the JDK actually running it, so that ambiguity disappears.

How to run it — no git clone needed

It is a single self-contained file, standard library only, nothing to install:

1. Download this one file:
   https://raw.githubusercontent.com/bernardladenthin/BitcoinAddressFinder/main/docs/measurements/register_machine.py
   (right-click → Save as, into any folder)

2. Open a command prompt in that folder and run:

   python register_machine.py --dry-run

   --dry-run means it prints the result and writes nothing at all — no file is created or modified anywhere on your system.

3. Paste the printed output here.

It detects CPU model, physical/logical cores, L3 cache, RAM, GPU, OS version and JDK version. Two things to watch for:

• If it prints !! l3_mb could not be detected, re-run it as
  python register_machine.py --dry-run --set cpu.l3_mb=24 (the Ultra 7 155H has 24 MB).
• It usually reports only one GPU. If it names just one of yours, please say so — I will record both.

If you would rather not run a script at all, that is completely fine — just tell me and I will fill the entry in from your description and mark the JDK version as reported rather than detected.

---

Exactly what I am expecting from you

#	What	Status
1	log_TuneConfigurationNvidiaRTX500.txt	✅ received
2	log_TuneConfigurationIntelArcPro.txt	✅ received
3	The text output of python register_machine.py --dry-run, pasted into a comment	⬜ this is the one thing outstanding
4	(optional) a third Arc log with [1024, 2048, 4096, 8192]	⬜ nice to have, not required

Nothing else — no CSV, no pull request, no edits on your side. I will do the rest.

Once item 3 arrives, your machine goes into docs/measurements/machines.json and both GPUs get their own rows in the measurement data. That fills a real gap: everything measured so far has been high-end desktop hardware, so a ThinkPad with a laptop RTX and an integrated Arc is the first data point of its kind here — especially now that we have measured how differently an integrated GPU behaves when a discrete card is running alongside it.

[bernardladenthin]

.

[Andrewp9]

log_TuneConfigurationIntelArcPro.txt
log_TuneConfigurationNvidiaRTX500.txt

[Andrewp9]

register_machine text:
machine_id: coreultra7155h-95g-win11
{
"cpu": {
"model": "Intel(R) Core(TM) Ultra 7 155H",
"cores_physical": 16,
"cores_logical": 22,
"l3_mb": 24
},
"ram_gb": 95.5,
"gpu": "NVIDIA RTX 500 Ada Generation Laptop GPU",
"storage": null,
"os": {
"name": "Microsoft Windows 11 Pro",
"version": "10.0.26200"
},
"jdk": "26.0.1",
"registered": "2026-07-25",
"notes": "",
"updated": "2026-07-25"
}

(dry run — nothing written)

[Andrewp9]

I think the extended values might be pushing it.
log_TuneConfigurationIntelArcPro2.txt

[bernardladenthin]

I think the extended values might be pushing it. log_TuneConfigurationIntelArcPro2.txt

Thank you — genuinely. You ran the sweep three times, re-ran it after my instructions arrived late, and then went and executed a script I asked you to trust on your own machine. That is a lot of patience for something you get nothing out of directly, and it produced the most useful contribution this project's measurement data has had in a while. Both your GPUs are now committed.

First: the missing Arc was our bug, not your mistake

I told you the script "usually reports only one GPU". That was me describing a symptom without having looked at the cause. Having looked:

nvidia = _run(["nvidia-smi", ...])
if names:
    return ...          # <- returns here
if system == "Windows":  # <- never reached

As soon as nvidia-smi named anything, the Windows device enumeration below it was unreachable. On any machine with an NVIDIA card plus a second GPU, the second one was silently dropped — which is exactly the class of machine whose data we most wanted. Your Arc did not fail to be detected; it never got the chance.

Fixed: every source is now queried and merged, with duplicates removed. And the field itself is now a proper JSON list instead of a comma-joined string, so a machine with two GPUs is a normal case rather than something a reader has to parse out of a sentence. Nothing left for you to do here — I completed your entry by hand from your report.

What went in

Machine coreultra7155h-95g-win11 — Core Ultra 7 155H, 24 MB L3, 95.5 GB RAM, JDK 26.0.1, Crucial P3 Plus 4 TB, both GPUs listed.

All 84 measured arms, parsed straight out of the two logs you attached rather than retyped, with the winners flagged:

	winner	measured
RTX 500 Ada	batchSizeInBits=24, keysPerWorkItem=2048	110.73 M keys/s
Arc Pro (integrated)	batchSizeInBits=23, keysPerWorkItem=2048	36.48 M keys/s

Yours is the first registered machine with laptop graphics, the first with an integrated GPU, and the first where one host drives two devices that measurably interfere with each other. Everything before it was single-GPU desktop hardware.

Two things your data changed beyond the numbers

A documented hardware effect. The 3× your Arc loses when the RTX runs alongside it is now written down in a new guide, so the next person with a laptop does not spend an evening concluding their config is broken. It was not documented anywhere before, and it only became visible because you ran the same device both alone and in a pair.

A guide that did not exist. docs/tuning-your-gpu.md — finding device indices, writing the config, capturing the log, reading the report, tuning a second GPU, troubleshooting. Essentially this conversation, written down properly so the next person does not have to have it. Several of its warnings are yours: the sweep that stops too early, the 0.00 rows that look like failures but are not, running two GPUs at once contaminating a measurement.

Also from your reports: the Statistics line now shows keys per producer with each device's share instead of batch counts (your 1156 vs 595 looked like 2:1 when the real split was 20:1), rates keep four significant digits so two runs stay comparable above 1 G/s, the tuner's 0.00 rows now say NOT MEASURED with the reason, and multi-line reports no longer arrive as one line separated by |.

Nothing further needed from you

You are done — no re-run, no script, no follow-up. The optional wider Arc sweep from my last message still stands if you are curious whether it keeps climbing past 2048, but that is for your own interest, not for us; what you have already sent is complete and usable as it is.

One practical thing for your own setup, since you have the numbers now: put batchSizeInBits=24, keysPerWorkItem=2048 in the RTX entry and 23, 2048 in the Arc entry of your Find config, with FUSE_16 on both. That is a solid step up from what you were running when this thread started.

Thanks again. 🙏

[bernardladenthin]

@Andrewp9 — on the third Arc sweep, and then a question about whether you would like to try the fixed build yourself.

Your instinct was half right, and the other half is our bug

You are right that there is nothing above 2048. At the two batch sizes that completed, throughput falls off monotonically as keysPerWorkItem grows past 1024:

batchSizeInBits=18:  1024 -> 3.97 M/s   2048 -> 2.42 M/s   4096 -> 1.31 M/s   8192 -> 0.67 M/s
batchSizeInBits=19:  1024 -> 7.92 M/s   2048 -> 4.43 M/s   4096 -> 2.54 M/s   8192 -> 1.02 M/s

So 2048 really is the end of the useful range on this device. Question settled — and settled by measurement, which is exactly why it was worth running.

But the CL_OUT_OF_HOST_MEMORY wall is not your hardware "being pushed". I compared every arm this run shares with your previous one:

batchSizeInBits / keysPerWorkItem	run 2	run 3
18 / 1024	3,997,467	3,971,465
19 / 1024	7,916,089	7,916,698
20 / 1024	15,622,584	FAILED
21 / 1024	29,358,578	FAILED
22 / 2048	32,070,933	FAILED
23 / 2048	36,481,092	FAILED
24 / 2048	26,843,233	FAILED

The arms at 18 and 19 reproduce to within about 1 %, so your machine and your setup are demonstrably fine. Yet every arm from batchSizeInBits=20 upward now fails — including 23 / 2048, the exact combination that produced your 36.5 M/s winner an hour earlier.

A device does not run out of host memory at a grid size it handled comfortably minutes before. What actually happened is visible one line above the wall: the 19 / 8192 arm returned a degenerate result (candidates counted, but 0.00 addresses checked), and from the very next arm onward everything fails. One bad arm poisons every arm after it — the sweep does not recover. That is our defect, not a limit of your Arc, and I am filing it. The tuner should reset the device context after a failed arm instead of carrying the damage forward, and it should not present a cascade of collateral failures as if each arm had been independently rejected.

Nothing changes for you. Your run-2 result stands unchanged: batchSizeInBits=23, keysPerWorkItem=2048 at 36.48 M keys/s. That is what is committed, and it is still the right setting for your Arc.

Would you like to build the fixed version yourself?

No pressure at all — if you would rather wait, I will put a build on GitHub shortly and you can just download it.

But if you are up for it, it is genuinely short. You need JDK 21 or newer (you already have 26) and Maven 3.6.3+; there is no wrapper script in the repo, so Maven has to be installed.

git clone -b claude/statistics-keys-per-producer https://github.com/bernardladenthin/BitcoinAddressFinder.git
cd BitcoinAddressFinder
mvn package -P assembly -DskipTests

• -P assembly activates the profile that bundles every dependency into one runnable jar, so there is nothing to put on a classpath afterwards.
• -DskipTests skips the test suite. It is what makes this quick — the full suite takes several minutes and none of it is needed to produce a working jar.

The result lands in target/ as bitcoinaddressfinder-<version>-jar-with-dependencies.jar (currently 1.8.0-SNAPSHOT). Copy it next to your config files and point your existing run_*.bat scripts at the new filename — nothing else about your setup changes.

That branch is where everything from this thread lives, and it will land on main shortly. What you would get:

• The statistics line reports keys per producer with each device's share, so the RTX/Arc split is readable directly instead of having to be reconstructed from batch counts.
• Rates keep four significant digits — no more everything above 1 G/s collapsing to 1 G/s.
• Multi-line reports print properly instead of arriving as one long line separated by |. No more pasting into Notepad++ to read a tuning report.
• The tuner distinguishes NOT MEASURED from FAILED, so an arm that simply ran out of measurement window no longer looks like a driver rejection.
• docs/tuning-your-gpu.md, the step-by-step guide this thread turned into.

The CL_OUT_OF_HOST_MEMORY recovery bug above is not in that branch yet — it is new as of your log today.

Either way, thanks again for the third run. It answered the open question and found a bug while doing it.

[bernardladenthin]

@Andrewp9 Quick follow-up: the bug your third log exposed is fixed, and everything from this thread is now on the claude/statistics-keys-per-producer branch.

Your 19 / 8192 arm reported 1,022,288 candidates/s while its producer had actually died of CL_OUT_OF_RESOURCES seconds in. The exception was logged on the producer thread and never reached the sweep, so a crashed arm was indistinguishable from a healthy one. Producers now record what killed them, and such an arm is recorded as FAILED with the cause instead of a partial rate. Written test-first, both tests confirmed failing before the fix existed.

Still open, and honestly not something I can fix from a log: why your Arc stays wedged after that first error, so that grids which measured fine an hour earlier then fail too. That looks like device or driver state. What changed is that the sweep no longer hides the arm that triggered it.

So if you do build that branch, you get this fix as well as everything listed in my previous comment. Thanks again for the log — it was the only reason this was findable.