Added delta local accumulation inside thread cleanup loop by toxicteddy00077 · Pull Request #7086 · TheHPXProject/hpx

toxicteddy00077 · 2026-03-22T17:23:19Z

Description

continuing in spirit of #7056, the changes are very simple, I have just repalced the repeated per-item atomic decrements with local counters within the loop and and a fetch_sub after the loop. Basically two bulk atomic operations isnteads of many.

StellarBot · 2026-03-22T17:25:03Z

Can one of the admins verify this patch?

toxicteddy00077 · 2026-03-22T17:33:04Z

Benchmarks for comparison, left is baseline, right is my imrpovement, very minimal speedups varying form 21-7% across API

hkaiser

Nice! LGTM, thanks!

hkaiser · 2026-03-22T18:05:30Z

/ok to test

hkaiser · 2026-03-22T18:06:46Z

@charan-003 please have a look here. The CIs are still not being triggered.

charan-003 · 2026-03-22T18:29:46Z

@charan-003 please have a look here. The CIs are still not being triggered.

@hkaiser

looks like the /ok to test workflows are getting stuck in queue

sad to say but we can either:

revert the changes in the PR i created on workflows and make it as it used to be earlier
keep it but the queuing is because we have 40+ parallel test jobs running at same time
even if we do something like https://github.com/marketplace/actions/verify-safe-to-test-label it won't fix the queuing - the issue is too many parallel jobs, not how they're triggered

hkaiser · 2026-03-22T18:37:35Z

@charan-003 please have a look here. The CIs are still not being triggered.

@hkaiser

looks like the /ok to test workflows are getting stuck in queue

sad to say but we can either:

revert the changes in the PR i created on workflows and make it as it used to be earlier

keep it but the queuing is because we have 40+ parallel test jobs running at same time
even if we do something like https://github.com/marketplace/actions/verify-safe-to-test-label it won't fix the queuing - the issue is too many parallel jobs, not how they're triggered

Things were run before (even with the queuing). The difference is that now github reports those CI jobs a skipped. Let's revert the change for now until we have a better understanding of how to do what you suggested.

@charan-003 could you create a PR that reverts it, please? It can't be automatically reverted anymore because of the second PR that modified things.

charan-003 · 2026-03-22T18:40:14Z

@charan-003 please have a look here. The CIs are still not being triggered.

@hkaiser
looks like the /ok to test workflows are getting stuck in queue
sad to say but we can either:

revert the changes in the PR i created on workflows and make it as it used to be earlier

keep it but the queuing is because we have 40+ parallel test jobs running at same time
even if we do something like https://github.com/marketplace/actions/verify-safe-to-test-label it won't fix the queuing - the issue is too many parallel jobs, not how they're triggered

Things were run before (even with the queuing). The difference is that now github reports those CI jobs a skipped. Let's revert the change for now until we have a better understanding of how to do what you suggested.

@charan-003 could you create a PR that reverts it, please? It can't be automatically reverted anymore because of the second PR that modified things.

let me do that quickly

hkaiser · 2026-03-23T12:38:55Z

@toxicteddy00077 Could you please take care of the clang-format issues reported?

StellarBot · 2026-03-23T15:37:59Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	---

Info

Property	Before	After
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-23T12:47:16+00:00
HPX Commit	`0eeca86`	`b8383e0`
Datetime	2026-03-09T09:15:24.034803-05:00	2026-03-23T10:34:57.975114-05:00
Envfile
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	+++

Info

Property	Before	After
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-23T12:47:16+00:00
HPX Commit	`0eeca86`	`b8383e0`
Datetime	2026-03-09T09:17:15.638328-05:00	2026-03-23T10:36:36.098640-05:00
Envfile
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	=	(=)	---
Stream Benchmark - Scale	(=)	--	---
Stream Benchmark - Triad	(=)	-	---
Stream Benchmark - Copy	(=)	+++	---

Info

Property	Before	After
HPX Datetime	2026-03-09T18:50:37+00:00	2026-03-23T12:47:16+00:00
HPX Commit	`ba89f5d`	`b8383e0`
Datetime	2026-03-09T17:49:10.837937-05:00	2026-03-23T10:37:09.849591-05:00
Envfile
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

Signed-off-by: Aryamaan Singh <aryamaan.singh05@gmail.com>

hkaiser · 2026-03-24T13:29:12Z

retest lsu

hkaiser · 2026-03-24T19:41:26Z

Could you please rebase onto master to fix the compilation errors reported? Sorry for the inconvenience.

StellarBot · 2026-03-24T23:47:25Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	---

Info

Property	Before	After
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-24T03:04:23+00:00
HPX Commit	`0eeca86`	`6a94eed`
Envfile
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Datetime	2026-03-09T09:15:24.034803-05:00	2026-03-24T18:44:33.993386-05:00
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	+++

Info

Property	Before	After
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-24T03:04:23+00:00
HPX Commit	`0eeca86`	`6a94eed`
Envfile
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Datetime	2026-03-09T09:17:15.638328-05:00	2026-03-24T18:46:11.438824-05:00
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	(=)	---
Stream Benchmark - Scale	(=)	--	---
Stream Benchmark - Triad	(=)	-	---
Stream Benchmark - Copy	(=)	+++	---

Info

Property	Before	After
HPX Datetime	2026-03-09T18:50:37+00:00	2026-03-24T03:04:23+00:00
HPX Commit	`ba89f5d`	`6a94eed`
Envfile
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Datetime	2026-03-09T17:49:10.837937-05:00	2026-03-24T18:46:45.584776-05:00
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

…atomic-cleanup Added delta local accumulation inside thread cleanup loop Signed-off-by: Vaibhav Joshi <joshivaibhav553@gmail.com>

toxicteddy00077 requested a review from hkaiser as a code owner March 22, 2026 17:23

toxicteddy00077 force-pushed the fea/batch-atomic-cleanup branch from c0f3fd4 to 990890b Compare March 22, 2026 17:25

toxicteddy00077 force-pushed the fea/batch-atomic-cleanup branch from 990890b to 5b34b08 Compare March 22, 2026 17:34

hkaiser added type: optimization type: enhancement category: scheduler labels Mar 22, 2026

hkaiser previously approved these changes Mar 22, 2026

View reviewed changes

hkaiser added this to the 2.0.0 milestone Mar 22, 2026

TheHPXProject deleted a comment from github-actions Bot Mar 22, 2026

github-actions Bot added the ok-to-test label Mar 22, 2026

github-advanced-security AI found potential problems Mar 22, 2026

View reviewed changes

Comment thread libs/core/schedulers/include/hpx/schedulers/thread_queue.hpp Fixed

Comment thread libs/core/schedulers/include/hpx/schedulers/thread_queue.hpp Fixed

toxicteddy00077 dismissed hkaiser’s stale review via 8f194de March 23, 2026 12:47

toxicteddy00077 force-pushed the fea/batch-atomic-cleanup branch from 5b34b08 to 8f194de Compare March 23, 2026 12:47

github-advanced-security AI found potential problems Mar 23, 2026

View reviewed changes

Comment thread libs/core/schedulers/include/hpx/schedulers/thread_queue.hpp Dismissed

Comment thread libs/core/schedulers/include/hpx/schedulers/thread_queue.hpp Dismissed

hkaiser reviewed Mar 23, 2026

View reviewed changes

Comment thread bench_results/pr2_batch_atomic_results.txt Outdated

Added delta local accumulation inside thread cleanup loop

933145a

Signed-off-by: Aryamaan Singh <aryamaan.singh05@gmail.com>

toxicteddy00077 force-pushed the fea/batch-atomic-cleanup branch from 8f194de to 933145a Compare March 24, 2026 03:04

toxicteddy00077 requested a review from hkaiser March 24, 2026 03:04

hkaiser merged commit 43d1ddd into TheHPXProject:master Mar 25, 2026
136 of 170 checks passed

Uh oh!

Conversation

toxicteddy00077 commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

StellarBot commented Mar 22, 2026

Uh oh!

toxicteddy00077 commented Mar 22, 2026

Uh oh!

hkaiser left a comment

Choose a reason for hiding this comment

Uh oh!

hkaiser commented Mar 22, 2026

Uh oh!

hkaiser commented Mar 22, 2026

Uh oh!

charan-003 commented Mar 22, 2026

Uh oh!

hkaiser commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charan-003 commented Mar 22, 2026 • edited by hkaiser Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hkaiser commented Mar 23, 2026

Uh oh!

Uh oh!

Uh oh!

StellarBot commented Mar 23, 2026

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

Uh oh!

Uh oh!

hkaiser commented Mar 24, 2026

Uh oh!

hkaiser commented Mar 24, 2026

Uh oh!

StellarBot commented Mar 24, 2026

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

toxicteddy00077 commented Mar 22, 2026 •

edited

Loading

hkaiser commented Mar 22, 2026 •

edited

Loading

charan-003 commented Mar 22, 2026 •

edited by hkaiser

Loading