Skip to content

Secondary variables: strided batches, up to four per thread - #385

Closed
Oddan wants to merge 2 commits into
sintefmath:mainfrom
Oddan:strided-secondary-batches
Closed

Oddan wants to merge 2 commits into
sintefmath:mainfrom
Oddan:strided-secondary-batches

Conversation

@Oddan

@Oddan Oddan commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Secondary variables are updated in one contiguous batch per thread. When some cells cost much more than others, and these lie together (e.g. around a front), one thread gets most of the work while the others wait.

This PR splits the cells into up to four batches per thread and gives thread t the batches t, t + N, …, so each thread's work is spread over the grid. Results are unchanged.

Trade-off: with uniform cost it is slightly slower. In an application with uneven cost, the secondary variables became 35 % faster, but the equations, which read them, about 12 % slower (likely cache locality); the total run time dropped 11 %. Builds on #384 (this branch includes its commit; I will rebase once #384 is merged).

Testing: test systems and test/mesh.jl pass. Speed-up on 8 threads (100 × 100 grid): uniform cost 7.5× (#384) → 7.2×; one block of expensive cells 1.6× → 5.2×.

Benchmark

As in #384, with n = 1000 < i <= 3000 ? 4000 : 40 sine evaluations per cell for the uneven case. Speed-ups against one serial pass in the same run (8 threads):

cost per cell main #384 this PR
uniform 1.0× 7.5× 7.2×
contiguous expensive region 1.0× 1.6× 5.2×

🤖 Generated with Claude Code

Oddan and others added 2 commits October 15, 2026 14:02
update_secondary_variables_state! passed the number of threads N, not the
number of batches, to threaded_loop_minbatch, together with minbatch. With
the default :batch thread type that is `@batch minbatch = minbatch for i in
1:N`, which Polyester runs on a single thread whenever N < 2 minbatch: the
secondary variables were updated serially. Loop over the N_batches batches
with threaded_loop instead, one contiguous batch per thread as before.

The serial check also compared the function thread_type, not its value tt,
with :serial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With one contiguous batch per thread, a model whose cost per entity varies
with the state (e.g. a vertical-equilibrium model whose plume occupies a
contiguous range of columns) gives one thread most of the work while the
others wait. Use up to four batches per thread, thread t taking batches
t, t + N, ..., so that each thread's share is spread over the entities.
Results are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@moyner

moyner commented Oct 22, 2026

Copy link
Copy Markdown
Member

This was incorporated in #404 together with another option for secondary strides

@moyner moyner closed this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants