Repository navigation
Conversation
Will be properly tested when distributed_* etc. methods are replaced. Add anysv_region, anyzv_region to things passed for electron_advance. (Might only be relevant for kinetic_electron_solver = "implicit_steady_state", and apparently not triggered in any existing examples.)
johnomotani
left a comment
johnomotani
left a comment
There was a problem hiding this comment.
Nice 👍
As you said, next is to implement these into the distributed_dot(), distributed_norm() and parallel_map() functions. I'm wondering if we can get the implementation down to a single version of all those functions. The generic ranges take care of the different dimension combinations, but we've still got "is it one variable or two variables?". If we set up the one-variable case as a length-1 Tuple of variables, would that make it simple to have just a single implementation for all the functions? [And also make trivial or at least easier generalisation to 3 or more variables if we ever wanted that? Although I'm not sure we ever will, so not something to spend time on.]
| function _dimension_excludes_boundary(c) | ||
| return c isa coordinate && c.nrank > 1 && c.irank < c.nrank - 1 | ||
| end |
There was a problem hiding this comment.
There's a complication here:
- The
parallel_map()functions that perform updates on solution vectors do include the distributed-domain-boundary points, so that the duplicated points get updated on both processes. [I'd need to check, but I don't think those points get explicitly communicated otherwise (e.g. in theresidual_func!()functions - we do communications, e.g. when calculating z-derivatives that ensure the results are identical on the duplicate points, but otherwise duplicate all operations between the distributed MPI ranks so that the duplicated points remain identical without further communication.] - In contrast, the
distributed_norm()anddistributed_dot()functions skip the duplicated points, to ensure that they give the same result regardless of the size of the distributed communicator. The skipping might not be necessary (the JFNK solve I guess should work OK even if they were not skipped), but I think it would be nice to keep for the sake of consistency.
I think this means we actually want two ranges (or a range and a set of ranges). The ones you've constructed so far in this PR for the distributed_norm() and distributed_dot() functions, and another set without the skips for the parallel_map() functions (but this other set should be simpler to construct, because it doesn't have to take care of the skips). The ranges you've constructed here look like the neatest way to generically take care of the duplicate-point skips in flattened loops 😃
Work on #433