Skip to content

Create the domain decomposition once the spline degree is known #628

Description

@yguclu

Each MPI process must own at least degree coefficients along each direction (at least degree + 1 along non-periodic directions of multipatch geometries), see psydac.fem.partitioning.partition_coefficients. However, the domain decomposition is created before the spline degree is known (in the Geometry constructors, or in discretize(domain, ncells=...)), hence nothing prevents a decomposition where some processes own too few cells. The problem is only detected later, when the spline space is created:

# Run with: mpirun -n 4 python example.py
import numpy as np
from mpi4py import MPI
from psydac.ddm.cart import DomainDecomposition
from psydac.fem.splines import SplineSpace
from psydac.fem.tensor import TensorFemSpace

ncells, degree = 8, 3     # 2 cells per process, i.e. 2 < 3 coefficients
W = SplineSpace(degree=degree, grid=np.linspace(0, 1, ncells + 1), periodic=True)
ddm = DomainDecomposition([ncells], [True], comm=MPI.COMM_WORLD)
V = TensorFemSpace(ddm, W)
  File "psydac/fem/partitioning.py", line 72, in partition_coefficients
    assert all(e-s+1 >= V.degree)
AssertionError

PR #527 replaces this assertion with a ValueError which explains the problem and suggests possible remedies (fewer processes along that direction, e.g. with mpi_dims_mask, or more cells), but the user still has to find a valid configuration by hand.

The domain decomposition should rather be created when the spline degree is known, so that the process grid can be chosen accordingly (e.g. with the argument min_blocksizes of psydac.ddm.partition.compute_dims, which is currently unused by DomainDecomposition).

Constraints

  • A domain decomposition is shared by all the spaces defined on a geometry, which may be created by separate calls to discretize with different degrees; only discretize(derham, ...) knows all the degrees of a sequence at once
  • A geometry read from a file contains spline mappings, which are defined on a domain decomposition created when the file is read, before any FEM space exists
  • The minimum number of cells per process depends on the degree, on the multiplicity (with multiplicity m, a process owning c cells owns m * c coefficients), on the periodicity, and on whether the geometry is multipatch

Possible strategies

  1. Explicit: pass the degree (or a minimum number of cells) when the geometry is created, e.g. discretize(domain, ncells=..., degree=...) and in the Geometry constructors; Geometry.from_file could also use the degrees of the mappings stored in the file. Simple and predictable, but it extends the API, and users must give the maximum degree of all the spaces they will create
  2. Lazy: Geometry stores the number of cells, the periodicity, the communicator, and mpi_dims_mask, and creates the domain decomposition when the first space (or de Rham sequence) is discretized. Transparent for users, but spaces of higher degree created later may not fit, and it does not work for geometries read from a file
  3. Sequence-level: create the domain decomposition in discretize(derham, ...), which knows all the degrees, when the geometry was created from the number of cells only. It covers the common FEEC use case, but not standalone spaces

These strategies may also be combined, e.g. strategy 1 for geometry files and for explicit control, and strategy 2 or 3 as default for geometries created from the number of cells.

Discussion started in #527 (comment).

Activity

  1. added theissue type on Oct 17, 2026
  2. added
    FEM APIObjects describing finite element concepts
    MPIRelated to MPI decomposition and parallel computing
    on Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FEM APIObjects describing finite element conceptsMPIRelated to MPI decomposition and parallel computing

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions