[RFC] Make shard balance aware of allocation filters

[guojialiang92]

Background

Weight calculation plays a key role in shard balancing. During the rebalance process, shards from high-weight nodes will be migrated to low-weight nodes, and a balanced state will be achieved eventually. The average number of shards per node is a key variable for calculating weights, and the formula for the average number of shards is total_shards/node_count. When calculating node_count, the allocation filter is not taken into account, and the total number of nodes is directly used.

There are the following places in the code where the average number of shards is calculated, and the total number of nodes is used.

• Average number of shards at index level
• Average number of shards
• Average number of primary shards at index level
• Average number of primary shards

In non-segment replication scenarios, since all nodes will perform segment building, even if the allocation filter is not taken into account, there will be no obvious skew in CPU overhead.

However, in scenarios where segment replication is enabled, skew in the primary shard will impact stability.

Reproduction

We mainly consider the scenarios where segment replication and primary shard balancing are enabled.
The following scenarios in a production environment can easily trigger primary shard imbalance. These scenarios usually require configuring cluster.routing.allocation.exclude._id

• Draining a node before offline.
• Zone / rack maintenance windows.
• Blue/green style rolling replacement where excluded and eligible nodes co-exist for a period.
• Tiered clusters where a subset of nodes is filtered per index.

I will reproduce this issue in the test cluster.

• In a 6-node cluster, create an test2 index with 18 primary shards and 1 replica.
• Execute the command to drain 3 of the nodes.
• It is expected that the remaining 3 nodes each hold 6 primary shards. However, byte-es-bytestore-test-cluster-data-0 holds 7 primary shards, while node byte-es-bytestore-test-cluster-data-2 holds 5 primary shards, and the system has been in a state of long-term primary shard imbalance.

• It can also be observed in the logs of the master node that the master considers the weights of byte-es-bytestore-test-cluster-data-0 and node byte-es-bytestore-test-cluster-data-2 to be similar, and thus no rebalance is required.

[2026-08-25T19:28:21,822][TRACE][o.o.c.r.a.a.BalancedShardsAllocator] [byte-es-bytestore-test-cluster-master-0] Stop balancing index [test2]  max_node [SmQs9LQYQXivTX5K6Zvz
CQ] weight: [1000035.25]  min_node [KwMlRXiBR5-dFN3R16qYUQ] weight: [1000034.8]  delta: [0.4375]

Describe the solution you'd like

To ensure the predictability of weight calculation, I will make all weight calculations aware of the allocation filter.

Related component

ShardManagement:Placement

Describe alternatives you've considered

No response

Additional context

No response

[guojialiang92]

Hi, @shwetathareja @linuxpi @gbbafna @Bukhtawar @atris
I'd also like to hear suggestions from the community, and I'll submit a PR later.

[alchemist51]

@guojialiang92 I'm kind of worried that at places we are using nodes.size() for example in avgShardsPerNode. What are the challenges to extend it for all calculations? I'm coming from the point that there are other constraints which are dependent on node size, changing it at few places will make it unpredictable for weight calculation

[guojialiang92]

@alchemist51 Thank you for your reply :)

I'm coming from the point that there are other constraints which are dependent on node size, changing it at few places will make it unpredictable for weight calculation

This is a valuable suggestion. To ensure the predictability of weight calculation, I will make all weight calculations aware of the allocation filter. The relevant descriptions in the RFC and the PR will also be updated.
I am looking forward to your feedback on PR.