Topology-Aware scheduling based on sched domains - #1
Conversation
Add helpers to get the NUMA node containing an LLC and the NUMA distance between two LLCs for the following patches. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
There is no straightforward way to retrieve the NODE-level sched domain, which will be used in subsequent patches. To simplify this, add a per-CPU variable `sd_node` for convenience. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
In the current implementation, the preferred LLC is selected based on the LLC with the largest running time of the thread group. However, the preferred LLC may be prone to frequent migration when the workload spreads across the entire system—especially when the number of CPUs sharing an LLC is small. A better approach is to first select a preferred NUMA node in the same way, and then select the preferred LLC within that preferred NUMA node. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
sched/fair: Decide whether to migrate for a generic node Cache-aware scheduling focuses only on the LLC. It makes migration decisions based on a strict policy: whether the destination LLC is the preferred LLC. This loses other important information such as NUMA distance. To utilize NUMA distance information, we extend the concept of a node from the LLC to a generic one. This generic node can refer to an LLC, a NUMA node, or a group of NUMA nodes, and can be represented by a sched domain above the LLC level. Based on this, a helper function to decide whether migration can occur between generic nodes is introduced. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
In the current implementation of the cache-aware scheduler, the policy for finding the best src group/rq only checks whether the preferred LLC of the task on the rq matches the dst LLC. However, migration may still improve affinity even if the preferred LLC does not match the dst LLC. For example, the src CPU resides in NODE0 and the dst CPU in NODE1. There is a task on the src CPU whose preferred LLC is in NODE2. Assume the node distance between NODE0 and NODE2 is 20, while the distance between NODE1 and NODE2 is 15. The task thus achieves better affinity after migrating to the dst CPU, but this case is not covered by the current implementation. To address this issue, introduce a new algorithm for calculating the affinity promotion given a dst LLC, src LLC, and rq. It can be roughly split into two steps: 1. Given the dst LLC and src LLC, iterate all LLCs in the system that can improve affinity when a task (with that LLC as its preferred LLC) migrates from the src CPU to the dst CPU. Compute the node distance difference for each corresponding LLC using the following formula: Di = llc_distance(src_llc, LLCi) - llc_distance(dst_llc, LLCi) (1) where i is the index of each system LLC. The minimum value of Di is clamped to 2 to avoid division by zero. 2. Given the LLCs from step 1 and a target rq, calculate the total affinity promotion quantity of the rq by accumulating the promotion value of each task on the rq. For each rq, the quantity is computed as: W_i = Rt_i * 1024 / Di (2) p = Σ_i W_i (3) where p represents the total affinity promotion quantity, and Rt_i is the number of tasks on the rq with LLCi as their preferred LLC. Rt_i can be obtained from rq->sd->pf. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
In the current implementation, only the preferred LLC is consideredwhen finding the source group and source rq. The previous patch provides a way to calculate affinity improvementfor a rq given a source LLC and a destination LLC. We can utilize this method to find src group and rq. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
As the group type for group_llc_balance focuses not only on the LLC level, but also on the NUMA level, the prefer_sibling constraint should be removed for it. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
The current implementation only focuses on LLC level and ignores affinity in wider domains such as NUMA level. This change takes all affinity information, including cache locality and NUMA-wide affinity, to evaluate whether migration can improve locality. Node load check is added to make the final migration decision. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
Cache-aware scheduling lacks memory awareness; remote memory access may still exist even with aggregated threads. Current designs let cache-aware scheduling follow NUMA balancing, but this breaks sched cache's preferred node selection logic. We propose an alternative: let sched cache override NUMA balancing. Sched cache handles task placement and migration, while NUMA balancing only manages memory migration. This combines the strengths of both mechanisms: sched cache is good at task aggregation and balancing, NUMA balancing is good at memory awareness. An interface is provided to disable this mode and revert to defaults. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
When a preferred LLC is selected and remains stable, task_cache_work does not need to run frequently. Because it scans all system CPUs for computation, high-frequency execution hurts performance. We thus reduce the scan rate in such cases. On the other hand, if the preferred node becomes suboptimal, we should increase the scan frequency to quickly find a better placement. The scan period is therefore dynamically adjusted. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
Consider a scenario with two thread groups (one larger than the other) running on an unsuitable node. The load balancer currently migrates tasks randomly, which slows down thread group migration. A new preferred node is selected only after most tasks of a thread group have been migrated. A better policy is to prioritize tasks from the smaller thread group to accelerate group migration. This patch introduces a secondary best task mechanism. The best task comes from the thread group with the highest CPU utilization in the sched group; the secondary best task comes from the thread group with the next highest utilization. Signed-off-by: Jianyong Wu <wujianyong@hygon.cn>
|
"This patch set introduces an improved mode that combines the advantages NUMA balancing is going to determine a preferred node, based on memory In version 4 of the cache aware scheduling Do you have workload performance data? Tim |
|
Hi Tim, sorry to late reply.
Agreed, memory is more expensive to migrate than threads.My goal is to let thread groups find their own node (LLC or NUMA), then migrate pages to their preferred node to avoid remote access. I believe this can be done with my approach, but I’m not sure the default implementation can achieve the same.
Another concern is that once NUMA balance is enabled, it may conflict with cache-aware scheduling.The root cause may be the difference in design philosophy between NUMA balance and cache-aware scheduling.NUMA balance tends to avoid conflicting with load balancing.For example, if we start just one thread group on a large system, its threads may spread across the entire system.Cache-aware scheduling, by contrast, would aggregate the threads within a single node.
I have designed a test: run several KVM virtual machines (which may have different vCPU numbers) on a multi-NUMA system. Each VM runs UnixBench internally. Thanks
|
2 participants
The current implementation of cache-aware scheduling only focuses on
cache awareness for task aggregation. It has two drawbacks: first, it
neglects NUMA-level affinity information; second, it cannot adapt to
tasks with large thread counts, especially on systems with a small LLC.
This patch set aims to overcome the above issues by extending cache-
aware scheduling to topology-aware scheduling, adding awareness of the
system topology and scheduling domains from the bottom up. With this
change, tasks with an arbitrary number of threads can be aggregated
starting from the MC level up to a corresponding scheduling domain
level. That is, if the CPU utilization of a workload is low, tasks may
be aggregated within one LLC; as the load increases, they may be
aggregated to a NUMA node, then to a socket, and vice versa.
Besides the above, this series includes further optimizations.
The existing cache-aware scheduler only focuses on task migration
without memory awareness, so it cannot avoid remote memory access.
NUMA balancing can help mitigate this issue. However, when NUMA
balancing is enabled, the current implementation causes interference
between cache-aware scheduling and NUMA balancing: the preferred node
selected by NUMA balancing can override the preferred LLC chosen by
cache-aware scheduling, which undermines the functionality of cache-
aware scheduling.
This patch set introduces an improved mode that combines the advantages
of cache-aware scheduling and NUMA balancing. In this mode, cache-aware
scheduling is responsible for selecting the preferred node and
performing task migration, while NUMA balancing handles memory
migration.
The last optimization is based on the observation that cache-aware
scheduling may introduce performance overhead because task_cache_work
consumes CPU cycles for computations on systems with a large number of
CPUs. It is therefore better to reduce or pause such computation once
the thread group becomes stable.
This patch set implements a dynamic mechanism to adjust the frequency
of task_cache_work, thereby reducing its performance impact.
Signed-off-by: Jianyong Wu wujianyong@hygon.cn