[PATCH v3 3/4] sched/fair: Allow load balancing between CPUs of identical capacity

[Ricardo Neri]

sched_balance_find_src_rq() avoids selecting a runqueue with a single
running task as busiest if doing so results in migrating the task to a
CPU with less than ~5% of extra capacity. It also unintentionally
prevents migrations between CPUs of identical capacity.

When CONFIG_SCHED_CLUSTER is enabled, load should be balanced across
clusters of CPUs with the same capacity. Allowing migration between CPUs
of identical capacity is necessary to meet this goal.

Use arch_scale_cpu_capacity() to reflect architectural capacity, excluding
runtime reductions due to side activity or thermal pressure. Guard this
check with the sched_cluster_active static key so that systems without
cluster topology are unaffected.

Signed-off-by: Ricardo Neri <ricardo.neri-calderon@xxxxxxxxxxxxxxx>
---
Changes in v3:
 * Reverted the inverted capacity check; the inverted form incorrectly
   allows migrations to CPUs of slightly less capacity.
 * Guarded the check for architectural capacity with the
   sched_cluster_active static key.

Changes in v2:
 * Used arch_scale_cpu_capacity() instead of capacity_of() to ignore
   runtime variability.
 * Inverted the check for runtime capacity. (Christian)
 * Reworded patch description for clarity.
---
 kernel/sched/fair.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index dcc02ceb44b5..d2a4c529f67f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -11846,8 +11846,14 @@ static struct rq *sched_balance_find_src_rq(struct lb_env *env,
 		 * eventually lead to active_balancing high->low capacity.
 		 * Higher per-CPU capacity is considered better than balancing
 		 * average load.
+		 *
+		 * CONFIG_SCHED_CLUSTER requires balancing load across clusters
+		 * of identical capacity. Use architectural capacity to ignore
+		 * runtime variability.
 		 */
 		if (env->sd->flags & SD_ASYM_CPUCAPACITY &&
+		    (!static_branch_unlikely(&sched_cluster_active) ||
+		     arch_scale_cpu_capacity(env->dst_cpu) != arch_scale_cpu_capacity(i)) &&
 		    !capacity_greater(capacity_of(env->dst_cpu), capacity) &&
 		    nr_running == 1)
 			continue;

-- 
2.43.0