Re: Cache-aware scheduling does not work well with amd big/little cores

[Klaus Kusche]

I don't know what your last patch does (does it dynamically disable 
cache aware scheduling on big/little CPU's?),
but it makes my two build tests running at almost the same speed
on a kernel with cache aware scheduling
compared to a kernel without cache aware scheduling, 
which means it is rougly 2 % faster 
than earlier cache aware scheduling kernels.

Just by looking at the core bar graph,
I also didn't see any obviously misplaced processes.

Klaus Kusche

On 14/09/2026 15:13, Chen Yu wrote:
> On Mon, Sep 14, 2026 at 12:27:53PM +0200, Klaus Kusche wrote:
>>
>> I did some very quick tests.
>>
>> 1.) /sys/kernel/sched/debug/domains/* does not exist on my system,
>> not even with debug_fs on.
>>
>> /sys/kernel/debug/x86/sched_itmt_enabled is "Y",
>> /sys/kernel/debug/x86/sched_core_priority looks good
>> (big cores have values almost twice as high as small cores)
>>
>> 2.) The situation with 7.2.5 which seems to include 
>> https://lore.kernel.org/lkml/20260825174112.2580942-1-tim.c.chen@xxxxxxxxxxxxxxx/
>> is almost unchanged: Without cache aware scheduling, 
>> my build jobs run faster (wallclock time):
>> Just 6:16 compared to 6:19 for my kernel build with full LTO,
>> but 4:50 compared to 5:20 (???) for my python uv build
>> (the python uv build seems to be a very interesting test case?)
>>
>> However, with cache sched enabled, in spite of the longer 
>> wallclock time, cpu seconds are sometimes a little bit lower.
>>
>> 3.) https://lore.kernel.org/lkml/20260810033742.1688718-1-yu.c.chen@xxxxxxxxx/
> 
> This patch inhibits ASYM_PACKING and favors Cache-Aware-Scheduling,
> which is the opposite of what your platform expects.
> 
>> seems to make things much worse:
>> Kernel builds had the LTO step and the CC compressed step
>> placed on little cores for significant amounts of time,
>> resulting in total build times above 8 minutes.
>> Same impression by watching the bar graph for individual cores
>> for the uv build.
>>
> 
> After a second thought, I wonder if we should disable Cache-Aware
> Scheduling if ASYM_PACKING is enabled, because the latter would
> prefer a higher priority CPU rather than just choosing a random
> L3 to aggregate the threads. Would the following patch work
> for you? (just compile tested, as I do not have a multi-LLC hybrid
> AMD platform for testing) Thanks.
> 
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 0248227d983a..07bc8a302574 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -684,6 +684,7 @@ DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
>  
>  DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
>  DEFINE_STATIC_KEY_FALSE(sched_cluster_active);
> +DEFINE_STATIC_KEY_FALSE(sched_asym_packing_active);
>  
>  static void update_top_cache_domain(int cpu)
>  {
> @@ -957,6 +958,11 @@ static void _sched_cache_active_set(void)
>  		return;
>  	}
>  
> +	if (static_branch_unlikely(&sched_asym_packing_active)) {
> +		static_branch_disable_cpuslocked(&sched_cache_active);
> +		return;
> +	}
> +
>  	/*
>  	 * user wants it or not ?
>  	 * TBD: read before writing the static key.
> @@ -3085,6 +3091,7 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
>  	int i, ret = -ENOMEM;
>  	bool has_asym = false;
>  	bool has_cluster = false;
> +	bool has_asym_packing = false;
>  
>  	if (WARN_ON(cpumask_empty(cpu_map)))
>  		goto error;
> @@ -3204,6 +3211,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
>  
>  		if (lowest_flag_domain(i, SD_CLUSTER))
>  			has_cluster = true;
> +
> +		if (highest_flag_domain(i, SD_ASYM_PACKING))
> +			has_asym_packing = true;
>  	}
>  	rcu_read_unlock();
>  
> @@ -3213,6 +3223,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
>  	if (has_cluster)
>  		static_branch_inc_cpuslocked(&sched_cluster_active);
>  
> +	if (has_asym_packing)
> +		static_branch_inc_cpuslocked(&sched_asym_packing_active);
> +
>  	if (rq && sched_debug_verbose)
>  		pr_info("root domain span: %*pbl\n", cpumask_pr_args(cpu_map));
>  
> @@ -3318,6 +3331,9 @@ static void detach_destroy_domains(const struct cpumask *cpu_map)
>  	if (static_branch_unlikely(&sched_cluster_active))
>  		static_branch_dec_cpuslocked(&sched_cluster_active);
>  
> +	if (rcu_access_pointer(per_cpu(sd_asym_packing, cpu)))
> +		static_branch_dec_cpuslocked(&sched_asym_packing_active);
> +
>  	rcu_read_lock();
>  	for_each_cpu(i, cpu_map)
>  		cpu_attach_domain(NULL, &def_root_domain, i);