Re: Cache-aware scheduling does not work well with amd big/little cores

[Klaus Kusche]

I did some very quick tests.

1.) /sys/kernel/sched/debug/domains/* does not exist on my system,
not even with debug_fs on.

/sys/kernel/debug/x86/sched_itmt_enabled is "Y",
/sys/kernel/debug/x86/sched_core_priority looks good
(big cores have values almost twice as high as small cores)

2.) The situation with 7.2.5 which seems to include 
https://lore.kernel.org/lkml/20260825174112.2580942-1-tim.c.chen@xxxxxxxxxxxxxxx/
is almost unchanged: Without cache aware scheduling, 
my build jobs run faster (wallclock time):
Just 6:16 compared to 6:19 for my kernel build with full LTO,
but 4:50 compared to 5:20 (???) for my python uv build
(the python uv build seems to be a very interesting test case?)

However, with cache sched enabled, in spite of the longer 
wallclock time, cpu seconds are sometimes a little bit lower.

3.) https://lore.kernel.org/lkml/20260810033742.1688718-1-yu.c.chen@xxxxxxxxx/
seems to make things much worse:
Kernel builds had the LTO step and the CC compressed step
placed on little cores for significant amounts of time,
resulting in total build times above 8 minutes.
Same impression by watching the bar graph for individual cores
for the uv build.

Prof. Dr. Klaus Kusche
Privat: Söllmnitz 32 d, D-07554 Gera/Söllmnitz
036695/859909 klaus.kusche@xxxxxxxxxxxxxxx https://www.computerix.info
Dienstlich: DHGE Gera, Weg der Freundschaft 4, D-07546 Gera
klaus.kusche@xxxxxxx https://www.dhge.de

On 10/09/2026 03:29, Chen, Yu C wrote:
> On 9/10/2026 3:51 AM, Tim Chen wrote:
>> On Wed, 2026-09-09 at 08:19 -0500, Mario Limonciello wrote:
>>>
>>> On 9/9/26 03:59, Klaus Kusche wrote:
>>>>
>>>> Hello,
>>>>
>>>> On 08/09/2026 23:54, Tim Chen wrote:
>>>>>> I did some quick testing (no perfect benchmark environment,
>>>>>> just checking runtime and CPU consumption with "time").
>>>>>>
>>>>>> I timed a kernel build (-j 24 and full lto, my own .config)
>>>>>> and an application build (also with a lot of parallelism)
>>>>>> with three different kernels:
>>>>>>
>>>>>> a) Cache aware scheduling completely configured off
>>>>>>
>>>>>> b) Cache aware scheduling turned on, but without patch
>>>>>>
>>>>>> c) Cache aware scheduling turned on, with patch
>>>>>>
>>>>>> Big/little scheduling was always on,
>>>>>> Mario's patch was always applied
>>>>>> (without it, results are significantly worse,
>>>>>
>>>>> Sorry, I am a bit confused.  You mentioned later the result for (b)
>>>>> and (c) are about the same with or without the patch
>>>>> exposing debugfs (commit c1e7fe5e75ed11fa85368e5a186472afd3858f3a
>>>>> Mario mentioned in another mail).
>>>>> But here you say the result is much worse without Mario's patch.
>>>>> Is Mario's patch the one above or some other patch?
>>>>
>>>> The with/without patch in (b) and (c)
>>>> refers to the patch you sent on 31/08/2026
>>>> ( https://lore.kernel.org/lkml/20260825174112.2580942-1-tim.c.chen@xxxxxxxxxxxxxxx/ ),
>>>> not to Mario's patch.
>>>>
>>>
>>> Just to clarify Mario's patch in this context refers to the fix to
>>> ITMT/debugfs fixes as Klaus doesn't nominally enable debugfs in Kconfig:
>>>
>>> eaece4849991d62fcd6f46637c55dcce00e25d70
>>>
>>
>> Ricardo reminded me that AMD's hybrid CPU relies on SD_ASYM_PACKING instead
>> of SD_ASYM_CPUCAPACITY. So the patch I pointed to
>> (https://lore.kernel.org/lkml/20260825174112.2580942-1-tim.c.chen@xxxxxxxxxxxxxxx/)
>> only fixes the SD_ASYM_CPUCAPACITY case and would not have an effect on
>> your test system.
>>
>> The fact that ITMT needs to be turned on to improve performance also point
>> in that direction.
>>
>> There was a bug that Chen Yu fixes for ITMT compatability with cache aware scheduling.
>> It prevents a task from getting stuck in the wrong LLC in the ITMT case
>> (https://lore.kernel.org/lkml/20260810033742.1688718-1-yu.c.chen@xxxxxxxxx/).
>> Klaus, Can you apply that with Mario's ITMT/debugfs patch to see if that
>> improves performance on your system?
>>   
>> Tim
> 
> Yes, that patch tries to coordinate ITMT with cache-aware scheduling. It 
> would also
> be helpful to get a dump of /sys/kernel/sched/debug/domains/*  to 
> confirm that the
> SD_ASYM_PACKING flag is set at the MC domains as well as 
> /sys/kernel/debug/x86/sched_itmt_enabled
> to double-check that ITMT is enabled and 
> /sys/kernel/debug/x86/sched_core_priority to look at
> CPU priorities.
> 
> thanks,
> Chenyu