Re: [PATCH v3 14/20] kbuild: batch module finalisation

[Kees Cook]

On Thu, Sep 17, 2026 at 05:06:24PM +0100, Lorenzo Stoakes (ARM) wrote:
> With the .mod.S change in place, module finalisation on allmodconfig builds
> consists of a large number of very short-lived jobs.
> 
> For allmodconfig x86-64 this can be on the order of ~22,000 jobs of a few
> milliseconds in duration each.
> 
> Each job entails processing ~22k .cmd files, so the combination of heavy
> overhead and small individual job results in a lot of unnecessary and
> repeated work even with all cores being utilised.
> 
> The solution is to batch by a number of jobs. Determining which value makes
> sense was done empirically.
> 
> On a 128-thread threadripper box doing an allmodconfig build, best of
> 2, *.ko, *.mod.o deleted each time:
> 
>     modules per chunk    instances    wall
>     -----------------    ---------    ------
>                     1        11171    10.22s
>                     2         5586     6.72s
>                     4         2793     4.97s
>                     8         1397     4.26s
>                    16          699     4.05s
>                    32          350     4.05s
>                    64          175     4.07s
>                   128           88     4.05s
>                   256           44     4.03s
>                   512           22     4.18s
> 
> Wall time flattens for 16-256 module batches.
> 
> A slower/lower core machine will do better with fewer modules-per-batch, a
> faster/higher core machine will do better with more modules-per-batch.
> 
> Therefore, take the midpoint which works in the most margin in either
> direction - 128 modules per batch.

This is easy to tweak, so if we get negative feedback on this for
smaller machines, it's an easy fix.

> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>

Reviewed-by: Kees Cook <kees@xxxxxxxxxx>

-- 
Kees Cook