Re: [RFC PATCH v2 0/3] BPF-based per-cgroup NUMA balancing mode selection

[Yafang Shao]

On Tue, Oct 13, 2026 at 11:49 PM Gregory Price <gourry@xxxxxxxxxx> wrote:
>
> On Mon, Oct 12, 2026 at 08:04:35PM +0800, Yafang Shao wrote:
> > The Problem
> > ===========
> >
> > In our large fleet of Kubernetes-managed servers, NUMA balancing has
> > been historically disabled globally on each server. With the increasing
> > deployment of multi-socket AMD EPYC servers, cross-NUMA access has
> > become a critical performance issue, which made us reconsider.
> >
> > Enabling NUMA balancing globally is still not acceptable: it increases
> > the overall system overhead and can introduce latency spikes for
> > latency-sensitive workloads. What we need is to choose per workload
> > whether NUMA balancing runs at all, and which mode it runs in. A
> > latency-sensitive service may want the scheduler's NUMA statistics
> > without tiering's promotion machinery, while a batch service wants its
> > PMEM-resident memory promoted when hot without paying for
> > scheduler-driven migration. No single global switch serves both.
>
> Have you tested with recent fixes?  NUMA balancing has been broken for a
> few years now (starvation issues causing VMAs to never be scanned,
> east-west filters causing south->north promotions to be skipped, shmem
> being entirely exempt, others).
>
> https://lore.kernel.org/linux-mm/20260629163337.1264881-1-hannes@xxxxxxxxxxx/
> https://lore.kernel.org/linux-mm/20260930112206.205083-1-gourry@xxxxxxxxxx/

Thanks for sharing these.

These look different from the latency spikes we've seen on our
production servers — ours occurred with NUMA balancing in normal mode
— though I haven't yet figured out whether ours is shmem-related.
Given its problematic history, it's hard for us to re-enable it in
production globally. That said, I'll try to backport the fixes above
to our internal kernel.

>
> I also concur with Tejun that ops seems like overkill for this feature,
> this seems like something that can already be managed.

Tejun also suggested starting with a smaller kfunc rather than the
heavyweight struct_ops, in a separate mailing list thread. I agree
that this would be a good starting point.

>
> For example, why not add a new numa balancing mode bit that says
> "enabled but only for explicitly opted in tasks" and then ignore the
> global default mempolicy, e.g.:
>
>   if (numab_mode & NUMA_BALANCING_OPT_IN &&
>       mpol != default &&
>       mpol.flags & MPOL_F_NUMA_BALANCING)
>       /* do balancing */

Another option would be to add a cgroup::numab_mode field (without
exposing a cgroupfs interface file for now) to implement per-cgroup
numab mode. I will think it over, considering both your suggestion and
Tejun's.


--
Regards
Yafang