Re: [PATCH net-next v9] net: reduce RFS/ARFS flow updates by checking LLC affinity

[Eric Dumazet]

On Mon, Sep 14, 2026 at 5:01 AM chuang <nashuiliang@xxxxxxxxx> wrote:
>
> Although there is a slight semantic difference compared to
> net_hotdata.rps_sock_flow_table, meaning the CPU recorded in
> `sock_flow_table[flow_id].ent` might not perfectly align with the one
> used in `set_rps_cpu`. It is still the least intrusive approach.
>
> How about the following simple change?
>
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 3a0dd1f98084..7c513bf14221 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -5236,7 +5236,7 @@ static int get_rps_cpu(struct net_device *dev,
> struct sk_buff *skb,
>                  *     have been dequeued, thus preserving in order delivery.
>                  */
>                 if (unlikely(tcpu != next_cpu) &&
> -                   (tcpu >= nr_cpu_ids || !cpu_online(tcpu) ||
> +                   (tcpu >= nr_cpu_ids || !cpu_online(tcpu) ||
> !rps_check_llc_affinity(tcpu, next_cpu) ||
>                      ((int)(READ_ONCE(per_cpu(softnet_data,
> tcpu).input_queue_head) -
>                       rflow->last_qtail)) >= 0)) {
>                         tcpu = next_cpu;

This diff in get_rps_cpu() has two issues:

Boolean logic / packet reordering bug:
Adding || !rps_check_llc_affinity(tcpu, next_cpu) inside the || list
does not prevent calling set_rps_cpu() when the queue is drained
(because false || queue_drained still evaluates to true). Worse, when
!rps_check_llc_affinity() is true, it short-circuits the || expression
and bypasses the input_queue_head - rflow->last_qtail >= 0 check,
switching CPUs before the backlog drains and causing packet
reordering.

Per-packet slow path:
Even if gated with &&, skipping set_rps_cpu() leaves rflow->cpu (tcpu)
unchanged while sock_flow_table holds next_cpu.
This causes unlikely(tcpu != next_cpu) to evaluate to true on every
subsequent packet for that flow instead of once per migration.

Instead, place the check in set_rps_cpu() under #ifdef CONFIG_RFS_ACCEL:

diff --git a/net/core/dev.c b/net/core/dev.c
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5113,6 +5113,10 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
  if (!skb_rx_queue_recorded(skb) || !dev->rx_cpu_rmap ||
     !(rxqueue = dev->_rx))
  goto out;
+
+ if (static_branch_unlikely(&rps_feat_llc_affinity) &&
+    cpus_share_cache(raw_smp_processor_id(), next_cpu))
+ goto out;

  rxq_index = cpu_rmap_lookup_index(dev->rx_cpu_rmap, next_cpu);
  if (rxq_index == skb_get_rx_queue(skb))

This skips ndo_rx_flow_steer() when the CPU servicing the current RX
queue already shares an LLC with next_cpu,
while still reaching WRITE_ONCE(rflow->cpu, next_cpu) at out:
so subsequent packets take the fast path (tcpu == next_cpu).