Re: [PATCH net-next 2/3] ipv6: take fib6_sernum from a sequence shared by all netns

[Eric Dumazet]

Le ven. 16 oct. 2026 à 08:10, Qiliang Yuan <odys.yuan@xxxxxxxxx> a écrit :
>
> A per-cpu IPv6 route of a nexthop object records fib6_sernum of the
> netns it was created in, and rt6_is_valid() compares it with fib6_sernum
> of the netns its device belongs to when the route is checked. Every netns
> starts its fib6_sernum at 1 and increments it on each tree change, so
> different netns often hold the same value.
>
> When a device moves to another netns, such a route remains usable until
> nexthop_free_rcu() releases it, and fib6_info_destroy_rcu() queues that
> callback from inside an RCU callback, after the device has moved. If the
> fib6_sernum of the new netns matches the cached one in that window,
> ip6_dst_check() accepts the route and a socket in the old netns keeps
> sending through a device that now belongs to another netns.
>
> Take every fib6_sernum, including the initial one, from one global
> sequence, so that a cached route never matches the fib6_sernum of
> another netns.
>
> A connected UDP socket in netns A caches a route through an IPv6 nexthop
> object on a veth. The veth is moved to netns B and brought up with one
> RTM_NEWLINK, then the socket sends 200000 datagrams. A route is added
> to and removed from B 0 to 20 times before the move, 63 trials:
>
>                                 before     after
>   trials sending after move          3         0
>   datagrams sent after move       8187         0
>
> Fixes: 8f34e53b60b3 ("ipv6: Use global sernum for dst validation with nexthop objects")
> Signed-off-by: Qiliang Yuan <odys.yuan@xxxxxxxxx>
> ---
>  net/ipv6/af_inet6.c |  1 -
>  net/ipv6/ip6_fib.c  | 14 ++++++++++++--
>  2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
> index a9b8b4b42a1af..db4cccfdc565c 100644
> --- a/net/ipv6/af_inet6.c
> +++ b/net/ipv6/af_inet6.c
> @@ -945,7 +945,6 @@ static int __net_init inet6_net_init(struct net *net)
>         net->ipv6.sysctl.max_dst_opts_len = IP6_DEFAULT_MAX_DST_OPTS_LEN;
>         net->ipv6.sysctl.max_hbh_opts_len = IP6_DEFAULT_MAX_HBH_OPTS_LEN;
>         net->ipv6.sysctl.fib_notify_on_flag_change = 0;
> -       atomic_set(&net->ipv6.fib6_sernum, 1);
>
>         net->ipv6.sysctl.ioam6_id = IOAM6_DEFAULT_ID;
>         net->ipv6.sysctl.ioam6_id_wide = IOAM6_DEFAULT_ID_WIDE;
> diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
> index 0dc6ebee6e568..c66466be64451 100644
> --- a/net/ipv6/ip6_fib.c
> +++ b/net/ipv6/ip6_fib.c
> @@ -89,13 +89,21 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
>         write_unlock_bh(&net->ipv6.fib6_walker_lock);
>  }
>
> +/*
> + * Draw every fib6_sernum from one sequence shared by all netns, so that a
> + * route cached against one netns never matches the sernum of another one,
> + * e.g. after its device moved.
> + */
> +static atomic_t fib6_sernum_seq;
> +
>  static int fib6_new_sernum(struct net *net)
>  {
> -       int new, old = atomic_read(&net->ipv6.fib6_sernum);
> +       int new, old = atomic_read(&fib6_sernum_seq);
>
>         do {
>                 new = old < INT_MAX ? old + 1 : 1;
> -       } while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
> +       } while (!atomic_try_cmpxchg(&fib6_sernum_seq, &old, new));
> +       atomic_set(&net->ipv6.fib6_sernum, new);
>

The atomic_set() of net->ipv6.fib6_sernum after the cmpxchg()
 is not ordered with other writers. fib6_new_sernum() callers only
 hold the per-table tb6_lock, so two tables of the same netns can
 race and leave the netns with the older of two values. Since values
 are never reused, I think this only causes extra invalidations, but
 the changelog should say so: fib6_sernum is no longer monotonic per
 netns.

>         return new;
>  }
> @@ -2466,6 +2474,8 @@ static int __net_init fib6_net_init(struct net *net)
>         size_t size = sizeof(struct hlist_head) * FIB6_TABLE_HASHSZ;
>         int err;
>
> +       fib6_new_sernum(net);

There is no need to draw a value from the shared sequence at netns
creation. Netns creation is exactly what you are trying to make
cheaper, and the shared cacheline does not need to be written there.

You can instead keep per-netns initial values that the shared
sequence never hands out:

keep atomic_set(&net->ipv6.fib6_sernum, 1) in
inet6_net_init() (0 is not usable, rt->sernum == 0 and
FIB6_NO_SERNUM_CHANGE have their own meaning), and make the
sequence never return 1 (restart at 2)

A netns that never bumped can then not match a value from the
sequence. Also, the moved device is down after the move, and
NETDEV_UP in the new netns bumps it (rt_cache_flush() for IPv4,
the route inserted by addrconf_add_dev() for IPv6) before the
device can transmit.