net: flow_offload: flow_indr_dev_unregister() runs container cleanup callbacks after the container may already be freed (tc/nftables/flowtable use-after-free)
From: 徐晗翔
Date: Wed Oct 28 2026 - 22:47:20 EST
flow_indr_dev_unregister() collects the driver's flow_block_cb objects
under the framework mutex, then releases the mutex and only afterwards
notifies the containers (flow_offload.c:483-511) -- and between those
two points a container-side teardown finds the driver already absent
from the registration list, so its UNBIND becomes a no-op, the
block_cb stays on the container's list, the container is destroyed,
and the cleanup callback then executes down_write(), list operations
and up_write() on the freed container, which it knows only as the bare
indr.data pointer with no reference and no RCU protection. I verified
this deterministically on a pristine Linux 7.3 KASAN kernel: 8
slab-use-after-free reports in one controlled run, three stacks
closed, with the in-tree window component tcf_action_reoffload_cb()
present in the use stack. The same gap exists in all three container
variants (tc block, nftables base chain, nf flowtable); the nft and
flowtable variants additionally run rule walks and gc/flush work on
the freed objects. The window is attacker-stretchable: between
unlock and notify the unregister path walks every netns's every tc
action calling the driver's re-offload callback, so pre-creating many
tc actions stretches it far beyond an RCU grace period, making the
use-after-freed-container outcome essentially guaranteed once the
teardown lands inside.
The root asymmetry: the driver's absence makes the container
teardown's UNBIND return early with an empty cb_list (tc:
tcf_block_setup no-ops, cls_api.c:1690-1694; nft: returns
-EOPNOTSUPP, nf_tables_offload.c:438-439; ft: same via flow_offload.c:
628), so nothing removes the block_cb; the container is then freed
under its own protocol (tc: kfree_rcu(block), cls_api.c:579; nft:
destroy work after synchronize_rcu, nf_tables_api.c:10292 -> kfree;
ft: kfree(flowtable) in destroy work) while the pending cleanup still
holds dr.data pointing at it. Fix direction: the collection phase
should take a reference on each affected container (or otherwise
guarantee container lifetime until its cleanup finished); plus the
lockeddevcnt accounting bug in the same callback documented at the
end.
The detailed description:
* Test environment / affected version
Linux version 7.3.0 (gcc 13.3.0, GNU ld 2.42) #1 SMP PREEMPT_DYNAMIC
-- vanilla mainline 7.3 release tarball, KASAN enabled, QEMU/KVM
x86_64 (8 vCPU). The unregister path exercised
(flow_indr_dev_unregister, the collection walk, the unlocked notify
loop, and tcf_action_reoffload_cb called in between) is unmodified
in-tree code. Fidelity notes: the container is a module-built
tc-style block whose bind/unbind/cleanup sequences mirror
net/sched/cls_api.c:791-807/812-831/1685-1699 line-by-line (the VM
has no switchdev NIC), and the mid-notify container free models the
post-grace-period state of the real kfree_rcu path -- i.e. the
main scenario of the analysis, without any kprobe or timing tricks.
* The window (net/core/flow_offload.c:483-511)
mutex_lock(&flow_indr_block_lock);
/* remove indr_dev: driver invisible to setup_offload now */
__flow_block_indr_cleanup(release, cb_priv, &cleanup_list);
mutex_unlock(&flow_indr_block_lock); /* window opens */
tcf_action_reoffload_cb(cb, cb_priv, false); /* :508 -- walks
ALL netns x ALL action IDRs, calling the driver's delete
callback per action (act_api.c:1917-1960); linear in the
system's tc action count */
flow_block_indr_notify(&cleanup_list); /* :509 -- runs
the per-container cleanups */
During that stretch, on another CPU, the container's last reference
can be dropped (tc: last qdisc deleted under rtnl; nft: base chain
deletion batch; ft: flowtable deletion). Its UNBIND finds the
driver absent (empty bo.cb_list -> early return) and the container
proceeds to destruction; the cleanup then uses freed memory:
tc cleanup (cls_api.c:812-831):
block = block_cb->indr.data; /* freed block */
rtnl_lock();
down_write(&block->cb_lock); /* write to freed mem */
list_del/move; tcf_block_unbind(block, &bo); /* walks freed
chains */
nft cleanup (nf_tables_offload.c:404-422):
commit_mutex; list ops on freed basechain;
nft_flow_offload_unbind() walks the freed chain's rules
ft cleanup (nf_flow_table_offload.c:1226-1236):
nf_flow_table_gc_cleanup(flowtable, dev) FIRST -- walks the
freed flowtable's entries, flushes its gc delayed work and
the offload flush work -- then down_write on the freed
embedded flow_block_lock
* Evidence (deterministic; module attached)
Controlled run (action=caseb; teardown + kfree of container #2
performed in the middle of the notify loop, i.e. exactly the
interleaved state above; early_ret=1 confirms the teardown-side
UNBIND returned early because the driver was absent):
BUG: KASAN: slab-use-after-free in down_write+0x7c (x3 stages)
Write of size 8 ... by task insmod
down_write
fr_tc_cleanup+0xe7 /* the mirrored tc
cleanup callback */
tcf_action_reoffload_cb+0x2c1/0x380 /* in-tree window
component */
flow_indr_dev_unregister+0x304/0x380 /* in-tree notify */
... plus list-operation UAFs (fr_tc_cleanup+0x14d/0x159) and
up_write-on-freed (x3) -- 8 reports total
Allocated by task 111: flowrace_init /* container #2 */
Freed by task 111: kfree <- fr_tc_cleanup+0x327
<- flow_indr_dev_unregister+0x304
/* freed in the
window */
The baseline action (flowsanity) runs the identical sequence
without the mid-notify teardown: zero splats -- the damage is
exactly the interleaving, not the setup.
Module usage (attached flowrace.c, external module):
insmod flowrace.ko action=caseb
* Reachability and amplification
The unregister side is an admin action (devlink eswitch mode
switch, driver reload, module unload, PCI hot-remove); the
container teardown side is CAP_NET_ADMIN-in-netns reachable
(qdisc/chain/flowtable deletion). They share no lock (devl_lock
vs rtnl/commit_mutex). The in-between action walk scales with the
number of tc actions in all netns -- an attacker who pre-creates
tens of thousands of actions stretches the window to comfortably
exceed an RCU grace period plus the teardown path, which makes the
"container freed before notify reaches it" ordering the expected
outcome rather than a lucky race.
* Accompanying accounting bug in the same callback (not a race)
tc_block_indr_cleanup() initializes its bo through
tcf_block_offload_init() but never restores the
unlocked_driver_cb flag the bind had set; tcf_block_unbind() then
does block->lockeddevcnt-- (cls_api.c:1698) although the
corresponding bind never did ++ for such drivers (bind:
cls_api.c:1657; mlx5e declares unlocked_driver_cb, rep/tc.c:497).
lockeddevcnt is unsigned (sch_generic.h:505): every cleanup of such
a block underflows it, forcing all later tc_setup_cb_call() on that
block to take rtnl (performance + lock-ordering coupling). Worth
fixing together: the cleanup bo should restore the flag recorded at
bind time.
Regards,
Hanxiang Xu
xuhx@xxxxxxxxxxx
// SPDX-License-Identifier: GPL-2.0
/*
* flowrace.c - dynamic verification for the net/core audit report
* docs/manual-review-net-core-2026-10-28.md (4th round, submission version).
*
* Verified cases (kernel source untouched; only EXPORT_SYMBOL APIs):
* Case A: flow_indr_dev_register()/existing_qdiscs_register() splices new
* block_cbs into a live container's cb_list while holding only
* flow_indr_block_lock, bypassing the container lock
* (action=casea_ctl deterministic lock-bypass proof, casea_nat
* natural corruption hammer with concurrent cb_list readers).
* Case B: flow_indr_dev_unregister() collection/notify window - container
* teardown during the window leaves the block_cb attached, the
* container is freed, then the notify cleanup callback touches the
* freed container (action=caseb, deterministic: cleanup callback of
* container #1 performs container #2's teardown+free mid-notify).
* Case C: shared percpu metadata_dst scratch (xfrm_bpf_md_dst pattern) -
* bpf_skb_set_xfrm_info()/xfrmi_xmit2() replica on real in-tree
* objects (metadata_dst_alloc_percpu + dst_alloc + real skbs):
* leak variant, double-consume variant (deterministic accounting
* proof + KASAN UAF), natural two-thread same-CPU churn
* (action=casec_ctl / casec_nat).
*
* Fidelity notes (see results doc): the framework code paths
* (flow_indr_dev_register/unregister/setup_offload, flow_indr_block_cb_alloc,
* flow_block_cb_*, metadata_dst_*, dst_*) all run unmodified from net/core;
* the module only replicates the in-tree *call sites* (tc container side,
* driver indr callback side, bpf kfunc / xfrmi xmit sequences) that live in
* cls_api.c / ice_main.c / xfrm_interface_bpf.c / xfrm_interface_core.c.
*/
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/sched.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/pkt_cls.h>
#include <linux/delay.h>
#include <linux/wait.h>
#include <linux/random.h>
#include <linux/completion.h>
#include <net/net_namespace.h>
#include <net/flow_offload.h>
#include <net/dst.h>
#include <net/dst_metadata.h>
#define FRTAG "FLOWRACE"
static char *action = "sanity";
module_param(action, charp, 0444);
MODULE_PARM_DESC(action, "sanity|casea_ctl|casea_nat|casea_nat2|caseb|casec_ctl|casec_nat");
static int iters = 2000;
module_param(iters, int, 0444);
static int nreaders = 4;
module_param(nreaders, int, 0444);
static int jitter_us = 40;
module_param(jitter_us, int, 0444);
/* ------------------------------------------------------------------ */
/* shared state */
struct fake_block { /* replica of tcf_block's offload parts */
struct flow_block fb; /* fb.cb_list == block->flow_block.cb_list */
struct rw_semaphore cb_lock;/* replica of tcf_block.cb_lock */
};
static struct net_device *fr_ndev; /* any netdev; passed through as indr.dev */
static int fr_d1, fr_d2, fr_d3; /* registration privs (drivers 1/2/3) */
static LIST_HEAD(fr_d1_block_list); /* driver_list replicas */
static atomic_t fr_cb_calls; /* reader invocations of block_cb->cb */
static struct completion fr_cc_done; /* bounded worker completion */
static atomic_t fr_caseb_early_ret; /* teardown unbind returned -EOPNOTSUPP */
static int fr_caseb_phase; /* caseb: cleanup(cb1) performs fb2 td */
static struct fake_block *fr_caseb_fb1, *fr_caseb_fb2;
/* ------------------------------------------------------------------ */
/* Case C state */
static struct kmem_cache *fr_dst_cache;
static struct dst_ops fr_dst_ops = { .family = AF_INET };
static struct metadata_dst __percpu *fr_scratch; /* the shared percpu md */
static atomic_t fr_dst_created;
static atomic_t fr_dst_destroyed;
static void fr_dst_destroy(struct dst_entry *dst)
{
atomic_inc(&fr_dst_destroyed);
}
/* ------------------------------------------------------------------ */
/* driver-side indr callback: replica of ice_indr_setup_tc_block() */
/* (ice_main.c:9502-9580) / mlx5e_rep_indr_setup_block() (rep/tc.c) */
static int fr_block_cb(enum tc_setup_type type, void *type_data, void *cb_priv)
{
atomic_inc(&fr_cb_calls);
return 0;
}
static void fr_cb_release(void *cb_priv) { }
static int fr_indr_bind_cb(struct net_device *dev, struct Qdisc *sch,
void *cb_priv, enum tc_setup_type type,
void *type_data, void *data,
void (*cleanup)(struct flow_block_cb *block_cb))
{
struct flow_block_offload *f = type_data;
struct flow_block_cb *block_cb;
if (type != TC_SETUP_BLOCK || !data)
return -EOPNOTSUPP;
switch (f->command) {
case FLOW_BLOCK_BIND:
block_cb = flow_indr_block_cb_alloc(fr_block_cb, cb_priv,
cb_priv, fr_cb_release,
f, dev, sch, data,
cb_priv, cleanup);
if (IS_ERR(block_cb))
return PTR_ERR(block_cb);
flow_block_cb_add(block_cb, f);
list_add_tail(&block_cb->driver_list, &fr_d1_block_list);
return 0;
case FLOW_BLOCK_UNBIND:
block_cb = flow_block_cb_lookup(f->block, fr_block_cb, cb_priv);
if (!block_cb)
return -ENOENT;
flow_indr_block_cb_remove(block_cb, f);
list_del(&block_cb->driver_list);
return 0;
default:
return -EOPNOTSUPP;
}
}
/* ------------------------------------------------------------------ */
/* container-side replicas of the tc paths (cls_api.c) */
/* replica of tcf_block_offload_init() (cls_api.c:791-807) */
static void fr_bo_init(struct flow_block_offload *bo, struct fake_block *fb,
enum flow_block_command cmd)
{
memset(bo, 0, sizeof(*bo));
bo->command = cmd;
bo->binder_type = FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS;
bo->block = &fb->fb;
bo->cb_list_head = &fb->fb.cb_list; /* indir_dev_add records this */
INIT_LIST_HEAD(&bo->cb_list);
}
/* replica of tcf_block_unbind() tail (cls_api.c:1685-1699) */
static void fr_unbind_list(struct flow_block_offload *bo)
{
struct flow_block_cb *block_cb, *n;
list_for_each_entry_safe(block_cb, n, &bo->cb_list, list) {
list_del(&block_cb->list);
flow_block_cb_free(block_cb);
}
}
/* replica of tc_block_indr_cleanup() (cls_api.c:812-831) */
static void fr_tc_cleanup(struct flow_block_cb *block_cb)
{
struct fake_block *fb = block_cb->indr.data;
struct flow_block_offload bo;
fr_bo_init(&bo, fb, FLOW_BLOCK_UNBIND);
/* NOTE: no reference to fb, no RCU read section - exactly like the
* in-tree callback: if fb was freed during the unregister window,
* every access below is use-after-free. */
down_write(&fb->cb_lock);
list_del(&block_cb->driver_list);
list_move(&block_cb->list, &bo.cb_list);
fr_unbind_list(&bo);
up_write(&fb->cb_lock);
/* caseb choreography: while notify() is between container #1 and
* container #2, perform container #2's teardown exactly like a
* concurrent qdisc destroy would: driver absent -> unbind is a
* no-op leaving the cb attached, then the container is freed
* (models tcf_block_destroy() after the RCU grace period). */
if (fr_caseb_phase && fb == fr_caseb_fb1 && fr_caseb_fb2) {
struct fake_block *fb2 = fr_caseb_fb2;
struct flow_block_offload bo2;
int err;
fr_bo_init(&bo2, fb2, FLOW_BLOCK_UNBIND);
err = flow_indr_dev_setup_offload(fr_ndev, NULL,
TC_SETUP_BLOCK, fb2, &bo2,
fr_tc_cleanup);
if (err == -EOPNOTSUPP && list_empty(&bo2.cb_list))
atomic_set(&fr_caseb_early_ret, 1);
kfree(fb2); /* container destroyed, cb attached */
fr_caseb_fb2 = NULL;
pr_info(FRTAG " caseb: fb2 teardown+free done mid-notify (early_ret=%d)\n",
atomic_read(&fr_caseb_early_ret));
}
}
/* replica of tcf_block_offload_bind() (cls_api.c:870-908 incl. cmd+splice) */
static int fr_container_bind(struct fake_block *fb)
{
struct flow_block_offload bo;
int err;
fr_bo_init(&bo, fb, FLOW_BLOCK_BIND);
down_write(&fb->cb_lock);
err = flow_indr_dev_setup_offload(fr_ndev, NULL, TC_SETUP_BLOCK, fb,
&bo, fr_tc_cleanup);
/* in-tree indr path ignores err here and splices unconditionally */
list_splice(&bo.cb_list, &fb->fb.cb_list);
up_write(&fb->cb_lock);
return err;
}
/* replica of tcf_block_offload_unbind() (cls_api.c:910-926) */
static void fr_container_unbind(struct fake_block *fb)
{
struct flow_block_offload bo;
fr_bo_init(&bo, fb, FLOW_BLOCK_UNBIND);
down_write(&fb->cb_lock);
flow_indr_dev_setup_offload(fr_ndev, NULL, TC_SETUP_BLOCK, fb, &bo,
fr_tc_cleanup);
fr_unbind_list(&bo);
up_write(&fb->cb_lock);
}
/* replica of tc_setup_cb_call()/__tc_setup_cb_call() (cls_api.c:3638-3682) */
static int fr_reader_pass(struct fake_block *fb)
{
struct flow_block_cb *block_cb;
int cnt = 0;
down_read(&fb->cb_lock);
list_for_each_entry(block_cb, &fb->fb.cb_list, list) {
block_cb->cb(TC_SETUP_CLSFLOWER, NULL, block_cb->cb_priv);
cnt++;
}
up_read(&fb->cb_lock);
return cnt;
}
static struct fake_block *fr_block_alloc(void)
{
struct fake_block *fb = kzalloc(sizeof(*fb), GFP_KERNEL);
if (!fb)
return NULL;
INIT_LIST_HEAD(&fb->fb.cb_list);
init_rwsem(&fb->cb_lock);
return fb;
}
/* ------------------------------------------------------------------ */
/* action: sanity */
static int fr_do_sanity(void)
{
struct fake_block *fb = fr_block_alloc();
int cnt, err;
if (!fb)
return -ENOMEM;
err = flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
pr_info(FRTAG " sanity: register D1 err=%d\n", err);
err = fr_container_bind(fb);
cnt = fr_reader_pass(fb);
pr_info(FRTAG " sanity: bind err=%d reader entries=%d\n", err, cnt);
fr_container_unbind(fb);
cnt = fr_reader_pass(fb);
pr_info(FRTAG " sanity: after unbind entries=%d (expect 0)\n", cnt);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " sanity: unregister D1 done, cb_calls=%d\n",
atomic_read(&fr_cb_calls));
kfree(fb);
return 0;
}
/* ------------------------------------------------------------------ */
/* action: casea_ctl - deterministic container-lock bypass proof */
static atomic_t holder_in, reg_done, holder_found;
static int fr_holder_fn(void *data)
{
struct fake_block *fb = data;
struct flow_block_cb *block_cb;
int found = 0;
down_write(&fb->cb_lock); /* exclusive: container locked */
atomic_set(&holder_in, 1);
while (!atomic_read(®_done))
msleep(5);
/* we still hold the exclusive lock: if D2's block_cb is on the list
* now, the register-path splice bypassed cb_lock by construction */
list_for_each_entry(block_cb, &fb->fb.cb_list, list)
if (block_cb->cb_ident == &fr_d2)
found = 1;
atomic_set(&holder_found, found);
up_write(&fb->cb_lock);
return 0;
}
static int fr_do_casea_ctl(void)
{
struct fake_block *fb = fr_block_alloc();
struct task_struct *holder;
int err;
if (!fb)
return -ENOMEM;
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb);
holder = kthread_run(fr_holder_fn, fb, "fr_holder");
while (!atomic_read(&holder_in))
msleep(5);
/* D2 registers while the container lock is held exclusively by the
* holder thread: existing_qdiscs_register() -> list_splice() runs
* with no cb_lock (ice/mlx5e probe context in the real system). */
err = flow_indr_dev_register(fr_indr_bind_cb, &fr_d2);
atomic_set(®_done, 1);
kthread_stop(holder);
pr_info(FRTAG " CASEA-CTL: register_D2 err=%d splice_landed_while_exclusive_lock_held=%d d2_cb_on_list=%d\n",
err, atomic_read(&holder_found), atomic_read(&holder_found));
/* clean teardown: container still alive, both drivers' cbs removed
* through their proper paths; zero KASAN expected for this plan */
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d2, fr_cb_release);
fr_container_unbind(fb);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " CASEA-CTL: teardown done, list_empty=%d\n",
list_empty(&fb->fb.cb_list));
kfree(fb);
return 0;
}
/* ------------------------------------------------------------------ */
/* action: casea_nat - natural corruption hammer */
static atomic_t fr_stop;
static atomic_t fr_passes, fr_entries, fr_cycles;
static atomic_t fr_readers_alive;
static int fr_reader_fn(void *data)
{
struct fake_block *fb = data;
atomic_inc(&fr_readers_alive);
while (!atomic_read(&fr_stop)) {
int n = fr_reader_pass(fb);
atomic_inc(&fr_passes);
atomic_add(n, &fr_entries);
cond_resched();
}
atomic_dec(&fr_readers_alive);
return 0;
}
static int fr_do_casea_nat(void)
{
struct fake_block *fb = fr_block_alloc();
struct task_struct **rd;
int i, err;
if (!fb)
return -ENOMEM;
atomic_set(&fr_stop, 0);
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb);
rd = kcalloc(nreaders, sizeof(*rd), GFP_KERNEL);
for (i = 0; i < nreaders; i++)
rd[i] = kthread_run(fr_reader_fn, fb, "fr_rd%d", i);
/* churn: each D2 register performs one lockless splice into the live
* cb_list (existing_qdiscs_register), racing the locked readers */
for (i = 0; i < iters; i++) {
err = flow_indr_dev_register(fr_indr_bind_cb, &fr_d2);
if (!err)
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d2,
fr_cb_release);
atomic_inc(&fr_cycles);
if (!(i % 256))
cond_resched();
}
atomic_set(&fr_stop, 1);
for (i = 0; i < nreaders; i++)
if (rd[i])
kthread_stop(rd[i]);
kfree(rd);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " CASEA-NAT: cycles=%d reader_passes=%d entries=%d cb_calls=%d readers_alive=%d\n",
atomic_read(&fr_cycles), atomic_read(&fr_passes),
atomic_read(&fr_entries), atomic_read(&fr_cb_calls),
atomic_read(&fr_readers_alive));
kfree(fb);
return 0;
}
/* concurrent-driver variant: two drivers register at the same time (as
* with ice+sfc hotplug probing in parallel) - two lockless splices race
* on the same list head; an interleave orphans or double-links a chain
* and LIST_HARDENED fires on the next list_del. */
static int fr_churn_fn(void *priv)
{
int i;
for (i = 0; i < iters; i++) {
if (!flow_indr_dev_register(fr_indr_bind_cb, priv))
flow_indr_dev_unregister(fr_indr_bind_cb, priv,
fr_cb_release);
atomic_inc(&fr_cycles);
cond_resched();
}
complete(&fr_cc_done);
return 0;
}
static int fr_do_casea_nat2(void)
{
struct fake_block *fb = fr_block_alloc();
struct task_struct *c2, *c3, *r0, *r1;
if (!fb)
return -ENOMEM;
atomic_set(&fr_stop, 0);
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb);
init_completion(&fr_cc_done);
r0 = kthread_run(fr_reader_fn, fb, "fr_rd0");
r1 = kthread_run(fr_reader_fn, fb, "fr_rd1");
c2 = kthread_run(fr_churn_fn, &fr_d2, "fr_ch2");
c3 = kthread_run(fr_churn_fn, &fr_d3, "fr_ch3");
wait_for_completion(&fr_cc_done);
wait_for_completion(&fr_cc_done);
atomic_set(&fr_stop, 1);
kthread_stop(r0);
kthread_stop(r1);
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
pr_info(FRTAG " CASEA-NAT2: cycles=%d reader_passes=%d entries=%d cb_calls=%d\n",
atomic_read(&fr_cycles), atomic_read(&fr_passes),
atomic_read(&fr_entries), atomic_read(&fr_cb_calls));
kfree(fb);
return 0;
}
/* ------------------------------------------------------------------ */
/* action: caseb - deterministic unregister-window UAF */
static int fr_do_caseb(void)
{
struct fake_block *fb1, *fb2;
fb1 = fr_block_alloc();
fb2 = fr_block_alloc();
if (!fb1 || !fb2)
return -ENOMEM;
flow_indr_dev_register(fr_indr_bind_cb, &fr_d1);
fr_container_bind(fb1);
fr_container_bind(fb2);
pr_info(FRTAG " caseb: two containers bound\n");
/* unregister: mutex section removes D1 + collects both cbs; the
* notify phase then calls fr_tc_cleanup(cb_fb1) first - which we
* use as the controlled point to run container #2's teardown+free
* (equivalent to a concurrent qdisc destroy inside the window),
* after which notify calls fr_tc_cleanup(cb_fb2) on freed fb2. */
fr_caseb_fb1 = fb1;
fr_caseb_fb2 = fb2;
fr_caseb_phase = 1;
flow_indr_dev_unregister(fr_indr_bind_cb, &fr_d1, fr_cb_release);
fr_caseb_phase = 0;
pr_info(FRTAG " CASEB: unregister done, early_ret=%d fb1_list_empty=%d (fb2 kfree'd mid-notify; UAF expected in dmesg)\n",
atomic_read(&fr_caseb_early_ret), list_empty(&fb1->fb.cb_list));
kfree(fb1);
return 0;
}
/* ------------------------------------------------------------------ */
/* Case C: percpu scratch race replica */
static struct dst_entry *fr_route_dst_new(void)
{
struct dst_entry *d = dst_alloc(&fr_dst_ops, fr_ndev,
DST_OBSOLETE_NONE, DST_NOCOUNT);
if (d)
atomic_inc(&fr_dst_created);
return d;
}
/* replica of bpf_skb_set_xfrm_info() (xfrm_interface_bpf.c:64-94) */
static int fr_set_xfrm_info(struct sk_buff *skb, u32 if_id, int link)
{
struct metadata_dst *md_dst;
struct xfrm_md_info *info;
if (unlikely(skb_metadata_dst(skb)))
return -EINVAL;
md_dst = this_cpu_ptr(fr_scratch);
info = &md_dst->u.xfrm_info;
info->if_id = if_id;
info->link = link;
skb_dst_force(skb);
info->dst_orig = skb_dst(skb); /* unconditional overwrite */
dst_hold((struct dst_entry *)md_dst);
skb_dst_set(skb, (struct dst_entry *)md_dst);
return 0;
}
/* replica of xfrmi_xmit2() collect_md block (xfrm_interface_core.c:434-455) */
static void fr_xmit2_swap(struct sk_buff *skb)
{
struct xfrm_md_info *md_info = skb_xfrm_md_info(skb);
struct dst_entry *dst;
if (!md_info)
return;
if (md_info->dst_orig) { /* read-take-clear, non-atomic */
struct dst_entry *tmp_dst = skb_dst(skb);
dst = md_info->dst_orig;
skb_dst_set(skb, dst);
md_info->dst_orig = NULL;
dst_release(tmp_dst); /* drop this skb's scratch ref */
}
}
static int fr_casec_setup(void)
{
fr_dst_cache = kmem_cache_create("frdst", sizeof(struct dst_entry),
0, 0, NULL);
if (!fr_dst_cache)
return -ENOMEM;
fr_dst_ops.kmem_cachep = fr_dst_cache;
fr_dst_ops.destroy = fr_dst_destroy;
fr_scratch = metadata_dst_alloc_percpu(0, METADATA_XFRM, GFP_KERNEL);
if (!fr_scratch)
return -ENOMEM;
return 0;
}
static void fr_casec_teardown(void)
{
if (fr_scratch) {
metadata_dst_free_percpu(fr_scratch);
fr_scratch = NULL;
}
if (fr_dst_cache) {
kmem_cache_destroy(fr_dst_cache);
fr_dst_cache = NULL;
}
}
static int fr_do_casec_ctl(void)
{
struct dst_entry *r1, *r2, *rx;
struct sk_buff *s1, *s2, *s3;
int before, leaks;
if (fr_casec_setup())
return -ENOMEM;
/* --- baseline: single-packet protocol closes cleanly --- */
r1 = fr_route_dst_new();
s1 = alloc_skb(64, GFP_KERNEL);
skb_dst_set(s1, r1); /* skb owns r1's single ref */
preempt_disable(); /* keep this_cpu_ptr stable */
fr_set_xfrm_info(s1, 1, 0);
fr_xmit2_swap(s1);
preempt_enable();
kfree_skb(s1); /* drops r1 -> freed via RCU */
synchronize_rcu();
pr_info(FRTAG " casec baseline: created=%d destroyed=%d (expect 1/1)\n",
atomic_read(&fr_dst_created), atomic_read(&fr_dst_destroyed));
/* --- leak variant: unconditional overwrite of pending dst_orig --- */
before = atomic_read(&fr_dst_destroyed);
r1 = fr_route_dst_new();
r2 = fr_route_dst_new();
s1 = alloc_skb(64, GFP_KERNEL);
s2 = alloc_skb(64, GFP_KERNEL);
skb_dst_set(s1, r1);
skb_dst_set(s2, r2);
preempt_disable();
fr_set_xfrm_info(s1, 1, 0); /* scratch.dst_orig = r1 */
fr_set_xfrm_info(s2, 2, 0); /* OVERWRITES: r1 leaked */
fr_xmit2_swap(s1); /* gets r2 (cross-packet) */
fr_xmit2_swap(s2); /* dst_orig==NULL -> skip */
preempt_enable();
kfree_skb(s1);
kfree_skb(s2);
synchronize_rcu();
leaks = atomic_read(&fr_dst_created) - atomic_read(&fr_dst_destroyed);
pr_info(FRTAG " CASEC-LEAK: leaked=%d (r1 refs_held=%d, expected 0 if not leaked; baseline destroyed+%d of 2)\n",
leaks, rcuref_read(&r1->__rcuref),
atomic_read(&fr_dst_destroyed) - before);
dst_release(r1); /* reclaim the leaked ref */
/* --- double-consume: interleaved read-take-clear (the exact
* instruction order two concurrent xfrmi_xmit2 executions produce;
* natural reachability is shown by casec_nat) --- */
rx = fr_route_dst_new(); /* refcnt = 1 */
s1 = alloc_skb(64, GFP_KERNEL);
s2 = alloc_skb(64, GFP_KERNEL);
s3 = alloc_skb(64, GFP_KERNEL);
skb_dst_set(s3, rx); /* the "original skb" hands rx */
preempt_disable();
fr_set_xfrm_info(s3, 3, 0); /* scratch.dst_orig = rx */
{
struct xfrm_md_info *mi = this_cpu_ptr(&fr_scratch->u.xfrm_info);
struct dst_entry *t1, *t2;
t1 = mi->dst_orig; /* skb1 xmit2: read */
t2 = mi->dst_orig; /* skb2 xmit2: read */
skb_dst_set(s1, t1); /* skb1 takes */
skb_dst_set(s2, t2); /* skb2 takes - 1 ref, 2 owners */
mi->dst_orig = NULL;
}
preempt_enable();
kfree_skb(s3); /* drops scratch md ref */
pr_info(FRTAG " casec dcons: rx refs_held=%d (2 claims on 1 real ref)\n",
rcuref_read(&rx->__rcuref));
kfree_skb(s1); /* first owner releases */
pr_info(FRTAG " casec dcons: after skb1 free rx dead=%d\n",
rcuref_is_dead(&rx->__rcuref));
kfree_skb(s2); /* second owner: imbalanced put */
synchronize_rcu(); /* dst_destroy_rcu frees rx */
pr_info(FRTAG " CASEC-DCONS: touching rx after grace period (UAF read expected below)\n");
pr_info(FRTAG " CASEC-DCONS: rx->obsolete=%d created=%d destroyed=%d\n",
rx->obsolete, atomic_read(&fr_dst_created),
atomic_read(&fr_dst_destroyed));
fr_casec_teardown();
return 0;
}
/* natural churn: two threads on the SAME cpu (shared percpu scratch) */
static atomic_t fr_cc_cycles, fr_cc_confused, fr_cc_skipped;
static int fr_cc_fn(void *unused)
{
int i;
for (i = 0; i < iters; i++) {
struct dst_entry *r = fr_route_dst_new();
struct sk_buff *skb = alloc_skb(64, GFP_KERNEL);
int us;
if (r && skb) {
skb_dst_set(skb, r);
fr_set_xfrm_info(skb, i & 1, 0);
us = get_random_u32_below(jitter_us + 1);
if (us)
udelay(us); /* widen set->xmit window */
if (skb_dst(skb) == (struct dst_entry *)
this_cpu_ptr(fr_scratch)) {
struct dst_entry *want = r;
fr_xmit2_swap(skb);
if (skb_dst(skb) != want) {
if (skb_dst(skb) == (struct dst_entry *)
this_cpu_ptr(fr_scratch))
atomic_inc(&fr_cc_skipped);
else
atomic_inc(&fr_cc_confused);
}
}
kfree_skb(skb);
} else {
if (skb)
kfree_skb(skb);
if (r)
dst_release(r);
}
atomic_inc(&fr_cc_cycles);
cond_resched();
}
complete(&fr_cc_done);
return 0;
}
static void fr_scratch_drain(void)
{
int cpu;
/* replicate metadata_dst_free_percpu()'s dst_orig release loop so
* accounting is settled before the final destroy-count read */
for_each_possible_cpu(cpu) {
struct xfrm_md_info *mi =
per_cpu_ptr(&fr_scratch->u.xfrm_info, cpu);
if (mi->dst_orig) {
dst_release(mi->dst_orig);
mi->dst_orig = NULL;
}
}
}
static int fr_do_casec_nat(void)
{
struct task_struct *t1, *t2;
int leaked;
if (fr_casec_setup())
return -ENOMEM;
init_completion(&fr_cc_done);
t1 = kthread_create(fr_cc_fn, NULL, "fr_cc0");
t2 = kthread_create(fr_cc_fn, NULL, "fr_cc1");
if (IS_ERR(t1) || IS_ERR(t2))
return -ENOMEM;
kthread_bind(t1, 1); /* same CPU -> same scratch */
kthread_bind(t2, 1);
wake_up_process(t1);
wake_up_process(t2);
/* let both run their full bounded loops, then settle accounting */
wait_for_completion(&fr_cc_done);
wait_for_completion(&fr_cc_done);
fr_scratch_drain();
synchronize_rcu();
leaked = atomic_read(&fr_dst_created) - atomic_read(&fr_dst_destroyed);
pr_info(FRTAG " CASEC-NAT: cycles=%d created=%d destroyed=%d leaked=%d confused(other R)=%d swap_skipped=%d\n",
atomic_read(&fr_cc_cycles), atomic_read(&fr_dst_created),
atomic_read(&fr_dst_destroyed), leaked,
atomic_read(&fr_cc_confused), atomic_read(&fr_cc_skipped));
fr_casec_teardown();
return 0;
}
/* ------------------------------------------------------------------ */
static int __init flowrace_init(void)
{
int ret = 0;
fr_ndev = dev_get_by_name(&init_net, "lo");
if (!fr_ndev) {
pr_err(FRTAG " no lo netdev\n");
return -ENODEV;
}
pr_info(FRTAG " action=%s iters=%d readers=%d jitter_us=%d\n",
action, iters, nreaders, jitter_us);
if (!strcmp(action, "sanity"))
ret = fr_do_sanity();
else if (!strcmp(action, "casea_ctl"))
ret = fr_do_casea_ctl();
else if (!strcmp(action, "casea_nat"))
ret = fr_do_casea_nat();
else if (!strcmp(action, "casea_nat2"))
ret = fr_do_casea_nat2();
else if (!strcmp(action, "caseb"))
ret = fr_do_caseb();
else if (!strcmp(action, "casec_ctl"))
ret = fr_do_casec_ctl();
else if (!strcmp(action, "casec_nat"))
ret = fr_do_casec_nat();
else
ret = -EINVAL;
dev_put(fr_ndev);
pr_info(FRTAG "-DONE action=%s ret=%d cb_calls=%d\n", action, ret,
atomic_read(&fr_cb_calls));
return ret;
}
static void __exit flowrace_exit(void)
{
}
module_init(flowrace_init);
module_exit(flowrace_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("net/core flow_indr / xfrm scratch race verification");