Re: [PATCH bpf-next v2 1/6] kallsyms: Add KALLSYMS_QUERY ioctl to /proc/kallsyms

[Andrii Nakryiko]

On Thu, Oct 29, 2026 at 3:36 PM Samuel Wu <wusamuel@xxxxxxxxxx> wrote:
>
> On Thu, Oct 29, 2026 at 3:09 PM Andrii Nakryiko
> <andrii.nakryiko@xxxxxxxxx> wrote:
> >
> > On Thu, Oct 29, 2026 at 2:58 PM Samuel Wu <wusamuel@xxxxxxxxxx> wrote:
> > >
> > > On Tue, Oct 27, 2026 at 3:25 PM Andrii Nakryiko
> > > <andrii.nakryiko@xxxxxxxxx> wrote:
> > > >
> > > > On Thu, Oct 22, 2026 at 5:17 PM Samuel Wu <wusamuel@xxxxxxxxxx> wrote:
> > > > >
> > > > > Add an extensible KALLSYMS_QUERY ioctl() interface to /proc/kallsyms
> > > > > to support fast point queries without scanning the entire text file.
> > > > > Exactly one query mode must be set in query_flags.
> > > > >
> > > > > When KALLSYMS_QUERY_BY_SYMBOL is set in query_flags, KALLSYMS_QUERY
> > > > > resolves a NUL-terminated symbol name in vmlinux or loaded modules and
> > > > > then populates sym_addr:
> > > > > - Searches vmlinux first via kallsyms_lookup_names() (binary search),
> > > > >   which takes precedence over loaded modules, and falls back to
> > > > >   module_kallsyms_on_each_symbol() under module_mutex for module
> > > > >   symbols
> > > > > - Matches names exactly, like kallsyms_lookup_name(), disregarding the
> > > > >   .llvm.<hash> matching, following precedent set by commit fb6a421fb615
> > > > >   ("kallsyms: Match symbols exactly with CONFIG_LTO_CLANG")
> > > > > - Resolves 0x0 symbol addresses in both vmlinux and modules
> > > > > - Rejects ambiguous symbols (multiple entries with the same name at
> > > > >   different addresses within vmlinux, or across loaded modules) with
> > > > >   -EADDRNOTAVAIL, while allowing same-address aliases
> > > > >
> > > > > Userspace symbolizers resolve kernel addresses to symbol names with
> > > > > /proc/kallsyms. This is typically done by reading the entire
> > > > > /proc/kallsyms text file, and keeping a compressed sorted copy of it in
> > > > > order to resolve addresses at runtime.
> > > > >
> > > > > When KALLSYMS_QUERY_BY_ADDR is set in query_flags, KALLSYMS_QUERY
> > > > > resolves one address to the symbol that contains it:
> > > > > - sym_addr holds the address to resolve on input.
> > > > > - sym_name_addr must point to a user buffer of sym_name_size bytes
> > > > >   that receives the NUL-terminated symbol name. On output,
> > > > >   sym_name_size holds the number of bytes written (including the
> > > > >   terminating NUL). -ENAMETOOLONG is returned if the buffer is too
> > > > >   small; a buffer of KSYM_NAME_LEN (512 at the time of writing) bytes
> > > > >   is always large enough.
> > > > > - Symbols are resolved via kallsyms_lookup(), the same lookup as %pS,
> > > > >   covering vmlinux, loaded modules, BPF JIT images, and freed module
> > > > >   init functions still tracked by ftrace.
> > > > >
> > > > > On an 8 CPU Cuttlefish (x86-64 VM), using this patchseries to load a BPF
> > > > > program with 11 typeless ksyms has ~220x speedup (157ms -> 0.7ms).
> > > > >
> > > > > On an 8 CPU x86-64 VM with a 175k line kallsyms, there are broad
> > > > > performance and memory savings for the symbolization and callstack
> > > > > sampling tool, Perfetto:
> > > > > - Init time: ~350x speedup (155.7ms -> 0.44ms)
> > > > > - 50k symbolizations: ~12x speedup (25.2ms -> 2.2ms)
> > > > > - VmRSS: -68% (15.1MB -> 4.8MB)
> > > > >
> > > > > Suggested-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
> > > > > Signed-off-by: Samuel Wu <wusamuel@xxxxxxxxxx>
> > > > > ---
> > > > >  include/uapi/linux/fs.h |  41 +++++++++++++
> > > > >  kernel/kallsyms.c       | 132 ++++++++++++++++++++++++++++++++++++++++
> > > > >  2 files changed, 173 insertions(+)
> > > > >
> > > > > diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
> > > > > index 34c6f219462a..b43f227dad0e 100644
> > > > > --- a/include/uapi/linux/fs.h
> > > > > +++ b/include/uapi/linux/fs.h
> > > > > @@ -674,6 +674,47 @@ struct procmap_query {
> > > > >         __u64 build_id_addr;            /* in */
> > > > >  };
> > > > >
> > > > > +/* /proc/kallsyms ioctl */
> > > > > +#define KALLSYMS_QUERY _IOWR(PROCFS_IOCTL_MAGIC, 18, struct kallsyms_query)
> > > > > +
> > > > > +/* Exactly one query mode must be set in query_flags. */
> > > > > +enum kallsyms_query_flags {
> > > > > +       /*
> > > > > +        * Look up a symbol in vmlinux or loaded modules (vmlinux first; BPF
> > > > > +        * and ftrace symbols are not searched) by exact NUL-terminated name
> > > > > +        * at sym_name_addr and return sym_addr. sym_addr and sym_name_size
> > > > > +        * must be 0 on input.
> > > > > +        */
> > > > > +       KALLSYMS_QUERY_BY_SYMBOL = 0x01,
> > > > > +       /*
> > > > > +        * Look up the symbol covering sym_addr and write its NUL-terminated
> > > > > +        * name to the sym_name_addr buffer of sym_name_size bytes. The same
> > > > > +        * symbols as %pS are searched: vmlinux, loaded modules, BPF JIT
> > > > > +        * images, and freed module init functions still tracked by ftrace.
> > > > > +        * A buffer of KSYM_NAME_LEN (512 at the time of writing) bytes is
> > > > > +        * always enough. On success, sym_name_size is set to the number of
> > > > > +        * bytes written (including the terminating NUL).
> > > > > +        */
> > > > > +       KALLSYMS_QUERY_BY_ADDR = 0x02,
> > > > > +};
> > > > > +
> > > > > +/*
> > > > > + * Returns -ENOENT if no symbol matches, -EADDRNOTAVAIL if the name maps to
> > > > > + * more than one address, -ENAMETOOLONG if the name does not fit in
> > > > > + * KSYM_NAME_LEN (512 at the time of writing) or sym_name_size bytes, and
> > > > > + * -EPERM if kallsyms addresses are hidden from the opener of /proc/kallsyms
> > > > > + * (e.g. kptr_restrict).
> > > > > + */
> > > > > +struct kallsyms_query {
> > > > > +       __u64 size;             /* in - struct size for compatibility */
> > > > > +       __u64 query_flags;      /* in - query mode (enum kallsyms_query_flags) */
> > > > > +       __u64 sym_name_addr;    /* in - address of symbol name buffer */
> > > > > +       __u64 sym_addr;         /* out (BY_SYMBOL) / in (BY_ADDR) */
> > > > > +       __u64 sym_name_size;    /* in/out - name buffer size / bytes written */
> > > > > +};
> > > >
> > > > it might be smart to have QUERY_BY_ADDR and QUERY_BY_NAME to be two
> > > > separate ioctl commands with its own query in/out struct. This would
> > > > remove addr vs name ambiguity in sym_name_addr field, and will allow
> > > > us to evolve APIs independently, if necessary. They should still
> > > > probably reuse the same type to represent symbol information, and have
> > > > kallsyms_query struct provide a pointer to that struct (and its
> > > > maximum size), which the kernel will fill out and set the size it
> > > > actually knows about.
> > > >
> > > > struct kallsyms_query_by_addr {
> > > >     __u64 size;
> > > >     __u64 query_flags;
> > > >     __u64 sym_addr; /* in */
> > > >     struct kallsyms_sym *sym; /* out */
> > > >     __u64 sym_len;
> > > > };
> > > >
> > > > struct kallsyms_sym {
> > > >     __u64 sym_addr;
> > > >     __u64 sym_name;
> > > >     __u64 sym_name_len;
> > > >     __u64 sym_module;
> > > >     __u64 sym_module_len; /* if zero, no module requested/available */
> > > > };
> > > >
> > > > e.g. for symbol you'd want to return its name, address, but also
> > > > module name. and we should leave a way to extend it further with more
> > > > optional information (that's the kallsyms extensibility bit I was
> > > > talking about).
> > >
> > > Agree, two ioctls with a shared symbol type is cleaner. v3 plan is to
> > > have QUERY_BY_NAME and QUERY_BY_ADDR both fill a shared struct
> > > kallsyms_sym (address, size, name, module), passed by pointer so it
> > > can grow later.
> > >
> > > > aslo, we should think about adding ways to paginate (or at least leave
> > > > a possibility of that in the future) if there are multiple matches
> > > > (e.g., for static functions with the same name resolving to multiple
> > > > addresses); not sure how it should look like API wise, but it would be
> > > > good to still return "any" or "first" match, but let user know there
> > > > are multiple, so user can decide how to proceed...
> > >
> > > Makes sense, but I'm thinking v3 doesn't need pagination or batching.
> > > I didn't see any performance wins when profiling libbpf, and it can be
> > > added later without breaking UAPI.
> > >
> > > > on input I'd also add module name as optional input argument, with
> > > > "vmlinux" limiting to just vmlinux, set module name to just that name,
> > > > and NULL/zero meaning "we don't care"
> > >
> > > Makes sense, so here is a possible plan for QUERY_BY_NAME:
> > > - Optional module filter: NULL = any, "vmlinux", or a module name
> > > - Returns the first match by default (vmlinux first, then modules). An
> > > ALL_MATCHES flag searches everything and reports the number of matches
> >
> > searches everything as in all modules, or you mean just full linear scan?
>
> I envision ALL_MATCHES to do some form of lower_bound() +
> upper_bound() on vmlinux since it's sorted, and then linear scan of
> all modules. Of course, just one of the two or both aforementioned
> things depending on the module filter (any, vmlinux, module).

no point in upper_bound() if it's sorted, you just start at
lower_bound and go linearly from there. I don't think it's realistic
to assume large number of duplicates, so I wouldn't optize for that,
one log(N) lower bound + linear scan will be faster in most cases,
probably

>
> > >
> > > And an open question I'm unsure about: Should libbpf keep the light
> > > skeleton semantics (take the first match, no ambiguity check), or set
> > > ALL_MATCHES to keep today's check that fails on names with multiple
> > > addresses?
> >
> > I'd say fail on ambiguity
>
> Ack
>
> > >
> > > > for QUERY_BY_ADDR we should have a flag saying whether an exact match
> > > > by address is requested or we should find a symbol that covers the
> > > > address (that's most useful for symbolization). maybe even "at or
> > > > after" address mode, so you can paginate (though probably explicitly
> > > > designed batch API would be even more useful later)
> > >
> > > For v3 I think we can keep BY_ADDR to the covering lookup only. The
> > > result can fill in the covering symbol's (start address, size, name,
> > > module), and callers can check for an exact match themselves by
> > > comparing the start address with the queried address. A batch API can
> > > be added later when there is a user for it.
> >
> > There is a user for batch API, that's blazesym, wprof, retsnoop and
> > other applications that use kallsyms for symbolization. I didn't
> > insist on adding it from the get go just to reduce the scope, but
> > there is definitely a use case for having fast way to dump all or
> > parts of kallsyms
>
> Ahh, ok, hopefully reducing the scope for v3 is fine.
>
> > >
> > > > tl;dr, this is the right direction, but I think we should polish API a
> > > > bit more to be more universally useful and extensible
> > > > > +
> > > > > +#define KALLSYMS_QUERY_SIZE_VER0       40 /* sizeof first published struct */
> > > > > +
> >
> > [...]
> >
> > > >
> > > > PROCMAP_QUERY uses offsetofend() for the last defined field, any
> > > > reason to have this #define and BUILD_BUG_ON here?
> > >
> > > David pointed out that the size field in struct kallsyms_query is
> > > redundant. In v3, I can drop the size field and use _IOC_SIZE(cmd),
> > > keep the *_SIZE_VER0 define (since we need minimum size). Perfectly
> > > fine to switch to the PROCMAP_QUERY style though if preferred.
> >
> > I'm not sure how size is redundant, I don't see ioctl subsystem
> > accepting argument size as a separate parameter, so the size has to
> > come from the user and be embedded in user-provided structure, if we
> > want any sort of backward/forward compatibility. At least looking at
> > PROCMAP_QUERY I don't see how we could have avoided it, so not sure
> > how we can do that here.
>
> My understanding is that the size is encoded in the ioctl command
> itself: _IOWR() puts sizeof(struct) into the command number when
> userspace is compiled, so _IOC_SIZE(cmd) is the struct size of the
> header the caller was built against.
>

oh, interesting, I didn't realize we embed struct size into ioctl cmd
and match it exactly. I see that pm_scan_arg has the same __u64 size
in the struct approach, I'd stick to that, and when we extend any of
them we'd need to make sure to either ignore or hard-code the size
part, hm..

> That said, either way is fine. We could mirror PROCMAP_QUERY style for
> consistency.
>
> > >
> > > Thanks Andrii for all the thoughtful feedback!
> > >
> > > > > +               return -EINVAL;
> > > > > +
> > > > > +       err = copy_struct_from_user(&karg, sizeof(karg), uarg, usize);
> > > > > +       if (err)
> > > > > +               return err;
> > > > > +
> > > >
> > > > [...]