Re: [PATCH] io_uring/rsrc: send exact nr_segs for fixed buffer

[Pavel Begunkov]

On 4/16/25 06:44, Nitesh Shetty wrote:
> Sending exact nr_segs, avoids bio split check and processing in
> block layer, which takes around 5%[1] of overall CPU utilization.
>
> In our setup, we see overall improvement of IOPS from 7.15M to 7.65M [2]
> and 5% less CPU utilization.
>
> [1]
>       3.52%  io_uring         [kernel.kallsyms]     [k] bio_split_rw_at
>       1.42%  io_uring         [kernel.kallsyms]     [k] bio_split_rw
>       0.62%  io_uring         [kernel.kallsyms]     [k] bio_submit_split
>
> [2]
> sudo taskset -c 0,1 ./t/io_uring -b512 -d128 -c32 -s32 -p1 -F1 -B1 -n2
> -r4 /dev/nvme0n1 /dev/nvme1n1
>
> Signed-off-by: Nitesh Shetty <nj.shetty@xxxxxxxxxxx>
> ---
>   io_uring/rsrc.c | 3 +++
>   1 file changed, 3 insertions(+)
>
> diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
> index b36c8825550e..6fd3a4a85a9c 100644
> --- a/io_uring/rsrc.c
> +++ b/io_uring/rsrc.c
> @@ -1096,6 +1096,9 @@ static int io_import_fixed(int ddir, struct iov_iter *iter,
>   			iter->iov_offset = offset & ((1UL << imu->folio_shift) - 1);
>   		}
>   	}
> +	iter->nr_segs = (iter->bvec->bv_offset + iter->iov_offset +
> +		iter->count + ((1UL << imu->folio_shift) - 1)) /
> +		(1UL << imu->folio_shift);

That's not going to work with ->is_kbuf as the segments are not uniform in
size.

And can we make it saner? Split it into several statements, add variables
for folio size and so, or maybe just use ALIGN. If moved above, you
probably don't even need to recalc

iter->bvec->bv_offset + iter->iov_offset

--
Pavel Begunkov