Re: [PATCH net-next 0/2] net/mlx5e: Batch RCU grace periods on channel open and close

[Toke Høiland-Jørgensen]

Tariq Toukan <tariqt@xxxxxxxxxx> writes:

> Hi,
>
> This series by Carolina reduces the time mlx5e spends waiting for RCU
> grace periods when bringing an interface up or down with many channels.
>
> Today the channel open and close flows call synchronize_net() per
> object. On close, RQ, ICOSQ, TX SQ and NAPI teardown each require a
> grace period, so the number of waits scales with the number of channels.
> TX SQ teardown adds a wait per traffic class, and XDP and QoS can add
> more. On activation, mlx5e_activate_channels() triggers the ICOSQ of
> every channel, and each trigger waits for its own grace period. With a
> large channel count this makes link down and link up slow.
>
> Patch 1 batches the waits on channel close. Channel deactivate and close
> are split into pre_sync and post_sync phases: the ENABLED state is first
> cleared on all queues of all channels, a single grace period is awaited,
> and only then does the teardown continue. NAPI instances are likewise
> unlinked with __netif_napi_del_locked(), followed by one
> synchronize_net() before the channels are freed. The existing per object
> synchronize_net() calls are kept for single queue paths. The change
> touches the regular channels as well as the PTP, QoS and XSK queues.
>
> Patch 2 does the same for the ICOSQ trigger on activation: all channel
> ICOSQs are first marked as requiring locking, one grace period is
> awaited, and then all of them are triggered. mlx5e_trigger_napi_icosq()
> keeps its existing semantics for standalone callers.
>
> Results with 248 channels and rcu_normal=1:
> - ip link set eth2 down: 10.183s before, 0.715s after.
> - ip link set eth2 up: 3.095s before, 1.794s after.

Nice! Thanks for tackling this :)

Don't know the driver well enough to comment on the details, but in
principle, emphatically:

Acked-by: Toke Høiland-Jørgensen <toke@xxxxxxxxxx>