[PATCH v6 0/5] dmaengine: dw-edma: Support dynamic LL appends
From: Koichiro Den
Date: Fri Oct 30 2026 - 11:09:36 EST
Hi,
This series lets dw-edma append requests to a running circular linked
list (LL) using PCS-CCS-CB-TCB Producer-Consumer Synchronization. This
reduces stop/start cycles between descriptors.
The work started with Frank's RFT:
https://lore.kernel.org/r/20260109-edma_dymatic-v1-0-9a98c9c98536@xxxxxxx/
v6 throws away most of v5. v5 included recovery code for stalls seen on
R-Car S4 (DWC IP 5.20a). v6 drops that recovery code and the defensive
doorbell handling built around it. The S4 stalls are still under
investigation [1]. I have not seen the same fatal stall on RK3588 (DWC
IP 5.60a) and SpacemiT K3 (DWC IP 6.30a, native HDMA), either with
nvmet-pci-epf or with the out-of-tree NTB transport described below.
Note for maintainers: the prep work [2], which has already landed, added
the ll_irq_clear core operation. v6 does not use it. If the v6 approach is
acceptable, I will remove it in a follow-up.
Frank, thanks for your review so far. v6 is close to your RFT and my v1
again, but keeps what the v2-v5 reviews established (e.g.
running-boundary accounting, STOP/PAUSE draining, HDMA watermarks).
Based on dmaengine-7.4-rc1 (aff5e6a77fc6).
v6 Testing
==========
I used three setups for testing:
Id Endpoint Link Host
--- ------------------------ --------- -----------------
(A) RK3588 ROCK 5B Gen3 x4 CIX Sky1 Orion O6
(B) SpacemiT K3 [3] Pico-ITX Gen3 x4 CIX Sky1 Orion O6
(C) R-Car S4 Spider Gen4 x2 R-Car S4 Spider
1. pci-epf-test:
For all (A)~(C), same PASS/FAIL/SKIP set with and without this series.
2. nvmet-pci-epf:
I ran fio on the host against a null_blk target, using the same 26
cases as in v5. Each setup was tested five times without the series
(base) and five times with it (new), alternating as
base1->new1->base2->new2->...->base5->new5. The table shows median
throughput in MiB/s.
(A) RK3588 (B) K3 (C) R-Car S4
(5.60a eDMA) (6.30a HDMA) (5.20a eDMA)
---------------------- --------------- --------------- --------------
rand read 4K q1 34-> 28 -17% 54-> 42 -21% 9.9-> 8.2 -18%
rand read 4K q32 180-> 173 -4% 147-> 150 +2% 46-> 52 +14%
rand read 4K q32/4 183-> 176 -4% 143-> 151 +6% 53-> 57 +7%
rand read 128K q1 610-> 765 +25% 933-> 851 -9% 146-> 139 -5%
rand read 128K q32 80->1139 x14 1121->1480 +32% 591-> 812 +37%
rand read 128K q32/4 47->1246 x27 1752->2235 +28% 1030->1160 +13%
rand read 512K q1 1002->1043 +4% 1266->1180 -7% 135-> 176 +30%
rand read 512K q32 54->1371 x25 1730->2131 +23% 940->1101 +17%
rand read 512K q32/4 47->1122 x24 1808->2367 +31% 1097->1178 +7%
rand write 4K q1 37-> 24 -35% 56-> 35 -38% 8.1-> 6.8 -16%
rand write 4K q32 73-> 153 x2.1 117-> 216 +85% 37-> 42 +12%
rand write 4K q32/4 71-> 154 x2.2 117-> 214 +83% 40-> 53 +32%
rand write 128K q1 480-> 498 +4% 768-> 634 -17% 111-> 97 -13%
rand write 128K q32 900->1312 +46% 1686->1908 +13% 469-> 632 +35%
rand write 128K q32/4 899->1292 +44% 1681->1900 +13% 601-> 776 +29%
seq read 128K q1 617-> 784 +27% 832-> 822 -1% 169-> 128 -24%
seq read 128K q32 71->1163 x16 1131->1499 +33% 600-> 811 +35%
seq read 512K q1 987->1054 +7% 1294->1229 -5% 138-> 178 +29%
seq read 512K q32 56->1535 x27 1726->2140 +24% 930->1080 +16%
seq read 1M q32 47->1232 x26 1792->2324 +30% 1071->1180 +10%
seq write 128K q1 471-> 488 +4% 789-> 698 -12% 117-> 102 -13%
seq write 128K q32 883->1312 +49% 1674->1903 +14% 477-> 646 +35%
seq write 512K q1 567-> 722 +27% 242-> 321 +33% 113-> 156 +38%
seq write 512K q32 881->1305 +48% 1701->1901 +12% 597-> 784 +31%
seq write 1M q32 843->1206 +43% 1685->1902 +13% 592-> 780 +32%
rw read 4K-1M q8/4 175-> 552 x3.2 529-> 712 +35% 363-> 433 +19%
rw write 4K-1M q8/4 179-> 553 x3.1 529-> 714 +35% 364-> 433 +19%
("/4" in "q32/4" means 4 jobs.)
More briefly, geometric mean of the new/base ratios across all runs
in each group (5 runs x N cases):
(A) RK3588 (B) K3 (C) S4
--------------- ---------- ---------- ----------
QD32 (16 cases) x4.59 +32% +58%
QD1 (9 cases) +3% -5% +23%
rw q8 (2 cases) x3.08 +44% +47%
Note for (C):
- S4 showed substantial run-to-run variation, particularly in
base2 and base3. This raises the all-run ratios relative to the
ratios of per-case medians. The cause is unknown.
Note for (A) and (C):
- commit efadda401c5c ("PCI: dwc: Set DMA_BYPASS bit in outbound
IATU CTRL2 register"), which is not in dmaengine-7.4-rc1, was
applied to both the base and the new kernels. Without it:
- (A) often hit NVMe I/O timeouts in the 4K q32 write cases with
the series applied. fio still exits successfully and the
numbers look plausible, so this is easy to miss unless you
check dmesg.
- (C) was unstable in general.
Test 3 below used next-20261022, which includes efadda401c5c.
3. Stress tests with an out-of-tree direct-DMA NTB transport
This is for reference only. These results use an out-of-tree NTB
transport, based on next-20261022.
(A).
- No stalls observed with various workloads.
- EP-to-RC ntb_netdev traffic reached about 25~26 Gbps,
with no CPU reaching 100% utilization.
(B).
- No stalls observed with various workloads.
- EP-to-RC ntb_netdev traffic reached about 28~29 Gbps,
with no CPU reaching 100% utilization.
(C).
- Can reach 18~19 Gbps, but the stall can be observed and is still
under investigation [1].
[1] https://lore.kernel.org/r/tkyu5rsxfa7i6rxxw6qxmmwke5qnlxptwx4j2s24fhftsmmf26@gamef2ec4wwm/
[2] https://lore.kernel.org/r/20260820173439.2004068-1-den@xxxxxxxxxxxxx/
[3] Based on Ubuntu-riscv-7.3.0-9.9.1 + some other out-of-tree patches,
especially to make EP mode work on K3.
Best regards,
Koichiro Den
---
v5->v6 changes:
- Drop the recovery code and the defensive doorbell handling added for
the R-Car S4 stall: IRQ-paired LLP snapshots and event
serialization, the delayed stopped-channel recheck, tx_status()
reclaim, the LL interrupt placement policy, engine reset and stall
recovery. Empirically, the stall so far reproduces only on S4 (5.20a
(GA)) with out-of-tree direct-DMA NTB transport.
- Read progress in the deferred handler instead. A channel counts as
stopped when its status is STOPPED and the transfer size is zero, on
both eDMA and native HDMA.
- Fold the STOP/PAUSE drain into the progress handler.
- Rebase on the circular LL groundwork in dmaengine-7.4-rc1. 14
patches -> 5. Each patch notes its v5 origin after "---".
For earlier changes and hardware observations, see v5:
https://lore.kernel.org/r/20260812155721.2807506-1-den@xxxxxxxxxxxxx/
Frank Li (2):
dmaengine: dw-edma: Dynamically append requests while running
dmaengine: dw-edma: Add trace support
Koichiro Den (3):
dmaengine: dw-edma: Add a channel transfer size operation
dmaengine: dw-edma: Reclaim LL entries from hardware progress
dmaengine: dw-edma: Use HDMA watermark interrupts as LL progress
drivers/dma/dw-edma/Makefile | 2 +
drivers/dma/dw-edma/dw-edma-core.c | 216 ++++++++++++++++++++++----
drivers/dma/dw-edma/dw-edma-core.h | 8 +
drivers/dma/dw-edma/dw-edma-trace.c | 4 +
drivers/dma/dw-edma/dw-edma-trace.h | 117 ++++++++++++++
drivers/dma/dw-edma/dw-edma-v0-core.c | 6 +
drivers/dma/dw-edma/dw-hdma-v0-core.c | 71 +++++++--
drivers/dma/dw-edma/dw-hdma-v0-regs.h | 1 +
8 files changed, 383 insertions(+), 42 deletions(-)
create mode 100644 drivers/dma/dw-edma/dw-edma-trace.c
create mode 100644 drivers/dma/dw-edma/dw-edma-trace.h
base-commit: aff5e6a77fc69a7c490246465940d18422d7c631
--
2.54.0