[PATCH v3 0/4] blk-iocost: charge flushes and zone appends

[Tao Cui]

From: Tao Cui <cuitao@xxxxxxxxxx>

While testing iocost's weight-based throttling under concurrent IO, we
observed that a cgroup limited to 1% weight could issue an unbounded
number of flushes without being throttled: an fsync loop produced ~510k
flushes in 12s with cost.usage staying at zero the entire time.  The
device was monopolized while iocost reported no activity.  Zone appends
were in the same position: the builtin linear cost model defines
coefficients only for READ and WRITE, so REQ_OP_ZONE_APPEND is priced
at zero and excluded from the latency statistics as well.

This series prices both, without changing the behavior of existing
setups:

 1/4: add a flushiops entry to io.cost.model, translated like the
      other iops coefficients (VTIME_PER_SEC / flushiops).  A bio with
      REQ_PREFLUSH is charged one flush on top of its data cost, and a
      bio with REQ_FUA one more flush on devices without native FUA
      support (bdev_fua()), mirroring the pre-flush and post-flush the
      block layer issues for them.  flushiops is zero in the builtin
      profiles with no fallback to the write coefficients, so nothing
      changes until it is configured.  Also documents flushiops in
      cgroup-v2.rst.
 2/4: treat zone appends as writes in the builtin cost model -
      REQ_OP_ZONE_APPEND falls through to the REQ_OP_WRITE
      coefficients.
 3/4: count zone append completions in the latency window so the vrate
      feedback loop can see ZA-induced latency.
 4/4: fix a stale comment in ioc_rqos_throttle().

Measured on virtio-blk with a bio test module issuing bios from a
cgroup (300 bios per phase):

  - plain writes cost 23.41 usec, unchanged
  - with the default model, WRITE|PREFLUSH and standalone flushes cost
    the same as before: the flush component is free until flushiops is
    configured
  - with flushiops=100, WRITE|PREFLUSH and WRITE|FUA are charged
    +10000 usec per bio and WRITE|PREFLUSH|FUA +20000, exactly
    VTIME_PER_SEC / 100 per flush (virtio-blk has no native FUA,
    confirmed via queue/fua)
  - standalone flushes are charged 10000.06 usec each, matching
    VTIME_PER_SEC / 100
  - with flushiops=10, a flush storm from a low-weight cgroup is
    throttled to one flush per 100ms

Changes in v2:
- charge the flush component of data-bearing REQ_PREFLUSH bios too,
  not only standalone flushes (Christoph)
- skip the iocg->cursor update for dataless flush bios

Changes in v3:
- the write-coefficient flush pricing is replaced by the flushiops
  model parameter: pricing a flush off the write coefficients is as
  arbitrary as pricing it at zero and changes what existing setups
  get charged, so flushes are only priced when flushiops is
  configured (Tejun)
- the zone append patch is reduced to the minimal change: ZA falls
  through to the REQ_OP_WRITE coefficients and the cursor is left
  alone (Tejun)
- the Fixes tags on 2/4 and 3/4 point at 0512a75b98f8, which added
  the op without pricing it; 3/4 is otherwise unchanged from v2

Tao Cui (4):
  blk-iocost: add flush cost support with the flushiops model parameter
  blk-iocost: charge zone appends as writes
  blk-iocost: account zone append completions in latency stats
  blk-iocost: fix stale comment in ioc_rqos_throttle()

 Documentation/admin-guide/cgroup-v2.rst |  8 ++++++++
 block/blk-iocost.c                      | 40 ++++++++++++++++++++++---------
 2 files changed, 48 insertions(+), 9 deletions(-)

-- 
2.43.0