[PATCH v3 0/4] Fix HugeTLB subpool used_hpages tracking

[Ackerley Tng via B4 Relay]

HugeTLB subpools currently only track used pages (used_hpages) when a
maximum size limit (max_hpages) is configured on the mount.

This breaks minimum size (min_hpages) guarantees. The subpool guarantee
requires that the sum of used pages and remaining subpool reserves
satisfies the configured minimum:

  used_hpages + rsv_hpages >= min_hpages

where used_hpages includes both allocated folios and active reservations.
Therefore, when pages or reservations are released, the maximum number of
reservations the subpool can absorb is:

  limit = max(0, min_hpages - used_hpages)

When a mount specifies a minimum size guarantee without a maximum limit,
used_hpages is untracked and remains zero. The subpool assumes limit ==
min_hpages, falsely restoring reservations to the subpool even when
existing usage already satisfies the guarantee.

This series addresses three problem areas resulting from untracked usage
and asymmetric error handling:

1. False restoration and premature subpool freeing (Patch 1):
    + Global reservations (resv_huge_pages) are not decremented when
      reservations are released, depleting available huge pages host-wide.
    + On unmount, subpool_is_free() checks (rsv_hpages == min_hpages).
      Because of the false reservation restoration, this evaluates to true,
      freeing the subpool while folios are still active in the page cache
      and causing a use-after-free when they are later released.
      (See Patch 1 for a detailed step-by-step trace.)

2. Reservation rollback races and underflow (Patches 2 & 4):
    + On hugetlb_acct_memory() failure in hugetlb_reserve_pages(), the
      error path manually calculates rollbacks using local variables.
      This is race-prone when concurrent threads interact with the
      subpool, risking reservation leaks or counter underflows.
    + On region_add() failure, prematurely releasing global reservations
      forces error cleanup to attempt fresh allocations that can fail
      under memory pressure.

3. Subpool usage leak on allocation failure (Patch 3):
    + In alloc_hugetlb_folio(), when folio allocation fails and
      gbl_chg == 1, the cleanup path skips hugepage_subpool_put_pages(),
      permanently leaking used_hpages.

Always tracking used_hpages establishes a single source of truth within the
subpool. Allocation and reservation error paths can now handle failures
symmetrically: return the full page count to the subpool via
hugepage_subpool_put_pages(), and rely on its return value to reconcile
global reservations with hugetlb_acct_memory().

This series is a subset of patches from [1] and replaces [1].

[1] https://lore.kernel.org/all/20260722-hugetlb-alloc-failure-fixes-v4-0-88e8b81970dc@xxxxxxxxxx/

Tested:

+ Reproducers (see below) pass
+ tools/testing/selftests/mm/ksft_hugetlb.sh passes
+ libhugetlbfs tests pass

Changes in v3:

+ Rebased on 7.3-rc3.
+ Picked up Reviewed-bys.
+ Rename resv_get_accted to resv_get_accounted as Joshua requested.
+ Now Cc-ing stable on patch 4.
+ Moved trace of a false reservation restoration from cover letter into
  patch 1.

v2: https://lore.kernel.org/r/20260909-hugetlb-subpool-always-track-used-v2-0-30c5d83b572a@xxxxxxxxxx
v1: https://lore.kernel.org/r/20260902-hugetlb-subpool-always-track-used-v1-0-de1cd14bd713@xxxxxxxxxx

Get reproducers from:

https://github.com/googleprodkernel/linux-cc/commits/hugetlb-subpool-always-track-used-with-reproducers-v3

Signed-off-by: Ackerley Tng <ackerleytng@xxxxxxxxxx>
---
Ackerley Tng (4):
      mm: hugetlb: Track used_hpages when getting/putting pages from subpool
      mm: hugetlb: Fix out_put_pages subpool reserve calculation
      mm: hugetlb: Fix subpool usage leak on allocation failure
      mm: hugetlb: Avoid re-allocating global reservations on region add failure

 Documentation/mm/hugetlbfs_reserv.rst              |  17 +--
 .../translations/zh_CN/mm/hugetlbfs_reserv.rst     |  11 +-
 fs/hugetlbfs/inode.c                               |   8 +-
 include/linux/hugetlb.h                            |   4 +-
 mm/hugetlb.c                                       | 142 +++++++++++----------
 5 files changed, 85 insertions(+), 97 deletions(-)
---
base-commit: fd73f4a6659897191fa0d40695fe370925dd3780
change-id: 20260902-hugetlb-subpool-always-track-used-2624840f4c08

Best regards,
--
Ackerley Tng <ackerleytng@xxxxxxxxxx>