Re: [PATCH 00/13] virtio_balloon, efi/unaccepted: support memory ballooning and free page reporting in CoCo guests

[David Hildenbrand (Arm)]

On 10/27/26 03:56, Kevin Loughlin wrote:
> This series enables virtio-balloon free page reporting and memory balloon
> inflation/deflation in Confidential Computing (CoCo) guests, starting
> with architectural hooks implemented for AMD SEV variants, in order to
> support dynamic CoCo guest resizing.
> 
> Today, memory ballooning and free page reporting are essentially unusable
> in CoCo guests due to three main roadblocks:
> 
> 1. SWIOTLB Bounce-Buffering on Free Page Reporting:
>    Under VIRTIO_F_ACCESS_PLATFORM, virtqueue_add_inbuf() maps every
>    scatterlist entry through the DMA API. For reporting_vq, these are
>    multi-megabyte ranges of free buddy pages. The host never DMA-reads
>    or writes the page contents; it only inspects the GPA to punch holes
>    in backing memory. Running these ranges through SWIOTLB exhausts
>    bounce buffer pools and wastes cycles copying gigabytes of
>    unencrypted free memory.
> 
> 2. Dynamic Memory Unacceptance & Re-acceptance:
>    When the host reclaims guest physical pages via hole punching,
>    the guest must transition those GPAs back to unaccepted/shared before
>    the host reclaims them (rescinding PVALIDATE and issuing a GHCB Page
>    State Change on SEV-SNP). Otherwise, any subsequent access triggers a
>    fatal #NPF or #VC. While the EFI unaccepted memory subsystem handles
>    boot-time lazy acceptance, it lacks an API or state tracking to
>    dynamically unaccept memory at runtime or route unaccepted freed
>    pages in the page allocator.
> 
> 3. Acceptance Granularity (2 MiB) vs. Balloon Pages (4 KiB):
>    On x86, the EFI unaccepted memory bitmap tracks state at 2 MiB
>    (PMD_ORDER) granularity both to keep the bitmap compact and for
>    performance: accepting and unaccepting in 2 MiB blocks preserves
>    2 MiB host hugepage mappings (NPT/RMP on SEV-SNP and Secure EPT on
>    TDX), avoids 512x per-4 KiB page state transition and validation
>    overhead, and prevents RMP/SEPT size-mismatch splits. Because a
>    2 MiB bitmap unit cannot be partially unaccepted, CoCo free page
>    reporting and balloon inflation require 2 MiB blocks and do not
>    fall back to 4 KiB allocations (though low-level PVALIDATE still
>    falls back to 4 KiB instructions if the host RMP entry was already
>    split). Standard virtio-balloon allocates 4 KiB pages and caps its
>    PFN array at 256 PFNs (1 KiB descriptor), so CoCo guests must
>    allocate 2 MiB compound pages while streaming PFNs in 256-PFN chunks
>    to respect existing VMM descriptor limits (such as QEMU's 1024-byte
>    cap).
> 
>    As a trade-off of requiring 2 MiB blocks rather than 4 KiB pages,
>    the effectiveness of balloon inflation/deflation in CoCo guests
>    depends on guest memory not being overly fragmented (which guest
>    kernel command-line options like movablecore / ZONE_MOVABLE or
>    background compaction can help mitigate). Even when unmovable
>    allocations fragment some pageblocks, free page reporting already
>    operates at this same 2 MiB (pageblock_order) granularity on x86_64
>    and continues to reclaim contiguous free blocks opportunistically.
> 
> Key Design Elements in This Series
> ==================================
> - Per-virtqueue DMA map ops (Patches 1-2):
>   Extends virtio_ring to allow individual virtqueues to install custom
>   struct virtio_map_ops. For reporting_vq, virtio_balloon installs a
>   no-op GPA identity mapper so free page ranges are passed directly
>   without SWIOTLB bounce buffers, while keeping the descriptor ring and
>   any indirect descriptor tables securely mapped via the device-level
>   DMA API.
> 
> - Dynamic unacceptance & SEV-SNP hooks (Patches 3-4):
>   Adds arch_unaccept_memory() / snp_unaccept_memory() for SEV-SNP. Also
>   adds exception fixup in vc_raw_handle_exception() for
>   SVM_EXIT_PAGE_NOT_VALIDATED (0x404) and SVM_EXIT_NPF (0x400) at
>   EX_TYPE_ZEROPAD instructions so speculative reads from
>   load_unaligned_zeropad() that touch the edge of dynamically unaccepted
>   memory are handled cleanly without kernel panics.
> 
> - Dynamic unaccepted memory tracking & MM routing (Patches 5-9):
>   Anchors the EFI unaccepted table at base 0 when unaccepted memory is
>   present, introduces mark_memory_unaccepted() with error reporting and
>   proper locking, generalizes zone->unaccepted_pages to variable orders
>   >= unaccepted_memory_order(), and updates mm/page_alloc,
>   mm/page_isolation, mm/page_owner, and mm/page_reporting to route
>   unaccepted pages back to zone->unaccepted_pages without corrupting
>   them with page poisoning, KASAN, or init_on_free before re-acceptance.
> 
>   Generalizing zone->unaccepted_pages from MAX_PAGE_ORDER (4 MiB on x86)
>   down to >= unaccepted_memory_order() (2 MiB) aligns allocator tracking
>   with the underlying hardware validation/RMP/SEPT and EFI bitmap unit:
>   even when boot-time lazy acceptance processes a MAX_PAGE_ORDER block,
>   the hardware and bitmap validate it in 2 MiB chunks, and runtime
>   allocations, pageblocks, and mm/page_reporting split accepted memory
>   below MAX_PAGE_ORDER anyway.
> 
>   Note for reviewers: The three mm/ patches (Patches 7-9) are the most
>   intricate part of this series due to the interactions between lazy
>   acceptance, page isolation, and page reporting. Extra scrutiny from
>   mm reviewers to ensure no page-allocator or pageblock corner cases
>   were missed would be especially appreciated.
> 
> - 2 MiB compound ballooning with 256-PFN chunking (Patch 10):
>   Allocates compound pages at PMD_ORDER in CoCo guests that require page
>   state transitions, breaking them into 256-PFN chunks when queuing to
>   the host so virtqueue descriptors remain at 1 KiB, preserving
>   out-of-the-box compatibility with existing hypervisors.
> 
> - Balloon state transitions & CoCo ACCESS_PLATFORM (Patches 11-13):
>   Calls mark_memory_unaccepted() on reported ranges and inflated pages
>   before adding buffers to reporting_vq and kicking the virtqueue, routes
>   deflated pages to unaccepted freelists, preserves
>   VIRTIO_F_ACCESS_PLATFORM, disables VIRTIO_BALLOON_F_FREE_PAGE_HINT and
>   unsupported VIRTIO_BALLOON_F_PAGE_POISON in CoCo guests, and rejects
>   probing on CoCo platforms where balloon page state transitions are
>   unsupported.
> 
> Why Both Free Page Reporting and Inflation/Deflation
> ====================================================
> CoCo guests using lazy memory acceptance boot with their maximum RAM
> declared as EFI_UNACCEPTED_MEMORY in the initial EFI memory map,
> avoiding runtime ACPI/virtio memory hotplug. Prior to this series, lazy
> acceptance was a one-way ratchet: once a workload spike caused the
> guest to accept pages from zone->unaccepted_pages, those pages could
> never be unaccepted or returned to the host.
> 
> Turning EFI unaccepted memory into a bidirectional lifecycle enables two
> complementary mechanisms on initial boot RAM:
> - Free Page Reporting (reporting_vq): Autonomously reports idle 2 MiB
>   free buddy pages to the host for hole-punching, and lazily re-accepts
>   them locally from zone->unaccepted_pages on demand without a host
>   virtqueue round-trip.
> - Balloon Inflation/Deflation (inflate_vq / deflate_vq): Allows VMMs
>   that size guests without hotplugging to actively apply memory pressure
>   (triggering guest reclaim) and enforce dynamic memory targets on boot
>   RAM.
> 
> Architecture Support & Intel TDX Follow-Up
> ==========================================
> The efi/unaccepted, mm/, and virtio_balloon changes in this series are
> architecture-agnostic, with architecture hooks (arch_unaccept_memory())
> wired up first for AMD SEV-SNP via snp_unaccept_memory() (alongside
> VIRTIO_F_ACCESS_PLATFORM support for SEV-ES).
> 
> On Intel TDX, I think dynamic unacceptance should use
> TDG.MEM.PAGE.RELEASE to transition private pages from MAPPED back to
> PENDING so the host VMM can reclaim them (and the guest can later
> re-accept them via the existing TDG.MEM.PAGE.ACCEPT path in
> tdx_accept_memory()). Once TDG.MEM.PAGE.RELEASE support and
> load_unaligned_zeropad() handling for dynamically unaccepted pages
> (e.g., via direct-map unmapping under SEPT_VE_DISABLE) land upstream in
> arch/x86/coco/tdx/, tdx_unaccept_memory() can slot directly into
> arch_unaccept_memory().
> 
> Testing
> =======
> - AMD SEV-SNP Milan hardware (CoCo): Passed end-to-end memory pressure
>   soak testing with live guest_memfd hole punching (FALLOC_FL_PUNCH_HOLE)
>   and dynamic lazy re-acceptance under memory pressure on an internal
>   hypervisor.
> - Cloud Hypervisor & QEMU: Verified standard non-confidential
>   ballooning remains compatible. End-to-end SEV-SNP balloon testing on
>   upstream Cloud Hypervisor and QEMU has been limited so far (while
>   both punch holes in guest_memfd on shared page-state transitions,
>   QEMU's virtio-balloon still issues MADV_WILLNEED on shared HVA
>   mappings during deflate); any testing help or suggestions on these
>   hypervisors are greatly appreciated.
> 
> Note: An LLM coding assistant was used during the development of this
> series to assist with code and commit message drafting and refactoring.
> I have reviewed and verified all changes.

[...]

>  mm/page_alloc.c                               | 248 ++++++++++++-----

This diff stat should give you a good indication why this series is not acceptable.

-- 
Cheers,

David