RE: [PATCH v5 16/27] vfio/cxl: Create the CXL memdev and set media ready at bind

[Manish Honap]

> -----Original Message-----
> From: Alex Williamson <alex@xxxxxxxxxxx>
> Sent: Tuesday, September 22, 2026 7:44 AM
> To: Manish Honap <mhonap@xxxxxxxxxx>
> Cc: jgg@xxxxxxxx; Ankit Agrawal <ankita@xxxxxxxxxx>; jic23@xxxxxxxxxx;
> dave.jiang@xxxxxxxxx; alejandro.lucero-palau@xxxxxxx; Srirangan
> Madhavan <smadhavan@xxxxxxxxxx>; corbet@xxxxxxx;
> skhan@xxxxxxxxxxxxxxxxxxx; dave@xxxxxxxxxxxx; alison.schofield@xxxxxxxxx;
> vishal.l.verma@xxxxxxxxx; iweiny@xxxxxxxxxx; ming.li@xxxxxxxxxxxx; Yishai
> Hadas <yishaih@xxxxxxxxxx>; Shameer Kolothum Thodi
> <skolothumtho@xxxxxxxxxx>; kevin.tian@xxxxxxxxx; bhelgaas@xxxxxxxxxx;
> dmatlack@xxxxxxxxxx; kees@xxxxxxxxxx; gustavoars@xxxxxxxxxx; Neo Jia
> <cjia@xxxxxxxxxx>; Krishnakant Jaju <kjaju@xxxxxxxxxx>; Vikram Sethi
> <vsethi@xxxxxxxxxx>; Zhi Wang <zhiw@xxxxxxxxxx>; linux-
> doc@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx;
> linux-cxl@xxxxxxxxxxxxxxx; linux-pci@xxxxxxxxxxxxxxx; linux-
> kselftest@xxxxxxxxxxxxxxx; linux-hardening@xxxxxxxxxxxxxxx; alex@xxxxxxxxxxx
> Subject: Re: [PATCH v5 16/27] vfio/cxl: Create the CXL memdev and set media
> ready at bind
> 
> External email: Use caution opening links or attachments
> 
> 
> On Thu, 17 Sep 2026 00:05:29 +0530
> <mhonap@xxxxxxxxxx> wrote:
> 
> > From: Manish Honap <mhonap@xxxxxxxxxx>
> >
> > At bind, build the CXL memory device for the passed-through Type-2
> > accelerator so it joins the CXL topology and its HDM region resolves
> > to a host physical range. A Type-2 device has no mailbox, so there is
> > no media-ready register to poll: set media ready directly once the
> > component registers validate (mirroring
> > drivers/net/ethernet/sfc/efx_cxl.c)
> >
> > As per current vfio-cxl support, reject a device with:
> > - more than one HDM decoder
> > - interleaving enabled
> > - whose reset the host cannot service
> >
> > The CXL-core allocations are grouped with devres so a failed bind
> > unwinds
> > them: init failure falls back to plain vfio-pci with the device still
> > bound, so devm would otherwise hold them until unbind.
> >
> > A low-power transition would reset the CXL Type-2 function and lose
> > its CXL.mem contents, so keep it in D0 while it is assigned.
> 
> Is it the configuration or the mem contents?  Isn't it the user's problem if they
> put the device in low power state and lose their own data?  If it's our problem,
> what are we protecting?  It's not clear to me how we can save and restore the
> HDM state around reset, but can't allow low power.
> 
> This also undercuts your patch 12 justification for the open/close hooks under
> the rationale that we cannot sample the hardware while it might be in a low
> power state.
> 
> If we do block the low power features (less desirable), it should also be at the
> probe and dispatch level (-ENOTTY) rather than failing the SET operation with -
> EINVAL while reporting the feature available via PROBE.  Thanks,

On giving this another thought, I agree, nothing stops us from restoring HDM state
after low power either.
In v6 I will drop disable_idle_d3, the D0 clamp in vfio_pci_set_power_state() and
the LOW_POWER_ENTRY refusal, and handle low power the way reset works:

- On entry to D3hot or runtime low power, vfio-cxl revokes the HDM
   mapping and its dma-buf and closes the HDM access gate.
- On return to D0, after vfio-pci-core has restored config space
   (the existing needs_pm_restore path), vfio-cxl restores the decoder
   with the same cxl-core helper the reset path uses and reopens the
   gate.

With low power supported, for patch 12, the device can sit in low power
between bind and open. vfio-cxl will sample the live registers at open.

> 
> Alex