Re: Path forward for Virtualized Swap?

[Kairui Song]

On Tue, Oct 27, 2026 at 12:01 AM Nhat Pham <nphamcs@xxxxxxxxx> wrote:
>
> On Sat, Oct 24, 2026 at 9:22 PM Kairui Song <ryncsn@xxxxxxxxx> wrote:
> >
> > Rik van Riel <riel@xxxxxxxxxxx> 于 2026年10月25日周日 02:16写道：
> > >
> > > On Sat, 2026-10-24 at 23:50 +0800, Kairui Song wrote:
> > > > On Sat, Oct 24, 2026 at 4:03 AM Rik van Riel <riel@xxxxxxxxxxx>
> > > > wrote:
> > > > >
> > > > > On Fri, 2026-10-23 at 10:24 -0700, Nhat Pham wrote:
> > > > > >
> > > > > > * Swapoff: this was brought up as a request, but we ran out of
> > > > > > time
> > > > > > before we could discuss the motivations and use cases behind
> > > > > > them.
> > > > > > Folks interested, could you explain as a follow-up?
> > > > >
> > > > > This is one that I really don't understand.
> > > > >
> > > > > In my mind, we have vswap sitting as kind
> > > > > of a swap VFS between the rest of the system
> > > > > and the swap devices:
> > > > >
> > > > > +-------+     +------------+
> > > > > > vswap |-----| swap cache |
> > > > > +---+---+     +------------+
> > > > >     |
> > > > >     +----------+-----------+
> > > > >     |          |           |
> > > > > +---+---+  +---+---+   +---+---+
> > > > > > swap  |  | zswap |   | zram  |
> > > > > > file  |  |       |   |       |
> > > > > +-------+  +-------+   +-------+
> > > > >
> > > > > When a page is unmapped from userspace by the
> > > > > reclaim code, it goes into the swap cache.
> > > > >
> > > > > The vswap layer then decides where to store it
> > > > > at eviction time. This is configurable by userspace.
> > > > >
> > > > > If the zswap pool is shrunk, the vswap layer knows
> > > > > where to put the data next. There is no need to
> > > > > change page table entries to indicate that the
> > > > > data was moved from zswap to a swap file.
> > > > >
> > > > > If we decide to disable one of the backing swap
> > > > > devices, the vswap layer can just move the data
> > > > > into the swap cache, and the page reclaim code
> > > > > can evict it into a different backing store, if
> > > > > needed.
> > > >
> > > > And in this case we want to disable vswap itself.
> > >
> > > OK, so you are looking for a way to totally depopulate
> > > the swap cache, in addition to swap files, zswap, etc?
> > >
> > > I think we can accomplish that by simply keeping the
> > > current swapoff code around, which scans through the
> > > virtual memory of all processes and brings the memory
> > > out of swap.
> >
> > Yeah, that's the right starting point. The scan itself can also be
> > optimized to be cheaper or better later on, but that's orthogonal.
> >
> > > > I also hope there is a way to release swap metadata completely too.
> > > > e.g. for fs, you at least have umount, and can rmmod the fs itself
> > > > afterwards. We also have many ways to bypass the page cache. In cases
> > > > where a virtual layer is just a burden, we want raw physical swap,
> > > > and
> > > > the ability to swap off that too.
> > >
> > > The VM cannot operate without the swap cache, though.
> > >
> > > That is very much built into core parts of the VM.
> >
> > I never meant to strip swap cache out of VM... The opposite, actually
> > :), most of my work over the last couple of years has been about
> > removing the paths that bypass the swap cache (e.g. the
> > SWP_SYNCHRONOUS_IO fast path), to get a simpler model and better
> > performance.
> >
> > What I'd like is for swapoff of the last swap device to return the
> > system to its pre-swapon state: no leftover swap metadata, no leftover
> > per-slot or per-cgroup data or statistic, and ideally no residual cost
> > in the fast paths (e.g. minor fault).
>
> So basically remapping? I don't think that's terribly hard to do. I
> mean the swapoff code is still there, so it's just the question of
> exposing an off knob, basically?
>
> But wouldn't MADV_WILLNEED be the better tool here? Or is it not reliable?

I think I've mentioned it above? You will have to iterate through
every process in every container every namespace, from userspace, and
do some remote memory access / control on all of them, which is really
not a good solution.. It's not reliable and is really hard to maintain
or deploy..