[PATCH 09/17] KVM: x86: Post KVM_REQ_GET_NESTED_STATE_PAGES on memslot updates

[David Woodhouse]

From: David Woodhouse <dwmw@xxxxxxxxxxxx>

The physical addresses which nested page setup latches into hardware
control structures (vmcs02's APIC-access, virtual-APIC and posted
interrupt descriptor addresses) are derived from gPA→uHVA translations
which a memslot update can change. Software readers of the underlying
gfn_to_pfn_caches catch this lazily, via the memslot generation check
in kvm_gpc_check() at their next use — but the CPU's use of a latched
address from guest mode is continuous and checks nothing. A vCPU
running L2 across a memslot move would keep using the old translation
until something forced it to re-resolve; L0 exits which re-enter L2
without a nested VM-exit never re-run nested page setup.

(This is a staleness, not a lifetime, problem: freeing the underlying
page is the mmu_notifier's business and that path kicks pinned vCPUs
synchronously. The replaced kvm_host_map code had the same staleness
with no remedy at all.)

The alternative, checking each cache's memslot generation in the
VM-entry path after vcpu->mode is set, is strictly worse: the check
would run on every nested VM-entry forever, in a context which cannot
refresh (IRQs off), so its only possible action on a mismatch would be
to post KVM_REQ_GET_NESTED_STATE_PAGES and bail for the refresh to
happen outside. Posting that same request from the memslot update
itself — the single point where the generation actually changes, and
a slow path by definition — is the same mechanism minus the per-entry
cost. The request bit is also the artifact that survives racing with
a concurrent VM-entry: a bare kick landing before vcpu->mode is set
would be lost, and a vCPU which resolved its pages against the old
memslots but has not yet entered guest mode is invisible to any
is_guest_mode() filter, so the request is posted unconditionally to
every vCPU.

Accordingly, downgrade the WARN in svm_get_nested_state_pages(): a
spurious request outside guest mode is now expected, and a no-op.
(vmx_get_nested_state_pages already tolerates it.)

The memslot-move mode of the vmx_apic_update_test selftest exercises
this path.

Signed-off-by: David Woodhouse <dwmw@xxxxxxxxxxxx>
Assisted-by: Claude:claude-mythos-5
---
 arch/x86/kvm/svm/nested.c |  7 ++++++-
 arch/x86/kvm/x86.c        | 26 ++++++++++++++++++++------
 2 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 73f37b050d0a..acc423b13445 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -2106,7 +2106,12 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,
 
 static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)
 {
-	if (WARN_ON(!is_guest_mode(vcpu)))
+	/*
+	 * Memslot updates post this request to every vCPU (to make any
+	 * vCPU which has guest pages latched re-resolve them against the
+	 * new memslots), so it can arrive with nothing to do.
+	 */
+	if (!is_guest_mode(vcpu))
 		return true;
 
 	if (is_pae_paging(vcpu)) {
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 116932e13d59..07d1cfb051f5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -10172,18 +10172,32 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm,
 
 void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen)
 {
-	struct kvm_vcpu *vcpu;
-	unsigned long i;
-
 	/*
 	 * memslots->generation has been incremented.
 	 * mmio generation may have reached its maximum value.
 	 */
 	kvm_mmu_invalidate_mmio_sptes(kvm, gen);
 
-	/* Force re-initialization of steal_time cache */
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		kvm_vcpu_kick(vcpu);
+	/*
+	 * Force re-initialization of the steal_time cache, and of any
+	 * nested-state pages whose physical addresses a vCPU has latched
+	 * in hardware control structures (e.g. vmcs02) from a
+	 * gfn_to_pfn_cache. Software readers of such caches catch the
+	 * generation bump lazily, via kvm_gpc_check() at their next use;
+	 * the CPU's use from guest mode is continuous and checks nothing,
+	 * so the vCPU must be told to re-resolve and re-latch before it
+	 * next enters the guest. The request is the artifact that
+	 * survives racing with a concurrent VM-entry (a bare kick landing
+	 * before vcpu->mode is set would be lost); its handler re-runs
+	 * nested page setup, whose gPA lookups then see the new
+	 * generation and refresh.
+	 *
+	 * The wake/kick this performs on every vCPU is also what forces
+	 * re-initialization of the steal_time cache: its check-at-use
+	 * sites likewise only see the new generation once the vCPU goes
+	 * around its run loop.
+	 */
+	kvm_make_all_cpus_request(kvm, KVM_REQ_GET_NESTED_STATE_PAGES);
 }
 
 int kvm_arch_prepare_memory_region(struct kvm *kvm,
-- 
2.55.0