LM regression: fce886a60207 KVM: arm64: Plumb the pKVM MMU in KVM

[Sebastian Ott]

Hey,

Doing back and forth migrations currently fails on arm after a couple iterations.
During the failing migration KVM_RUN exits via guest_abort and returns -ENOMEM.
I can reliably reproduce this by migrating between 2 qemu instances on an ampere
altra machine. This fails after < 5 iterations. In this case qemu would spit out
smth like this (other than that - nothing in the logs):

error: kvm run failed Cannot allocate memory
 PC=0000aaaae7d48590 X00=0000aaaae80a2e00 X01=0000aaaae7ea2fc0
X02=0000000001d3a5d0 X03=0000aaaae7eace8c X04=000000003b9aca00
X05=000000000000004a X06=000000000000004a X07=0000000028000000
X08=0000000000001d70 X09=0000000000000018 X10=000144b7d0000000
X11=00ffffffffffffff X12=000000008378f367 X13=0000aaab1a202d70
X14=0000000000000000 X15=0000000000000000 X16=0000ffffa2e2f7a8
X17=0000ffffa2541f20 X18=000000000000a000 X19=84bfda6288cf2dd6
X20=0000aaab1a1f1ce0 X21=000000007fffffff X22=0000ffffc5431788
X23=0000aaab1a17db60 X24=0000ffffc5431770 X25=0000000100000000
X26=0000004100000000 X27=0000000000000001 X28=0000aaab1a1f1c20
X29=0000ffffc54316d0 X30=0000aaaae7f8cd24  SP=0000ffffc5431650
PSTATE=20001000 --C- EL0t

Guest and host are otherwise idle, kvm is in normal VHE mode.

Git bisect points to (fce886a60207 "KVM: arm64: Plumb the pKVM MMU in KVM")
I also double checked that by reverting this commit on top of 6.14.

Sebastiangit bisect log
# bad: [38fec10eb60d687e30c8c6b5420d86e8149f7557] Linux 6.14
# good: [ffd294d346d185b70e28b1a28abe367bbfe53c04] Linux 6.13
git bisect start 'bad' 'good'
# good: [2c8d2a510c15c003749e43ac2b8e1bc79a7a00d6] Merge tag 'sound-6.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
git bisect good 2c8d2a510c15c003749e43ac2b8e1bc79a7a00d6
# good: [9ff28f2fad67e173ed25b8c3a183b15da5445d2d] Merge tag 'loongarch-6.14' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
git bisect good 9ff28f2fad67e173ed25b8c3a183b15da5445d2d
# bad: [243899076c3efdf98d8e922a802896424a597580] Merge tag 'rust-fixes-6.14-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux
git bisect bad 243899076c3efdf98d8e922a802896424a597580
# bad: [95d7e8226106e3445b0d877015f4192c47d23637] Merge tag 'ata-6.14-rc1-part2' of git://git.kernel.org/pub/scm/linux/kernel/git/libata/linux
git bisect bad 95d7e8226106e3445b0d877015f4192c47d23637
# bad: [7c775c6056d07eb777f37c7ac1340115b27dc9f8] Merge tag 'dmaengine-6.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
git bisect bad 7c775c6056d07eb777f37c7ac1340115b27dc9f8
# bad: [7c775c6056d07eb777f37c7ac1340115b27dc9f8] Merge tag 'dmaengine-6.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
git bisect bad 7c775c6056d07eb777f37c7ac1340115b27dc9f8
# bad: [f785692ff545aecb517d2609864e1c6d838329e6] Merge tag 'stop-machine.2025.01.28a' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu
git bisect bad f785692ff545aecb517d2609864e1c6d838329e6
# bad: [080612b2942ab7947303029e1fa33117b5280ece] Merge branch kvm-arm64/nv-timers into kvmarm-master/next
git bisect bad 080612b2942ab7947303029e1fa33117b5280ece
# bad: [d0670128d42fa170bf8ba878cd23504c5c5cccc7] Merge branch kvm-arm64/pkvm-np-guest into kvmarm-master/next
git bisect bad d0670128d42fa170bf8ba878cd23504c5c5cccc7
# good: [c4a6ed85455979ef3fbadc2f1bdf18734b0ecea6] KVM: arm64: Promote guest ownership for DBGxVR/DBGxCR reads
git bisect good c4a6ed85455979ef3fbadc2f1bdf18734b0ecea6
# good: [d0bd3e6570aee42766e7bd884734ae078667ea1e] KVM: arm64: Introduce __pkvm_host_share_guest()
git bisect good d0bd3e6570aee42766e7bd884734ae078667ea1e
# good: [76f0b18b3db57868fb0cabe691201aad3085b712] KVM: arm64: Introduce __pkvm_host_mkyoung_guest()
git bisect good 76f0b18b3db57868fb0cabe691201aad3085b712
# good: [e912efed485a4c50bdc3934ae647e257ef568ef6] KVM: arm64: Introduce the EL1 pKVM MMU
git bisect good e912efed485a4c50bdc3934ae647e257ef568ef6
# bad: [fce886a6020734d6253c2c5a3bc285e385cc5496] KVM: arm64: Plumb the pKVM MMU in KVM
git bisect bad fce886a6020734d6253c2c5a3bc285e385cc5496
# first bad commit: [fce886a6020734d6253c2c5a3bc285e385cc5496] KVM: arm64: Plumb the pKVM MMU in KVM