Re: [PATCH v2 11/16] KVM: x86/pmu: Emulate the GLOBAL_STATUS_SET and GLOBAL_INUSE MSRs

[Yosry Ahmed]

> > @@ -865,6 +867,7 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
> >         case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
> >         case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
> >         case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
> > +       case MSR_CORE_PERF_GLOBAL_STATUS_SET:
> >                 msr_info->data = 0;
> >                 break;
> >         default:
> > @@ -931,6 +934,12 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
> >                 if (!msr_info->host_initiated)
> >                         pmu->global_status &= ~data;
> >                 break;
> > +       case MSR_CORE_PERF_GLOBAL_STATUS_SET:
> > +               if (data & pmu->global_status_rsvd)
> > +                       return 1;
> > +               if (!msr_info->host_initiated)
> > +                       pmu->global_status |= data;
> > +               break;
> 
> According to Intel SDM Vol. 4, Table 2-2 ("IA-32 Architectural MSRs"),
> IA32_PERF_GLOBAL_STATUS.CondChgd[bit 63] cannot be set by writing to
> IA32_PERF_GLOBAL_STATUS_SET, because IA32_PERF_GLOBAL_STATUS_SET[bit 63] is
> reserved.  In other words, it is not possible to save/restore
> IA32_PERF_GLOBAL_STATUS when bit 63 is set.
> 
> Because bit 63 is clear in pmu->global_status_rsvd:
> 
> 1. An intercepted guest WRMSR(MSR_CORE_PERF_GLOBAL_STATUS_SET, BIT_ULL(63))
>    succeeds instead of injecting #GP(0) and sets bit 63 in
>    pmu->global_status.
> 2. When bit 63 is set in pmu->global_status (whether it was set by
>    kvm_pmu_set_msr() or by intel_mediated_pmu_put()), the next
>    intel_mediated_pmu_load() will try to write an illegal value to
>    MSR_CORE_PERF_GLOBAL_STATUS_SET. I don't think we get a #GP, because
>    every wrmsr is "safe" these days, but the write will be ignored.

On SPR, it does produce a #GP that gets ignored (and logged) and the bit
is not set.  Interestingly, on CLX, the write goes through and the bit
is set. Didn't check other CPUs.

> OTOH, what about PerfMon v4 bits 58 (GLOBAL_STATUS_LBRS_FROZEN), 59
> (GLOBAL_STATUS_COUNTERS_FROZEN), and 61 (GLOBAL_STATUS_OVF_UNCORE)?  Per
> Intel SDM Vol. 4, Table 2-2, those bits are valid in
> IA32_PERF_GLOBAL_STATUS, IA32_PERF_GLOBAL_STATUS_RESET, and
> IA32_PERF_GLOBAL_STATUS_SET when CPUID.0AH:EAX[7:0] > 3.
> 
> When those MSRs are passed through in mediated mode, the guest can set bits
> 58, 59, or 61 in hardware via IA32_PERF_GLOBAL_STATUS_SET. Leaving those
> bits set in pmu->global_status_rsvd for pmu->version >= 4 will break
> KVM_SET_MSRS(MSR_CORE_PERF_GLOBAL_STATUS) as well as any intercepted guest
> writes to GLOBAL_OVF_CTRL / GLOBAL_STATUS_SET.
> 
> As I pointed out in patch 04/16, we can't pass through the PMU global
> controls unless their reserved bits match hardware.

I think there are two other relevant (but pre-existing) problems here:

1. GLOBAL_STATUS_TRACE_TOPAPMI_BIT can leak from the host into the
guest. If a host PT PMI arrives while the guest context is loaded, bit
55 will be set for the guest, and the guest won't be able to clear it if
OVF_CTRL is intercepted (it will #GP as it is reserved for the guest).
Similarly, the value can't be restored through KVM_SET_MSRS.

(I think the PMI may also be lost as LVTPC points at the mediated vPMU
vector)

2. On AMD, bit 58 and 59 have a similar problem. HW accepts those bits
but they are reserved for the guest, so the guest can set them if
GLOBAL_STATUS_SET is passed through, but userspace cannot restore them.