Re: [PATCH v5 1/2] KVM: x86: Advance guest TSC after deep suspend.

[Sean Christopherson]

On Tue, Apr 22, 2025, Tzung-Bi Shih wrote:
> On Tue, Mar 25, 2025 at 01:13:49PM +0900, Suleiman Souhlal wrote:
> > Advance guest TSC to current time after suspend when the host
> > TSCs went backwards.
> > 
> > This makes the behavior consistent between suspends where host TSC
> > resets and suspends where it doesn't, such as suspend-to-idle, where
> > in the former case if the host TSC resets, the guests' would
> > previously be "frozen" due to KVM's backwards TSC prevention, while
> > in the latter case they would advance.
> > 
> > Suggested-by: Sean Christopherson <seanjc@xxxxxxxxxx>
> > Signed-off-by: Suleiman Souhlal <suleiman@xxxxxxxxxx>
> 
> Tested with comparing `date` before and after suspend-to-RAM[1]:
>   echo deep >/sys/power/mem_sleep
>   echo $(date '+%s' -d '+3 minutes') >/sys/class/rtc/rtc0/wakealarm
>   echo mem >/sys/power/state
> 
> Without the patch, the guest's `date` is slower (~3 mins) than the host's
> after resuming.
> 
> Tested-by: Tzung-Bi Shih <tzungbi@xxxxxxxxxx>
> 
> [1]: https://www.kernel.org/doc/Documentation/power/states.txt
> 
> Some non-functional comments inline below.
> 
> > @@ -4971,7 +4971,37 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
> >  
> >  	/* Apply any externally detected TSC adjustments (due to suspend) */
> >  	if (unlikely(vcpu->arch.tsc_offset_adjustment)) {
> > -		adjust_tsc_offset_host(vcpu, vcpu->arch.tsc_offset_adjustment);
> > +		unsigned long flags;
> > +		struct kvm *kvm;
> > +		bool advance;
> > +		u64 kernel_ns, l1_tsc, offset, tsc_now;
> > +
> > +		kvm = vcpu->kvm;
> 
> It will be more clear (at least to me) if moving the statement to its declaration:
>   struct kvm *kvm = vcpu->kvm;
> 
> Other than that, the following code should better utilitize the local
> variable, e.g. s/vcpu->kvm/kvm/g.
> 
> > +		advance = kvm_get_time_and_clockread(&kernel_ns,
> > +		    &tsc_now);

In addition to Tzung-Bi's feedback...

Please don't wrap at weird points, and align when you do wrap.  The 80 char limit
isn't a super hard limit, and many of these wraps are well below that anyways.

		advance = kvm_get_time_and_clockread(&kernel_ns, &tsc_now);
		raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
		/*
		 * Advance the guest's TSC to current time instead of only
		 * preventing it from going backwards, while making sure
		 * all the vCPUs use the same offset.
		 */
		if (kvm->arch.host_was_suspended && advance) {
			l1_tsc = nsec_to_cycles(vcpu,
						vcpu->kvm->arch.kvmclock_offset + kernel_ns);
			offset = kvm_compute_l1_tsc_offset(vcpu, l1_tsc);
			kvm->arch.cur_tsc_offset = offset;
			kvm_vcpu_write_tsc_offset(vcpu, offset);
		} else if (advance) {
			kvm_vcpu_write_tsc_offset(vcpu, vcpu->kvm->arch.cur_tsc_offset);
		} else {
			adjust_tsc_offset_host(vcpu, vcpu->arch.tsc_offset_adjustment);
		}
		kvm->arch.host_was_suspended = 0;
		raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);


As for the correctness of this code with respect to masterclock and TSC
synchronization, I'm definitely going to have to stare even more, and probably
bring in at least Paolo for a consult, because KVM's TSC code is all kinds of
brittle and complex.