[PATCH v7 19/20] cpufreq: CPPC: Select the frequency-invariance callback per CPU

[Christian Loehle]

cppc_cpufreq_cpu_fie_init() initializes deferred work only for CPUs whose
feedback counters use PCC. However, if any CPU in a shared policy uses PCC,
it installs the PCC tick callback for the entire policy. With FIE enabled,
a non-PCC CPU then queues an uninitialized irq_work and can call a NULL
function pointer when that work runs.

Choose the frequency-invariance callback separately for each CPU using
the recorded work-initialization state. Processor removal can unpublish
the CPC descriptor before callback registration, so querying its counter
transport again can incorrectly select direct reads for a PCC CPU.

Register the callbacks after all counter initialization has completed, so
a failed initial read for an online CPU cannot leave a partially
registered policy. PCC CPUs use their initialized deferred work, while
non-PCC CPUs read their own counters directly from the tick.

Fixes: 997c021abc6e ("cpufreq: CPPC: Update FIE arch_freq_scale in ticks for non-PCC regs")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Christian Loehle <christian.loehle@xxxxxxx>
---
 drivers/cpufreq/cppc_cpufreq.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
index 9dde24d3e645..c0fa9073e4d3 100644
--- a/drivers/cpufreq/cppc_cpufreq.c
+++ b/drivers/cpufreq/cppc_cpufreq.c
@@ -150,7 +150,7 @@ static struct scale_freq_data cppc_sftd_pcc = {
 
 static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
 {
-	struct scale_freq_data *sftd = &cppc_sftd;
+	struct scale_freq_data *sftd;
 	struct cppc_freq_invariance *cppc_fi;
 	int cpu, ret;
 
@@ -165,7 +165,6 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
 			kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
 			init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
 			cppc_fi->pcc_work_initialized = true;
-			sftd = &cppc_sftd_pcc;
 		}
 
 		ret = cppc_get_perf_ctrs(cpu, &cppc_fi->prev_perf_fb_ctrs);
@@ -181,8 +180,15 @@ static void cppc_cpufreq_cpu_fie_init(struct cpufreq_policy *policy)
 		}
 	}
 
-	/* Register for freq-invariance */
-	topology_set_scale_freq_source(sftd, policy->cpus);
+	/* A shared policy may contain both PCC and non-PCC counters. */
+	for_each_cpu(cpu, policy->cpus) {
+		cppc_fi = &per_cpu(cppc_freq_inv, cpu);
+		if (cppc_fi->pcc_work_initialized)
+			sftd = &cppc_sftd_pcc;
+		else
+			sftd = &cppc_sftd;
+		topology_set_scale_freq_source(sftd, cpumask_of(cpu));
+	}
 }
 
 /*
-- 
2.34.1