cpufreq/sched: Explicitly synchronize limits_changed flag handling

JIRA: https://issues.redhat.com/browse/RHEL-96250

commit 79443a7
Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Date:   Tue Apr 15 11:59:15 2025 +0200

    cpufreq/sched: Explicitly synchronize limits_changed flag handling

    The handling of the limits_changed flag in struct sugov_policy needs to
    be explicitly synchronized to ensure that cpufreq policy limits updates
    will not be missed in some cases.

    Without that synchronization it is theoretically possible that
    the limits_changed update in sugov_should_update_freq() will be
    reordered with respect to the reads of the policy limits in
    cpufreq_driver_resolve_freq() and in that case, if the limits_changed
    update in sugov_limits() clobbers the one in sugov_should_update_freq(),
    the new policy limits may not take effect for a long time.

    Likewise, the limits_changed update in sugov_limits() may theoretically
    get reordered with respect to the updates of the policy limits in
    cpufreq_set_policy() and if sugov_should_update_freq() runs between
    them, the policy limits change may be missed.

    To ensure that the above situations will not take place, add memory
    barriers preventing the reordering in question from taking place and
    add READ_ONCE() and WRITE_ONCE() annotations around all of the
    limits_changed flag updates to prevent the compiler from messing up
    with that code.

    Fixes: 600f5ba ("cpufreq: schedutil: Don't skip freq update when limits change")
    Cc: 5.3+ <stable@vger.kernel.org> # 5.3+
    Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
    Reviewed-by: Christian Loehle <christian.loehle@arm.com>
    Link: https://patch.msgid.link/3376719.44csPzL39Z@rjwysocki.net

Signed-off-by: Phil Auld <pauld@redhat.com>

Author: auldp

kernel/sched/cpufreq_schedutil.c

@@ -81,9 +81,20 @@ static bool sugov_should_update_freq(struct sugov_policy *sg_policy, u64 time)
 	if (!cpufreq_this_cpu_can_update(sg_policy->policy))
 		return false;
 
-	if (unlikely(sg_policy->limits_changed)) {
-		sg_policy->limits_changed = false;
+	if (unlikely(READ_ONCE(sg_policy->limits_changed))) {
+		WRITE_ONCE(sg_policy->limits_changed, false);
 		sg_policy->need_freq_update = true;
+
+		/*
+		 * The above limits_changed update must occur before the reads
+		 * of policy limits in cpufreq_driver_resolve_freq() or a policy
+		 * limits update might be missed, so use a memory barrier to
+		 * ensure it.
+		 *
+		 * This pairs with the write memory barrier in sugov_limits().
+		 */
+		smp_mb();
+
 		return true;
 	}
 
@@ -356,7 +367,7 @@ static inline bool sugov_cpu_is_busy(struct sugov_cpu *sg_cpu) { return false; }
 static inline void ignore_dl_rate_limit(struct sugov_cpu *sg_cpu)
 {
 	if (cpu_bw_dl(cpu_rq(sg_cpu->cpu)) > sg_cpu->bw_min)
-		sg_cpu->sg_policy->limits_changed = true;
+		WRITE_ONCE(sg_cpu->sg_policy->limits_changed, true);
 }
 
 static inline bool sugov_update_single_common(struct sugov_cpu *sg_cpu,
@@ -892,7 +903,16 @@ static void sugov_limits(struct cpufreq_policy *policy)
 		mutex_unlock(&sg_policy->work_lock);
 	}
 
-	sg_policy->limits_changed = true;
+	/*
+	 * The limits_changed update below must take place before the updates
+	 * of policy limits in cpufreq_set_policy() or a policy limits update
+	 * might be missed, so use a memory barrier to ensure it.
+	 *
+	 * This pairs with the memory barrier in sugov_should_update_freq().
+	 */
+	smp_wmb();
+
+	WRITE_ONCE(sg_policy->limits_changed, true);
 }
 
 struct cpufreq_governor schedutil_gov = {