Re: [PATCH v2 sched_ext/for-7.1] sched_ext: Invalidate dispatch decisions on CPU affinity changes

[Kuba Piecuch]

On Thu Mar 19, 2026 at 10:31 AM UTC, Kuba Piecuch wrote:
>> @@ -2537,9 +2546,26 @@ static void dispatch_to_local_dsq(struct scx_sched *sch, struct rq *rq,
>>  	}
>>  
>>  	if (src_rq != dst_rq &&
>> -	    unlikely(!task_can_run_on_remote_rq(sch, p, dst_rq, true))) {
>> -		dispatch_enqueue(sch, rq, find_global_dsq(sch, task_cpu(p)), p,
>> -				 enq_flags | SCX_ENQ_CLEAR_OPSS | SCX_ENQ_GDSQ_FALLBACK);
>> +	    unlikely(!task_can_run_on_remote_rq(sch, p, dst_rq, false))) {
>> +		/*
>> +		 * Affinity changed after dispatch decision and the task
>> +		 * can't run anymore on the destination rq.
>> +		 *
>> +		 * Drop the dispatch, the task will be re-enqueued. Set the
>> +		 * task back to QUEUED so dequeue (if waiting) can proceed
>> +		 * using current qseq from the task's rq.
>> +		 */
>> +		if (src_rq != rq) {
>> +			raw_spin_rq_unlock(rq);
>> +			raw_spin_rq_lock(src_rq);
>> +		}
>> +		atomic_long_set_release(&p->scx.ops_state,
>> +			       SCX_OPSS_QUEUED |
>> +			       (src_rq->scx.ops_qseq << SCX_OPSS_QSEQ_SHIFT));
>> +		if (src_rq != rq) {
>> +			raw_spin_rq_unlock(src_rq);
>> +			raw_spin_rq_lock(rq);
>> +		}
>>  		return;
>>  	}
>
> My understanding is that task_can_run_on_remote_rq() can run without src_rq
> locked, so it's possible that @p's cpumask changes after the check, isn't it?
> In that case, I think it's still possible to move the task to the local DSQ
> of a CPU that is outside of its cpumask, triggering a warning in
> move_remote_task_to_local_dsq().

I've looked at the code more carefully and I don't think this is an issue.
It's true that task_can_run_on_remote_rq() can run without src_rq locked, and
it's possible that the cpumask changes after the check, but then the dequeue
preceding the cpumask change must have waited for ops_state to change from
SCX_OPSS_DISPATCHING to SCX_OPSS_NONE and it must have reset holding_cpu to -1,
so thanks to the holding_cpu check later we won't insert the task into the DSQ.

Apologies for the confusion.