workqueue: Changes for v7.3

- Worker wakeups moved out of pool->lock on the hot paths, shortening lock
  hold times. The wakeup can be expensive on arm64 due to the idle wakeup
  IPI, and the change improves the workqueue microbenchmark there by up to
  10%.

- Stall diagnostics now report pools stuck with no running worker, with a
  backtrace of what the CPU is executing and the likely culprit worker,
  instead of dumping every in-flight worker.

- Preparation for turning per-cpu workqueues into an affinity scope of
  unbound workqueues instead of a separate backend.

- Race annotations for KCSAN and sparse warnings, and doc and monitoring
  script fixes.
workqueue: annotate racy p->wake_cpu accesses in kick_pool_pick()

kick_pool_pick() reads and writes p->wake_cpu while the scheduler can
update it concurrently. KCSAN reports:

  BUG: KCSAN: data-race in kick_pool_pick+0xf8/0x2d8

  race at unknown origin, with read to 0xffff000663229da4 of 4 bytes by
  task 1817002 on cpu 40:
   kick_pool_pick+0xf8/0x2d8
   process_scheduled_works+0x2bc/0x888
   worker_thread+0x394/0x548
   kthread+0x1b8/0x1f0
   ret_from_fork+0x10/0x20

  value changed: 0x0000002b -> 0x0000002f

The race is harmless. wake_cpu is a best-effort placement hint: every
writer stores a valid CPU id and the wakeup path validates it through
select_task_rq(), so a stale value only affects which CPU the worker
wakes up on.

Mark both accesses with READ_ONCE() and WRITE_ONCE() to document that
they are intentionally racy and to stop the compiler from reloading or
tearing them.

Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Tejun Heo <tj@kernel.org>
1 file changed