Merge branch 'srcu-next.2025.08.21a' into HEAD
SRCU updates:
* Create srcu_read_{,un}lock_fast_notrace()
* Add srcu_read_lock_fast_notrace() and srcu_read_unlock_fast_notrace()
* Add guards for notrace variants of SRCU-fast readers
* Document srcu_read_{,un}lock_fast() use of implicit RCU readers
* Document srcu_flip() memory-barrier D relation to SRCU-fast
* Remove preempt_disable/enable() in Tiny SRCU srcu_gp_start_if_needed()
diff --git a/Documentation/RCU/Design/Requirements/Requirements.rst b/Documentation/RCU/Design/Requirements/Requirements.rst
index b039554..f24b3c0 100644
--- a/Documentation/RCU/Design/Requirements/Requirements.rst
+++ b/Documentation/RCU/Design/Requirements/Requirements.rst
@@ -1973,9 +1973,7 @@
Note that grace period initialization (rcu_gp_init()) must carefully sequence
CPU hotplug scanning with grace period state changes. For example, the
following race could occur in rcu_gp_init() if rcu_seq_start() were to happen
-after the CPU hotplug scanning.
-
-.. code-block:: none
+after the CPU hotplug scanning::
CPU0 (rcu_gp_init) CPU1 CPU2
--------------------- ---- ----
@@ -2008,22 +2006,22 @@
kfree(r1);
r2 = *r0; // USE-AFTER-FREE!
-By incrementing gp_seq first, CPU1's RCU read-side critical section
+By incrementing ``gp_seq`` first, CPU1's RCU read-side critical section
is guaranteed to not be missed by CPU2.
-**Concurrent Quiescent State Reporting for Offline CPUs**
+Concurrent Quiescent State Reporting for Offline CPUs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RCU must ensure that CPUs going offline report quiescent states to avoid
blocking grace periods. This requires careful synchronization to handle
race conditions
-**Race condition causing Offline CPU to hang GP**
+Race condition causing Offline CPU to hang GP
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-A race between CPU offlining and new GP initialization (gp_init) may occur
-because `rcu_report_qs_rnp()` in `rcutree_report_cpu_dead()` must temporarily
-release the `rcu_node` lock to wake the RCU grace-period kthread:
-
-.. code-block:: none
+A race between CPU offlining and new GP initialization (gp_init()) may occur
+because rcu_report_qs_rnp() in rcutree_report_cpu_dead() must temporarily
+release the ``rcu_node`` lock to wake the RCU grace-period kthread::
CPU1 (going offline) CPU0 (GP kthread)
-------------------- -----------------
@@ -2044,15 +2042,14 @@
// Reacquire lock (but too late)
rnp->qsmaskinitnext &= ~mask // Finally clears bit
-Without `ofl_lock`, the new grace period includes the offline CPU and waits
+Without ``ofl_lock``, the new grace period includes the offline CPU and waits
forever for its quiescent state causing a GP hang.
-**A solution with ofl_lock**
+A solution with ofl_lock
+^^^^^^^^^^^^^^^^^^^^^^^^
-The `ofl_lock` (offline lock) prevents `rcu_gp_init()` from running during
-the vulnerable window when `rcu_report_qs_rnp()` has released `rnp->lock`:
-
-.. code-block:: none
+The ``ofl_lock`` (offline lock) prevents rcu_gp_init() from running during
+the vulnerable window when rcu_report_qs_rnp() has released ``rnp->lock``::
CPU0 (rcu_gp_init) CPU1 (rcutree_report_cpu_dead)
------------------ ------------------------------
@@ -2065,21 +2062,20 @@
arch_spin_unlock(&ofl_lock) ---> // Now CPU1 can proceed
} // But snapshot already taken
-**Another race causing GP hangs in rcu_gpu_init(): Reporting QS for Now-offline CPUs**
+Another race causing GP hangs in rcu_gpu_init(): Reporting QS for Now-offline CPUs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
After the first loop takes an atomic snapshot of online CPUs, as shown above,
-the second loop in `rcu_gp_init()` detects CPUs that went offline between
-releasing `ofl_lock` and acquiring the per-node `rnp->lock`. This detection is
-crucial because:
+the second loop in rcu_gp_init() detects CPUs that went offline between
+releasing ``ofl_lock`` and acquiring the per-node ``rnp->lock``.
+This detection is crucial because:
1. The CPU might have gone offline after the snapshot but before the second loop
2. The offline CPU cannot report its own QS if it's already dead
3. Without this detection, the grace period would wait forever for CPUs that
are now offline.
-The second loop performs this detection safely:
-
-.. code-block:: none
+The second loop performs this detection safely::
rcu_for_each_node_breadth_first(rnp) {
raw_spin_lock_irqsave_rcu_node(rnp, flags);
@@ -2093,10 +2089,10 @@
}
This approach ensures atomicity: quiescent state reporting for offline CPUs
-happens either in `rcu_gp_init()` (second loop) or in `rcutree_report_cpu_dead()`,
-never both and never neither. The `rnp->lock` held throughout the sequence
-prevents races - `rcutree_report_cpu_dead()` also acquires this lock when
-clearing `qsmaskinitnext`, ensuring mutual exclusion.
+happens either in rcu_gp_init() (second loop) or in rcutree_report_cpu_dead(),
+never both and never neither. The ``rnp->lock`` held throughout the sequence
+prevents races - rcutree_report_cpu_dead() also acquires this lock when
+clearing ``qsmaskinitnext``, ensuring mutual exclusion.
Scheduler and RCU
~~~~~~~~~~~~~~~~~
diff --git a/Documentation/RCU/RTFP.txt b/Documentation/RCU/RTFP.txt
index db8f16b3..8d4e8de 100644
--- a/Documentation/RCU/RTFP.txt
+++ b/Documentation/RCU/RTFP.txt
@@ -641,7 +641,7 @@
,Month="July"
,Year="2001"
,note="Available:
-\url{http://www.linuxsymposium.org/2001/abstracts/readcopy.php}
+\url{https://kernel.org/doc/ols/2001/read-copy.pdf}
\url{http://www.rdrop.com/users/paulmck/RCU/rclock_OLS.2001.05.01c.pdf}
[Viewed June 23, 2004]"
,annotation={
@@ -1480,7 +1480,7 @@
,Year="2006"
,pages="v2 123-138"
,note="Available:
-\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184}
+\url{https://kernel.org/doc/ols/2006/ols2006v2-pages-131-146.pdf}
\url{http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf}
[Viewed January 1, 2007]"
,annotation={
@@ -1511,7 +1511,7 @@
,Year="2006"
,pages="v2 249-254"
,note="Available:
-\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184}
+\url{https://kernel.org/doc/ols/2006/ols2006v2-pages-249-262.pdf}
[Viewed January 11, 2009]"
,annotation={
Uses RCU-protected radix tree for a lockless page cache.
diff --git a/Documentation/RCU/checklist.rst b/Documentation/RCU/checklist.rst
index 7de3e30..c9bfb2b 100644
--- a/Documentation/RCU/checklist.rst
+++ b/Documentation/RCU/checklist.rst
@@ -69,7 +69,13 @@
Explicit disabling of preemption (preempt_disable(), for example)
can serve as rcu_read_lock_sched(), but is less readable and
prevents lockdep from detecting locking issues. Acquiring a
- spinlock also enters an RCU read-side critical section.
+ raw spinlock also enters an RCU read-side critical section.
+
+ The guard(rcu)() and scoped_guard(rcu) primitives designate
+ the remainder of the current scope or the next statement,
+ respectively, as the RCU read-side critical section. Use of
+ these guards can be less error-prone than rcu_read_lock(),
+ rcu_read_unlock(), and friends.
Please note that you *cannot* rely on code known to be built
only in non-preemptible kernels. Such code can and will break,
@@ -405,9 +411,11 @@
13. Unlike most flavors of RCU, it *is* permissible to block in an
SRCU read-side critical section (demarked by srcu_read_lock()
and srcu_read_unlock()), hence the "SRCU": "sleepable RCU".
- Please note that if you don't need to sleep in read-side critical
- sections, you should be using RCU rather than SRCU, because RCU
- is almost always faster and easier to use than is SRCU.
+ As with RCU, guard(srcu)() and scoped_guard(srcu) forms are
+ available, and often provide greater ease of use. Please note
+ that if you don't need to sleep in read-side critical sections,
+ you should be using RCU rather than SRCU, because RCU is almost
+ always faster and easier to use than is SRCU.
Also unlike other forms of RCU, explicit initialization and
cleanup is required either at build time via DEFINE_SRCU()
@@ -443,10 +451,13 @@
real-time workloads than is synchronize_rcu_expedited().
It is also permissible to sleep in RCU Tasks Trace read-side
- critical section, which are delimited by rcu_read_lock_trace() and
- rcu_read_unlock_trace(). However, this is a specialized flavor
- of RCU, and you should not use it without first checking with
- its current users. In most cases, you should instead use SRCU.
+ critical section, which are delimited by rcu_read_lock_trace()
+ and rcu_read_unlock_trace(). However, this is a specialized
+ flavor of RCU, and you should not use it without first checking
+ with its current users. In most cases, you should instead
+ use SRCU. As with RCU and SRCU, guard(rcu_tasks_trace)() and
+ scoped_guard(rcu_tasks_trace) are available, and often provide
+ greater ease of use.
Note that rcu_assign_pointer() relates to SRCU just as it does to
other forms of RCU, but instead of rcu_dereference() you should
diff --git a/Documentation/RCU/index.rst b/Documentation/RCU/index.rst
index 84a7990..ef26c78 100644
--- a/Documentation/RCU/index.rst
+++ b/Documentation/RCU/index.rst
@@ -1,13 +1,13 @@
.. SPDX-License-Identifier: GPL-2.0
-.. _rcu_concepts:
+.. _rcu_handbook:
============
-RCU concepts
+RCU Handbook
============
.. toctree::
- :maxdepth: 3
+ :maxdepth: 2
checklist
lockdep
diff --git a/Documentation/RCU/torture.rst b/Documentation/RCU/torture.rst
index 4b1f99c..1ad5cc7 100644
--- a/Documentation/RCU/torture.rst
+++ b/Documentation/RCU/torture.rst
@@ -344,7 +344,7 @@
And this is why the kvm-remote.sh script exists.
-If you the following command works::
+If the following command works::
ssh system0 date
@@ -364,7 +364,7 @@
The kvm.sh ``--dryrun scenarios`` argument is useful for working out
how many scenarios may be run in one batch across a group of systems.
-You can also re-run a previous remote run in a manner similar to kvm.sh:
+You can also re-run a previous remote run in a manner similar to kvm.sh::
kvm-remote.sh "system0 system1 system2 system3 system4 system5" \
tools/testing/selftests/rcutorture/res/2022.11.03-11.26.28-remote \
diff --git a/Documentation/RCU/whatisRCU.rst b/Documentation/RCU/whatisRCU.rst
index be2eb6b..cf0b0ac 100644
--- a/Documentation/RCU/whatisRCU.rst
+++ b/Documentation/RCU/whatisRCU.rst
@@ -1021,32 +1021,41 @@
list_entry_rcu
list_entry_lockless
list_first_entry_rcu
+ list_first_or_null_rcu
+ list_tail_rcu
list_next_rcu
+ list_next_or_null_rcu
list_for_each_entry_rcu
list_for_each_entry_continue_rcu
list_for_each_entry_from_rcu
- list_first_or_null_rcu
- list_next_or_null_rcu
+ list_for_each_entry_lockless
hlist_first_rcu
hlist_next_rcu
hlist_pprev_rcu
hlist_for_each_entry_rcu
+ hlist_for_each_entry_rcu_notrace
hlist_for_each_entry_rcu_bh
hlist_for_each_entry_from_rcu
hlist_for_each_entry_continue_rcu
hlist_for_each_entry_continue_rcu_bh
hlist_nulls_first_rcu
+ hlist_nulls_next_rcu
hlist_nulls_for_each_entry_rcu
+ hlist_nulls_for_each_entry_safe
hlist_bl_first_rcu
hlist_bl_for_each_entry_rcu
RCU pointer/list update::
rcu_assign_pointer
+ rcu_replace_pointer
+ INIT_LIST_HEAD_RCU
list_add_rcu
list_add_tail_rcu
list_del_rcu
list_replace_rcu
+ list_splice_init_rcu
+ list_splice_tail_init_rcu
hlist_add_behind_rcu
hlist_add_before_rcu
hlist_add_head_rcu
@@ -1054,34 +1063,53 @@
hlist_del_rcu
hlist_del_init_rcu
hlist_replace_rcu
- list_splice_init_rcu
- list_splice_tail_init_rcu
hlist_nulls_del_init_rcu
hlist_nulls_del_rcu
hlist_nulls_add_head_rcu
+ hlist_nulls_add_tail_rcu
+ hlist_nulls_add_fake
+ hlists_swap_heads_rcu
hlist_bl_add_head_rcu
- hlist_bl_del_init_rcu
hlist_bl_del_rcu
hlist_bl_set_first_rcu
RCU::
- Critical sections Grace period Barrier
+ Critical sections Grace period Barrier
- rcu_read_lock synchronize_net rcu_barrier
- rcu_read_unlock synchronize_rcu
- rcu_dereference synchronize_rcu_expedited
- rcu_read_lock_held call_rcu
- rcu_dereference_check kfree_rcu
- rcu_dereference_protected
+ rcu_read_lock synchronize_net rcu_barrier
+ rcu_read_unlock synchronize_rcu
+ guard(rcu)() synchronize_rcu_expedited
+ scoped_guard(rcu) synchronize_rcu_mult
+ rcu_dereference call_rcu
+ rcu_dereference_check call_rcu_hurry
+ rcu_dereference_protected kfree_rcu
+ rcu_read_lock_held kvfree_rcu
+ rcu_read_lock_any_held kfree_rcu_mightsleep
+ rcu_pointer_handoff cond_synchronize_rcu
+ unrcu_pointer cond_synchronize_rcu_full
+ cond_synchronize_rcu_expedited
+ cond_synchronize_rcu_expedited_full
+ get_completed_synchronize_rcu
+ get_completed_synchronize_rcu_full
+ get_state_synchronize_rcu
+ get_state_synchronize_rcu_full
+ poll_state_synchronize_rcu
+ poll_state_synchronize_rcu_full
+ same_state_synchronize_rcu
+ same_state_synchronize_rcu_full
+ start_poll_synchronize_rcu
+ start_poll_synchronize_rcu_full
+ start_poll_synchronize_rcu_expedited
+ start_poll_synchronize_rcu_expedited_full
bh::
Critical sections Grace period Barrier
- rcu_read_lock_bh call_rcu rcu_barrier
- rcu_read_unlock_bh synchronize_rcu
- [local_bh_disable] synchronize_rcu_expedited
+ rcu_read_lock_bh [Same as RCU] [Same as RCU]
+ rcu_read_unlock_bh
+ [local_bh_disable]
[and friends]
rcu_dereference_bh
rcu_dereference_bh_check
@@ -1092,9 +1120,9 @@
Critical sections Grace period Barrier
- rcu_read_lock_sched call_rcu rcu_barrier
- rcu_read_unlock_sched synchronize_rcu
- [preempt_disable] synchronize_rcu_expedited
+ rcu_read_lock_sched [Same as RCU] [Same as RCU]
+ rcu_read_unlock_sched
+ [preempt_disable]
[and friends]
rcu_read_lock_sched_notrace
rcu_read_unlock_sched_notrace
@@ -1104,46 +1132,104 @@
rcu_read_lock_sched_held
+RCU: Initialization/cleanup/ordering::
+
+ RCU_INIT_POINTER
+ RCU_INITIALIZER
+ RCU_POINTER_INITIALIZER
+ init_rcu_head
+ destroy_rcu_head
+ init_rcu_head_on_stack
+ destroy_rcu_head_on_stack
+ SLAB_TYPESAFE_BY_RCU
+
+
+RCU: Quiescents states and control::
+
+ cond_resched_tasks_rcu_qs
+ rcu_all_qs
+ rcu_softirq_qs_periodic
+ rcu_end_inkernel_boot
+ rcu_expedite_gp
+ rcu_gp_is_expedited
+ rcu_unexpedite_gp
+ rcu_cpu_stall_reset
+ rcu_head_after_call_rcu
+ rcu_is_watching
+
+
+RCU-sync primitive::
+
+ rcu_sync_is_idle
+ rcu_sync_init
+ rcu_sync_enter
+ rcu_sync_exit
+ rcu_sync_dtor
+
+
RCU-Tasks::
- Critical sections Grace period Barrier
+ Critical sections Grace period Barrier
- N/A call_rcu_tasks rcu_barrier_tasks
+ N/A call_rcu_tasks rcu_barrier_tasks
synchronize_rcu_tasks
RCU-Tasks-Rude::
- Critical sections Grace period Barrier
+ Critical sections Grace period Barrier
- N/A N/A
- synchronize_rcu_tasks_rude
+ N/A synchronize_rcu_tasks_rude rcu_barrier_tasks_rude
+ call_rcu_tasks_rude
RCU-Tasks-Trace::
- Critical sections Grace period Barrier
+ Critical sections Grace period Barrier
- rcu_read_lock_trace call_rcu_tasks_trace rcu_barrier_tasks_trace
+ rcu_read_lock_trace call_rcu_tasks_trace rcu_barrier_tasks_trace
rcu_read_unlock_trace synchronize_rcu_tasks_trace
+ guard(rcu_tasks_trace)()
+ scoped_guard(rcu_tasks_trace)
+
+
+SRCU list traversal::
+ list_for_each_entry_srcu
+ hlist_for_each_entry_srcu
SRCU::
- Critical sections Grace period Barrier
+ Critical sections Grace period Barrier
- srcu_read_lock call_srcu srcu_barrier
- srcu_read_unlock synchronize_srcu
- srcu_dereference synchronize_srcu_expedited
+ srcu_read_lock call_srcu srcu_barrier
+ srcu_read_unlock synchronize_srcu
+ srcu_read_lock_fast synchronize_srcu_expedited
+ srcu_read_unlock_fast get_state_synchronize_srcu
+ srcu_read_lock_nmisafe start_poll_synchronize_srcu
+ srcu_read_unlock_nmisafe start_poll_synchronize_srcu_expedited
+ srcu_read_lock_notrace poll_state_synchronize_srcu
+ srcu_read_unlock_notrace
+ srcu_down_read
+ srcu_up_read
+ srcu_down_read_fast
+ srcu_up_read_fast
+ guard(srcu)()
+ scoped_guard(srcu)
+ srcu_read_lock_held
+ srcu_dereference
srcu_dereference_check
+ srcu_dereference_notrace
srcu_read_lock_held
-SRCU: Initialization/cleanup::
+
+SRCU: Initialization/cleanup/ordering::
DEFINE_SRCU
DEFINE_STATIC_SRCU
init_srcu_struct
cleanup_srcu_struct
+ smp_mb__after_srcu_read_unlock
All: lockdep-checked RCU utility APIs::
diff --git a/include/linux/list.h b/include/linux/list.h
index e7e28af..e7bdad9 100644
--- a/include/linux/list.h
+++ b/include/linux/list.h
@@ -687,16 +687,6 @@ static inline void list_splice_tail_init(struct list_head *list,
for (pos = (head)->next; !list_is_head(pos, (head)); pos = pos->next)
/**
- * list_for_each_rcu - Iterate over a list in an RCU-safe fashion
- * @pos: the &struct list_head to use as a loop cursor.
- * @head: the head for your list.
- */
-#define list_for_each_rcu(pos, head) \
- for (pos = rcu_dereference((head)->next); \
- !list_is_head(pos, (head)); \
- pos = rcu_dereference(pos->next))
-
-/**
* list_for_each_continue - continue iteration over a list
* @pos: the &struct list_head to use as a loop cursor.
* @head: the head for your list.
diff --git a/include/linux/rculist.h b/include/linux/rculist.h
index 1b11926..2abba75 100644
--- a/include/linux/rculist.h
+++ b/include/linux/rculist.h
@@ -43,6 +43,16 @@ static inline void INIT_LIST_HEAD_RCU(struct list_head *list)
#define list_bidir_prev_rcu(list) (*((struct list_head __rcu **)(&(list)->prev)))
/**
+ * list_for_each_rcu - Iterate over a list in an RCU-safe fashion
+ * @pos: the &struct list_head to use as a loop cursor.
+ * @head: the head for your list.
+ */
+#define list_for_each_rcu(pos, head) \
+ for (pos = rcu_dereference((head)->next); \
+ !list_is_head(pos, (head)); \
+ pos = rcu_dereference(pos->next))
+
+/**
* list_tail_rcu - returns the prev pointer of the head of the list
* @head: the head of the list
*
diff --git a/kernel/cgroup/dmem.c b/kernel/cgroup/dmem.c
index 10b6343..e12b946 100644
--- a/kernel/cgroup/dmem.c
+++ b/kernel/cgroup/dmem.c
@@ -14,6 +14,7 @@
#include <linux/mutex.h>
#include <linux/page_counter.h>
#include <linux/parser.h>
+#include <linux/rculist.h>
#include <linux/slab.h>
struct dmem_cgroup_region {
diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
index f924435..2dc044f 100644
--- a/kernel/rcu/tasks.h
+++ b/kernel/rcu/tasks.h
@@ -553,13 +553,13 @@ static void rcu_tasks_invoke_cbs(struct rcu_tasks *rtp, struct rcu_tasks_percpu
rtpcp_next = rtp->rtpcp_array[index];
if (rtpcp_next->cpu < smp_load_acquire(&rtp->percpu_dequeue_lim)) {
cpuwq = rcu_cpu_beenfullyonline(rtpcp_next->cpu) ? rtpcp_next->cpu : WORK_CPU_UNBOUND;
- queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work);
+ queue_work_on(cpuwq, system_percpu_wq, &rtpcp_next->rtp_work);
index++;
if (index < num_possible_cpus()) {
rtpcp_next = rtp->rtpcp_array[index];
if (rtpcp_next->cpu < smp_load_acquire(&rtp->percpu_dequeue_lim)) {
cpuwq = rcu_cpu_beenfullyonline(rtpcp_next->cpu) ? rtpcp_next->cpu : WORK_CPU_UNBOUND;
- queue_work_on(cpuwq, system_wq, &rtpcp_next->rtp_work);
+ queue_work_on(cpuwq, system_percpu_wq, &rtpcp_next->rtp_work);
}
}
}
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 8eff357..31690ff 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3800,6 +3800,11 @@ static void rcu_barrier_handler(void *cpu_in)
* to complete. For example, if there are no RCU callbacks queued anywhere
* in the system, then rcu_barrier() is within its rights to return
* immediately, without waiting for anything, much less an RCU grace period.
+ * In fact, rcu_barrier() will normally not result in any RCU grace periods
+ * beyond those that were already destined to be executed.
+ *
+ * In kernels built with CONFIG_RCU_LAZY=y, this function also hurries all
+ * pending lazy RCU callbacks.
*/
void rcu_barrier(void)
{
@@ -4885,10 +4890,10 @@ void __init rcu_init(void)
rcutree_online_cpu(cpu);
/* Create workqueue for Tree SRCU and for expedited GPs. */
- rcu_gp_wq = alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM, 0);
+ rcu_gp_wq = alloc_workqueue("rcu_gp", WQ_MEM_RECLAIM | WQ_PERCPU, 0);
WARN_ON(!rcu_gp_wq);
- sync_wq = alloc_workqueue("sync_wq", WQ_MEM_RECLAIM, 0);
+ sync_wq = alloc_workqueue("sync_wq", WQ_MEM_RECLAIM | WQ_UNBOUND, 0);
WARN_ON(!sync_wq);
/* Respect if explicitly disabled via a boot parameter. */
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 4cd170b..d857633 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -626,11 +626,10 @@ notrace void rcu_preempt_deferred_qs(struct task_struct *t)
*/
static void rcu_preempt_deferred_qs_handler(struct irq_work *iwp)
{
- unsigned long flags;
struct rcu_data *rdp;
+ lockdep_assert_irqs_disabled();
rdp = container_of(iwp, struct rcu_data, defer_qs_iw);
- local_irq_save(flags);
/*
* If the IRQ work handler happens to run in the middle of RCU read-side
@@ -647,8 +646,6 @@ static void rcu_preempt_deferred_qs_handler(struct irq_work *iwp)
*/
if (rcu_preempt_depth() > 0)
WRITE_ONCE(rdp->defer_qs_iw_pending, DEFER_QS_IDLE);
-
- local_irq_restore(flags);
}
/*
