bpf, mprog: Defer RCU synchronization until peer reuse

bpf_mprog_attach() swaps the active entry and the caller then waits
for a grace period (tcx_entry_sync() / netkit_entry_sync()) before
returning. But attach releases nothing: the old entry only has to
stay intact until the next attach or detach overwrites it as the
peer. The unconditional wait buys nothing on the attach path, yet it
runs inside rtnl_lock() and serializes the whole machine's network
control plane. Creating containers in bulk pays 2N grace periods for
N devices, one first attach per direction.

Borrow the scheme used by mini_qdisc_pair_swap() in commit 267463823adb
("net: sch: eliminate unnecessary RCU waits in mini_qdisc_pair_swap()"):
record the grace-period state after the swap and only wait before the
old buffer is written again. bpf_mprog_defer_sync() records
start_poll_synchronize_rcu() on the attach swap, bpf_mprog_peer() calls
cond_synchronize_rcu() before returning the peer to be overwritten, and
a new bundle starts with get_completed_synchronize_rcu() so the first
attach never waits. The state is recorded only on the attach swap, not
in bpf_mprog_commit(): a detach or replace leaves it alone, so it does
not charge an extra grace period to the next change. This removes the
attach-path sync from both tcx and netkit with no 'created' special
case, and covers a clsact qdisc having published the entry first.

A/B on the same bpf-next base (8835113107b2), identical configs apart
from LOCALVERSION, lock/RCU debugging off, 64 vCPU KVM guest. Average
operation latency. The attach rows use 30 samples each. The detach
row measures close(link_fd), excluding the first detach in each run:
35 samples across five runs of eight links.

  scenario                 base        v3        speedup
  first link attach        14.377 ms   268.4 us    54x
  first prog attach        12.411 ms   290.0 us    43x
  clsact qdisc first       11.562 ms   116.2 us    99x
  detach after detach      6.2 ms     5.2 ms     ~1x

An attach waits only if the grace period recorded by an earlier
attach swap has not completed. Detach after detach stays at the base
cost: recording the cookie on every commit makes a detach following
another detach wait twice (N links cost 2N-1 grace periods rather
than N, measured 15.0 ms), because bpf_mprog_peer() charges the
previous commit's grace period to the next detach. Recording it only
on the attach swap avoids that.

Functional checks pass and a 300-cycle attach/detach churn under live
traffic shows no splats.

BPF CI: https://github.com/kernel-patches/bpf/pull/14821 (39 passed,
0 failed; x86_64 gcc-15/llvm-21, aarch64, s390x).

The implementation and measurement were produced with the help of an
AI coding assistant; the change was reviewed by the submitter.

Suggested-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Qi Wu <shijing.wuqi@linux.alibaba.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261116145318.34084-1-shijing.wuqi@linux.alibaba.com
3 files changed