bpf, x86: Support BPF_REG_ARENA_SP
BPF r12 (BPF_REG_ARENA_SP) is the stack pointer of the stack in arena. It
needs an x86 register for the full run of the prog. R9 is the only x86
register that has no fixed function. Map r12 to R9.
R9 has other uses too. In a prog that uses r12 the JIT changes them:
R9 is also in a prog with r12
-------------------------------- -------------------------------------
clobbered by helpers and kfuncs store R9 in the frame before the call,
load it after the call
arg 6 of a BPF-to-BPF call arg 6 is on the stack, before arg 7
arg 6 of a kfunc load arg 6 from the stack into R9,
move rsp to arg 7 for the call. Only
for a kfunc with more than 5 args
frame pointer of private stack the private stack is off
To reject 6 arguments in such progs is not possible, because Rust
functions have them. r12 in a slot of the frame costs a load for each
access.
Only JITed code sees the arguments of a BPF-to-BPF call, because no
trampoline attaches to a subprog of a prog with r12. Thus the JIT can
select the location of arg 6.
Progs without r12 get the same JITed code. The cost in a prog with r12, in
a hot loop on x86-64 (Zen 5), in nsec:
call of a helper or a kfunc store and load of R9 0.012
function that has such a call push r9, add rsp, 8 < 0.01
BPF-to-BPF call with 6 or 7 args arg 6 is on the stack < 0.007
call of a kfunc with 10 args load of R9, moves of rsp 0.13 to 0.27
The last time depends on the address of the kfunc.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261104012401.1354861-13-alexei.starovoitov@gmail.com
1 file changed