Yan Zhai c3e54a754b bpf: report RCU QS in cpumap kthread ...

[ Upstream commit 00bf631224 ]

When there are heavy load, cpumap kernel threads can be busy polling
packets from redirect queues and block out RCU tasks from reaching
quiescent states. It is insufficient to just call cond_resched() in such
context. Periodically raise a consolidated RCU QS before cond_resched
fixes the problem.

Fixes: 6710e11269 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP")
Reviewed-by: Jesper Dangaard Brouer <hawk@kernel.org>
Signed-off-by: Yan Zhai <yan@cloudflare.com>
Acked-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Jesper Dangaard Brouer <hawk@kernel.org>
Link: https://lore.kernel.org/r/c17b9f1517e19d813da3ede5ed33ee18496bb5d8.1710877680.git.yan@cloudflare.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>

2024-03-26 18:18:59 -04:00

..

preload

bpf: make preloaded map iterators to display map elements count

2023-07-06 12:42:25 -07:00

arraymap.c

bpf: Set need_defer as false when clearing fd array during map free

2024-02-05 20:16:51 +00:00

bloom_filter.c

bpf: Centralize permissions checks for all BPF map types

2023-06-19 14:04:04 +02:00

bpf_cgrp_storage.c

bpf: Teach verifier that certain helpers accept NULL pointer.

2023-04-04 16:57:16 -07:00

bpf_inode_storage.c

Merge tag 'net-next-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next

2023-04-26 16:07:23 -07:00

bpf_iter.c

bpf: Add __bpf_kfunc_{start,end}_defs macros

2023-11-01 22:33:53 -07:00

bpf_local_storage.c

bpf: bpf_sk_storage: Fix the missing uncharge in sk_omem_alloc

2023-09-06 11:08:14 +02:00

bpf_lru_list.c

bpf: Address KCSAN report on bpf_lru_list

2023-05-12 12:01:03 -07:00

bpf_lru_list.h

bpf: lru: Remove unused declaration bpf_lru_promote()

2023-08-08 17:21:42 -07:00

bpf_lsm.c

bpf: Fix the kernel crash caused by bpf_setsockopt().

2023-01-26 23:26:40 -08:00

bpf_struct_ops_types.h

bpf: Add dummy BPF STRUCT_OPS for test purpose

2021-11-01 14:10:00 -07:00

bpf_struct_ops.c

bpf: Charge modmem for struct_ops trampoline

2023-09-14 15:30:45 -07:00

bpf_task_storage.c

bpf: Teach verifier that certain helpers accept NULL pointer.

2023-04-04 16:57:16 -07:00

btf.c

bpf: Add bpf_sock_addr_set_sun_path() to allow writing unix sockaddr from bpf

2023-10-11 16:29:25 -07:00

cgroup_iter.c

bpf: Let verifier consider {task,cgroup} is trusted in bpf_iter_reg

2023-11-07 15:24:25 -08:00

cgroup.c

Merge tag 'for-6.7/io_uring-sockopt-2023-10-30' of git://git.kernel.dk/linux

2023-11-01 11:16:34 -10:00

core.c

bpf: hardcode BPF_PROG_PACK_SIZE to 2MB * num_possible_nodes()

2024-03-26 18:18:27 -04:00

cpumap.c

bpf: report RCU QS in cpumap kthread

2024-03-26 18:18:59 -04:00

cpumask.c

bpf: Add __bpf_kfunc_{start,end}_defs macros

2023-11-01 22:33:53 -07:00

devmap.c

bpf: Fix DEVMAP_HASH overflow check on 32-bit arches

2024-03-26 18:18:24 -04:00

disasm.c

bpf: change bpf_alu_sign_string and bpf_movsx_string to static

2023-08-04 16:15:50 -07:00

disasm.h

bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause

2021-09-02 14:49:23 +02:00

dispatcher.c

bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func

2022-12-14 12:02:14 -08:00

hashtab.c

bpf: Fix hashtab overflow check on 32-bit arches

2024-03-26 18:18:24 -04:00

helpers.c

bpf: Mark bpf_spin_{lock,unlock}() helpers with notrace correctly

2024-03-26 18:18:13 -04:00

inode.c

bpf: convert to new timestamp accessors

2023-10-18 14:08:30 +02:00

Kconfig

bpf: Add fd-based tcx multi-prog infra with link support

2023-07-19 10:07:27 -07:00

link_iter.c

bpf: Add bpf_link iterator

2022-05-10 11:20:45 -07:00

local_storage.c

Merge tag 'cgroup-for-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup

2023-04-29 10:05:22 -07:00

log.c

bpf: drop unnecessary user-triggerable WARN_ONCE in verifierl log

2023-05-16 22:34:50 -07:00

lpm_trie.c

bpf, lpm: Fix check prefixlen before walking trie

2024-01-25 15:44:45 -08:00

Makefile

bpf: Add fd-based tcx multi-prog infra with link support

2023-07-19 10:07:27 -07:00

map_in_map.c

bpf: Defer the free of inner map when necessary

2024-01-25 15:44:48 -08:00

map_in_map.h

bpf: Add map and need_defer parameters to .map_fd_put_ptr()

2024-01-25 15:44:48 -08:00

map_iter.c

bpf: Add __bpf_kfunc_{start,end}_defs macros

2023-11-01 22:33:53 -07:00

memalloc.c

bpf: Use c->unit_size to select target cache during free

2024-01-25 15:44:54 -08:00

mmap_unlock_work.h

bpf: Introduce helper bpf_find_vma

2021-11-07 11:54:51 -08:00

mprog.c

bpf: Handle bpf_mprog_query with NULL entry

2023-10-06 17:11:20 -07:00

net_namespace.c

net: Add includes masked by netdevice.h including uapi/bpf.h

2021-12-29 20:03:05 -08:00

offload.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net

2023-09-21 21:49:45 +02:00

percpu_freelist.c

bpf: Initialize same number of free nodes for each pcpu_freelist

2022-11-11 12:05:14 -08:00

percpu_freelist.h

bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI

2020-10-06 00:04:11 +02:00

prog_iter.c

bpf: Refactor bpf_iter_reg to have separate seq_info member

2020-07-25 20:16:32 -07:00

queue_stack_maps.c

bpf: Avoid deadlock when using queue and stack maps from NMI

2023-09-11 19:04:49 -07:00

reuseport_array.c

bpf: Centralize permissions checks for all BPF map types

2023-06-19 14:04:04 +02:00

ringbuf.c

bpf: Fold smp_mb__before_atomic() into atomic_set_release()

2023-10-24 14:26:07 +02:00

stackmap.c

bpf: Fix stackmap overflow check on 32-bit arches

2024-03-26 18:18:24 -04:00

syscall.c

bpf: Set uattr->batch.count as zero before batched update or deletion

2024-02-05 20:16:52 +00:00

sysfs_btf.c

bpf: Load and verify kernel module BTFs

2020-11-10 15:25:53 -08:00

task_iter.c

bpf: Let verifier consider {task,cgroup} is trusted in bpf_iter_reg

2023-11-07 15:24:25 -08:00

tcx.c

bpf, tcx: Get rid of tcx_link_const

2023-10-23 15:01:53 -07:00

tnum.c

bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul

2021-06-01 13:34:15 +02:00

trampoline.c

bpf, x64: Fix tailcall infinite loop

2023-09-12 13:06:12 -07:00

verifier.c

bpf: Fix warning for bpf_cpumask in verifier

2024-03-26 18:17:52 -04:00