Paul E. McKenney df5bd5144a rcu: Reduce expedited GP memory contention via per-CPU variables ...

Currently, the piggybacked-work checks carried out by sync_exp_work_done()
atomically increment a small set of variables (the ->expedited_workdone0,
->expedited_workdone1, ->expedited_workdone2, ->expedited_workdone3
fields in the rcu_state structure), which will form a memory-contention
bottleneck given a sufficiently large number of CPUs concurrently invoking
either synchronize_rcu_expedited() or synchronize_sched_expedited().

This commit therefore moves these for fields to the per-CPU rcu_data
structure, eliminating the memory contention.  The show_rcuexp() function
also changes to sum up each field in the rcu_data structures.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

2015-12-04 12:26:52 -08:00

..

bpf

bpf, verifier: annotate verbose printer with __printf

2015-11-03 11:29:56 -05:00

configs

kconfig: add xenconfig defconfig helper

2015-06-16 11:04:29 +01:00

debug

debug: prevent entering debug mode on panic/exception.

2015-02-19 12:39:03 -06:00

events

Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-11-15 09:36:24 -08:00

gcov

gcov: add support for GCC 5.1

2015-06-30 19:44:57 -07:00

irq

Merge branches 'irq-urgent-for-linus' and 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-11-15 09:30:48 -08:00

livepatch

livepatch: x86: fix relocation computation with kASLR

2015-11-11 17:36:04 +01:00

locking

mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd

2015-11-06 17:50:42 -08:00

power

mm, page_alloc: rename __GFP_WAIT to __GFP_RECLAIM

2015-11-06 17:50:42 -08:00

printk

printk: prevent userland from spoofing kernel messages

2015-11-06 17:50:42 -08:00

rcu

rcu: Reduce expedited GP memory contention via per-CPU variables

2015-12-04 12:26:52 -08:00

sched

Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-11-15 09:35:33 -08:00

time

Merge branches 'irq-urgent-for-linus' and 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-11-15 09:30:48 -08:00

trace

Merge tag 'trace-v4.4-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

2015-11-12 16:22:54 -08:00

.gitignore

certs: add .gitignore to stop git nagging about x509_certificate_list

2015-10-21 15:18:35 +01:00

acct.c

acct: check FMODE_CAN_WRITE

2015-04-11 22:27:55 -04:00

async.c

kernel/async.c: switch to pr_foo()

2014-10-09 22:26:04 -04:00

audit_fsnotify.c

audit: clean simple fsnotify implementation

2015-08-06 16:14:53 -04:00

audit_tree.c

audit: audit_tree_match can be boolean

2015-11-04 08:23:51 -05:00

audit_watch.c

Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit

2015-09-08 13:34:59 -07:00

audit.c

mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd

2015-11-06 17:50:42 -08:00

audit.h

audit: audit_tree_match can be boolean

2015-11-04 08:23:51 -05:00

auditfilter.c

audit: fix comment block whitespace

2015-11-04 08:23:51 -05:00

auditsc.c

Merge branch 'upstream' of git://git.infradead.org/users/pcmoore/audit

2015-09-08 13:34:59 -07:00

backtracetest.c

kernel/backtracetest.c: replace no level printk by pr_info()

2014-06-04 16:54:14 -07:00

bounds.c

page-cgroup: get rid of NR_PCG_FLAGS

2014-08-08 15:57:18 -07:00

capability.c

kernel: conditionally support non-root users, groups and capabilities

2015-04-15 16:35:22 -07:00

cgroup_freezer.c

cgroup: allow a cgroup subsystem to reject a fork

2015-07-14 17:29:23 -04:00

cgroup_pids.c

cgroup: add cgroup_subsys->free() method and use it to fix pids controller

2015-10-15 16:41:53 -04:00

cgroup.c

mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd

2015-11-06 17:50:42 -08:00

compat.c

compat: cleanup coding in compat_get_bitmap() and compat_put_bitmap()

2015-06-04 23:57:18 +02:00

configs.c

…

context_tracking.c

context_tracking: avoid irq_save/irq_restore on guest entry and exit

2015-11-10 12:06:23 +01:00

cpu_pm.c

kernel/cpu_pm: fix cpu_cluster_pm_exit comment

2015-09-03 02:42:20 +02:00

cpu.c

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-11-03 18:03:50 -08:00

cpuset.c

Merge branch 'akpm' (patches from Andrew)

2015-11-05 23:10:54 -08:00

crash_dump.c

crash_dump: Make is_kdump_kernel() accessible from modules

2014-08-25 15:42:19 -07:00

cred.c

kernel/cred.c: remove unnecessary kdebug atomic reads

2015-09-10 13:29:01 -07:00

delayacct.c

delayacct: Remove braindamaged type conversions

2014-07-23 10:18:06 -07:00

dma.c

…

elfcore.c

…

exec_domain.c

Remove rest of exec domains.

2015-04-12 21:03:31 +02:00

exit.c

Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-11-03 18:03:50 -08:00

extable.c

kernel/extable.c: remove duplicated include

2015-09-10 13:29:01 -07:00

fork.c

Merge branch 'akpm' (patches from Andrew)

2015-11-05 23:10:54 -08:00

freezer.c

freezer: remove obsolete comments in __thaw_task()

2014-10-21 23:44:20 +02:00

futex_compat.c

…

futex.c

Merge tag 'driver-core-4.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core

2015-11-04 21:50:37 -08:00

groups.c

kernel: conditionally support non-root users, groups and capabilities

2015-04-15 16:35:22 -07:00

hung_task.c

kernel/hung_task.c: change hung_task.c to use for_each_process_thread()

2015-04-15 16:35:22 -07:00

irq_work.c

percpu: Convert remaining __get_cpu_var uses in 3.18-rcX

2014-10-29 11:18:18 -04:00

jump_label.c

locking/static_keys: Add selftest

2015-08-03 11:34:16 +02:00

kallsyms.c

kernel/kallsyms.c: use __seq_open_private()

2014-10-14 02:18:16 +02:00

kcmp.c

kcmp: fix standard comparison bug

2014-09-10 15:42:12 -07:00

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.locks

locking/qrwlock: Rename QUEUE_RWLOCK to QUEUED_RWLOCKS

2015-05-12 09:46:00 +02:00

Kconfig.preempt

…

kexec_core.c

kexec: use file name as the output message prefix

2015-11-06 17:50:42 -08:00

kexec_file.c

kexec: use file name as the output message prefix

2015-11-06 17:50:42 -08:00

kexec_internal.h

kexec: split kexec_file syscall code to kexec_file.c

2015-09-10 13:29:01 -07:00

kexec.c

kexec: use file name as the output message prefix

2015-11-06 17:50:42 -08:00

kmod.c

kmod: don't run async usermode helper as a child of kworker thread

2015-10-23 17:55:10 +09:00

kprobes.c

perf/x86/hw_breakpoints: Disallow kernel breakpoints unless kprobe-safe

2015-08-04 10:16:54 +02:00

ksysfs.c

kexec: split kexec_load syscall from kexec core code

2015-09-10 13:29:01 -07:00

kthread.c

kernel/kthread.c:kthread_create_on_node(): clarify documentation

2015-09-04 16:54:41 -07:00

latencytop.c

kernel/latencytop.c: convert seq_printf to seq_puts

2014-06-04 16:54:15 -07:00

Makefile

sys_membarrier(): system-wide memory barrier (generic, x86)

2015-09-11 15:21:34 -07:00

membarrier.c

sys_membarrier(): system-wide memory barrier (generic, x86)

2015-09-11 15:21:34 -07:00

memremap.c

Merge tag 'libnvdimm-for-4.4' of git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm

2015-11-10 12:07:22 -08:00

module_signing.c

KEYS: Merge the type-specific data with the payload data

2015-10-21 15:18:36 +01:00

module-internal.h

…

module.c

module: Fix locking in symbol_put_addr()

2015-08-24 10:37:01 +09:30

notifier.c

Merge branch 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-09-01 08:40:25 -07:00

nsproxy.c

bury struct proc_ns in fs/proc

2014-12-04 14:34:54 -05:00

padata.c

padata: use %*pb[l] to print bitmaps including cpumasks and nodemasks

2015-02-13 21:21:38 -08:00

panic.c

kernel/panic.c: turn off locks debug before releasing console lock

2015-11-20 16:17:32 -08:00

params.c

Merge tag 'modules-next-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux

2015-11-09 15:53:39 -08:00

pid_namespace.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2014-12-16 15:53:03 -08:00

pid.c

rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()

2015-07-22 15:27:32 -07:00

profile.c

mm: rename alloc_pages_exact_node() to __alloc_pages_node()

2015-09-08 15:35:28 -07:00

ptrace.c

seccomp, ptrace: add support for dumping seccomp filters

2015-10-27 19:55:13 -07:00

range.c

kernel: avoid overflow in cmp_range

2015-01-17 10:02:23 +13:00

reboot.c

kexec: split kexec_load syscall from kexec core code

2015-09-10 13:29:01 -07:00

relay.c

kernel/relay.c: use kvfree() in relay_free_page_array()

2015-06-30 19:44:59 -07:00

resource.c

mm: enhance region_is_ram() to region_intersects()

2015-08-10 23:07:05 -04:00

seccomp.c

seccomp, ptrace: add support for dumping seccomp filters

2015-10-27 19:55:13 -07:00

signal.c

kernel/signal.c: unexport sigsuspend()

2015-11-20 16:17:32 -08:00

smp.c

mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd

2015-11-06 17:50:42 -08:00

smpboot.c

stop_machine: Kill smp_hotplug_thread->pre_unpark, introduce stop_machine_unpark()

2015-10-20 10:23:55 +02:00

smpboot.h

…

softirq.c

Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2015-02-09 15:24:03 -08:00

stacktrace.c

stacktrace: introduce snprint_stack_trace for buffer output

2014-12-13 12:42:48 -08:00

stop_machine.c

sched: Move cpu_active() tests from stop_two_cpus() into migrate_swap_stop()

2015-10-20 10:25:56 +02:00

sys_ni.c

mm: mlock: add new mlock system call

2015-11-05 19:34:48 -08:00

sys.c

pidns: fix set/getpriority and ioprio_set/get in PRIO_USER mode

2015-11-06 17:50:42 -08:00

sysctl_binary.c

kernel: add panic_on_warn

2014-12-10 17:41:10 -08:00

sysctl.c

kernel/watchdog.c: add sysctl knob hardlockup_panic

2015-11-05 19:34:48 -08:00

task_work.c

task_work: remove fifo ordering guarantee

2015-09-05 13:46:58 -07:00

taskstats.c

netlink: make nlmsg_end() and genlmsg_end() void

2015-01-18 01:03:45 -05:00

test_kprobes.c

kernel/test_kprobes.c: use current logging functions

2014-08-08 15:57:18 -07:00

torture.c

torture: Consolidate cond_resched_rcu_qs() into stutter_wait()

2015-10-06 11:25:01 -07:00

tracepoint.c

tracepoint: Give priority to probes of tracepoints

2015-10-25 21:33:54 -04:00

tsacct.c

sched: Make task->start_time nanoseconds based

2014-07-23 10:18:05 -07:00

uid16.c

groups: Consolidate the setgroups permission checks

2014-12-05 17:19:27 -06:00

up.c

smp: Rename __smp_call_function_single() to smp_call_function_single_async()

2014-02-24 14:47:15 -08:00

user_namespace.c

capabilities: ambient capabilities

2015-09-04 16:54:41 -07:00

user-return-notifier.c

scheduler: Replace __get_cpu_var with this_cpu_ptr

2014-08-26 13:45:45 -04:00

user.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace

2014-12-17 12:31:40 -08:00

utsname_sysctl.c

sysctl: convert use of typedef ctl_table to struct ctl_table

2014-06-06 16:08:16 -07:00

utsname.c

copy address of proc_ns_ops into ns_common

2014-12-04 14:34:47 -05:00

watchdog.c

kernel/watchdog.c: fix race between proc_watchdog_thresh() and watchdog_timer_fn()

2015-11-05 19:34:48 -08:00

workqueue_internal.h

workqueue: rename manager_mutex to attach_mutex

2014-05-20 10:59:32 -04:00

workqueue.c

Merge branch 'for-4.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq

2015-11-05 14:16:27 -08:00