Eric Dumazet 1c0d32fde5 net_sched: gen_estimator: complete rewrite of rate estimators ...

1) Old code was hard to maintain, due to complex lock chains.
   (We probably will be able to remove some kfree_rcu() in callers)

2) Using a single timer to update all estimators does not scale.

3) Code was buggy on 32bit kernel (WRITE_ONCE() on 64bit quantity
   is not supposed to work well)

In this rewrite :

- I removed the RB tree that had to be scanned in
  gen_estimator_active(). qdisc dumps should be much faster.

- Each estimator has its own timer.

- Estimations are maintained in net_rate_estimator structure,
  instead of dirtying the qdisc. Minor, but part of the simplification.

- Reading the estimator uses RCU and a seqcount to provide proper
  support for 32bit kernels.

- We reduce memory need when estimators are not used, since
  we store a pointer, instead of the bytes/packets counters.

- xt_rateest_mt() no longer has to grab a spinlock.
  (In the future, xt_rateest_tg() could be switched to per cpu counters)

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

2016-12-05 15:21:59 -05:00

..

ipset

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

ipvs

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

core.c

netfilter: handle NF_REPEAT from nf_conntrack_in()

2016-11-03 11:53:00 +01:00

Kconfig

netfilter: nf_tables: introduce routing expression

2016-11-01 20:50:31 +01:00

Makefile

netfilter: nf_tables: introduce routing expression

2016-11-01 20:50:31 +01:00

nf_conntrack_acct.c

netfilter: Remove uses of seq_<foo> return values

2015-03-18 10:51:35 +01:00

nf_conntrack_amanda.c

net: Remove state argument from skb_find_text()

2015-02-22 15:59:54 -05:00

nf_conntrack_broadcast.c

…

nf_conntrack_core.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2016-11-15 10:54:36 -05:00

nf_conntrack_ecache.c

netfilter: don't rely on DYING bit to detect when destroy event was sent

2016-08-30 11:43:08 +02:00

nf_conntrack_expect.c

netfilter: nf_ct_expect: remove the redundant slash when policy name is empty

2016-08-09 10:38:46 +02:00

nf_conntrack_extend.c

netfilter: move nat hlist_head to nf_conn

2016-07-11 11:47:50 +02:00

nf_conntrack_ftp.c

netfilter: ftp: Remove the useless code

2016-09-07 10:38:00 +02:00

nf_conntrack_h323_asn1.c

netfilter: nf_conntrack_h323: fix off-by-one in DecodeQ931

2016-07-11 12:32:45 +02:00

nf_conntrack_h323_main.c

netfilter: Remove explicit rcu_read_lock in nf_hook_slow

2016-09-24 21:29:53 +02:00

nf_conntrack_h323_types.c

…

nf_conntrack_helper.c

netfilter: conntrack: fix CT target for UNSPEC helpers

2016-11-08 23:53:37 +01:00

nf_conntrack_irc.c

netfilter: Add helper array register/unregister functions

2016-07-21 02:31:53 +02:00

nf_conntrack_l3proto_generic.c

netfilter: Convert print_tuple functions to return void

2014-11-05 14:10:33 -05:00

nf_conntrack_labels.c

netfilter: connlabels: move set helper to xt_connlabel

2016-07-22 17:05:10 +02:00

nf_conntrack_netbios_ns.c

…

nf_conntrack_netlink.c

netfilter: conntrack: remove packet hotpath stats

2016-09-12 19:59:39 +02:00

nf_conntrack_pptp.c

netfilter: conntrack: get rid of conntrack timer

2016-08-30 11:43:09 +02:00

nf_conntrack_proto_dccp.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nf_conntrack_proto_generic.c

netfilter: remove ip_conntrack* sysctl compat code

2016-08-13 13:27:13 +02:00

nf_conntrack_proto_gre.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nf_conntrack_proto_sctp.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nf_conntrack_proto_tcp.c

netfilter: remove ip_conntrack* sysctl compat code

2016-08-13 13:27:13 +02:00

nf_conntrack_proto_udp.c

netfilter: remove ip_conntrack* sysctl compat code

2016-08-13 13:27:13 +02:00

nf_conntrack_proto_udplite.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nf_conntrack_proto.c

netfilter: conntrack: simplify init/uninit of L4 protocol trackers

2016-11-09 23:49:25 +01:00

nf_conntrack_sane.c

netfilter: Add helper array register/unregister functions

2016-07-21 02:31:53 +02:00

nf_conntrack_seqadj.c

netfilter: seqadj: Fix the wrong ack adjust for the RST packet without ack

2016-09-25 14:54:01 +02:00

nf_conntrack_sip.c

netfilter: nf_conntrack_sip: extend request line validation

2016-10-27 18:27:59 +02:00

nf_conntrack_snmp.c

…

nf_conntrack_standalone.c

netfilter: evict stale entries when user reads /proc/net/nf_conntrack

2016-09-25 14:54:08 +02:00

nf_conntrack_tftp.c

netfilter: Add helper array register/unregister functions

2016-07-21 02:31:53 +02:00

nf_conntrack_timeout.c

netfilter: cttimeout: add netns support

2015-12-14 12:48:58 +01:00

nf_conntrack_timestamp.c

…

nf_dup_netdev.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nf_internals.h

netfilter: merge nf_iterate() into nf_hook_slow()

2016-11-03 11:52:59 +01:00

nf_log_common.c

netfilter: nf_log: add packet logging for netdev family

2016-11-01 20:50:30 +01:00

nf_log_netdev.c

netfilter: nf_log: add packet logging for netdev family

2016-11-01 20:50:30 +01:00

nf_log.c

netfilter: fix namespace handling in nf_log_proc_dostring

2016-10-04 08:41:06 +02:00

nf_nat_amanda.c

…

nf_nat_core.c

netfilter: nat: switch to new rhlist interface

2016-11-24 14:43:34 +01:00

nf_nat_ftp.c

…

nf_nat_helper.c

…

nf_nat_irc.c

…

nf_nat_proto_common.c

…

nf_nat_proto_dccp.c

net: Change pseudohdr argument of inet_proto_csum_replace* to be a bool

2015-08-17 21:33:06 -07:00

nf_nat_proto_sctp.c

…

nf_nat_proto_tcp.c

net: Change pseudohdr argument of inet_proto_csum_replace* to be a bool

2015-08-17 21:33:06 -07:00

nf_nat_proto_udp.c

net: Change pseudohdr argument of inet_proto_csum_replace* to be a bool

2015-08-17 21:33:06 -07:00

nf_nat_proto_udplite.c

net: Change pseudohdr argument of inet_proto_csum_replace* to be a bool

2015-08-17 21:33:06 -07:00

nf_nat_proto_unknown.c

…

nf_nat_redirect.c

netfilter: nf_nat_redirect: add missing NULL pointer check

2015-10-27 06:54:56 +01:00

nf_nat_sip.c

netfilter: replace strnicmp with strncasecmp

2014-10-14 02:18:24 +02:00

nf_nat_tftp.c

…

nf_queue.c

netfilter: merge nf_iterate() into nf_hook_slow()

2016-11-03 11:52:59 +01:00

nf_sockopt.c

netfilter: don't use mutex_lock_interruptible()

2014-08-08 16:47:23 +02:00

nf_synproxy_core.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nf_tables_api.c

netfilter: nf_tables: fix inconsistent element expiration calculation

2016-11-24 14:43:34 +01:00

nf_tables_core.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nf_tables_inet.c

netfilter: Add the missed return value check of nft_register_chain_type

2016-09-12 19:54:45 +02:00

nf_tables_netdev.c

netfilter: Add the missed return value check of nft_register_chain_type

2016-09-12 19:54:45 +02:00

nf_tables_trace.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nfnetlink_acct.c

netfilter: nfnetlink: use list_for_each_entry_safe to delete all objects

2016-08-25 13:11:00 +02:00

nfnetlink_cthelper.c

netfilter: Remove explicit rcu_read_lock in nf_hook_slow

2016-09-24 21:29:53 +02:00

nfnetlink_cttimeout.c

netfilter: cttimeout: unlink timeout objs in the unconfirmed ct lists

2016-08-25 13:11:30 +02:00

nfnetlink_log.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nfnetlink_queue.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

nfnetlink.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2016-02-23 00:09:14 -05:00

nft_bitwise.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nft_byteorder.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nft_cmp.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nft_compat.c

netfilter: nft_compat: fix crash when related match/target module is removed

2016-07-23 12:25:00 +02:00

nft_counter.c

libnl: nla_put_be64(): align on a 64-bit area

2016-04-23 20:13:24 -04:00

nft_ct.c

netfilter: nft_ct: add notrack support

2016-10-26 16:35:16 +02:00

nft_dup_netdev.c

netfilter: nf_tables: add packet duplication to the netdev family

2016-01-03 21:04:23 +01:00

nft_dynset.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2016-11-15 10:54:36 -05:00

nft_exthdr.c

netfilter: nft_exthdr: fix error handling in nft_exthdr_init()

2016-10-17 17:43:54 +02:00

nft_fib_inet.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_fib.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_fwd_netdev.c

netfilter: nf_tables: add forward expression to the netdev family

2016-01-04 17:48:38 +01:00

nft_hash.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2016-12-03 12:29:53 -05:00

nft_immediate.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nft_limit.c

netfilter: nft_limit: fix divided by zero panic

2016-10-04 08:59:03 +02:00

nft_log.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_lookup.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nft_masq.c

netfilter: nft_masq: support port range

2016-03-02 20:05:27 +01:00

nft_meta.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_nat.c

netfilter: nf_tables: switch registers to 32 bit addressing

2015-04-13 17:17:29 +02:00

nft_numgen.c

netfilter: nft_numgen: start round robin from zero

2016-10-26 16:35:16 +02:00

nft_payload.c

netfilter: nf_tables: simplify the basic expressions' init routine

2016-11-09 23:42:23 +01:00

nft_queue.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_quota.c

netfilter: nft_quota: introduce nft_overquota()

2016-09-07 11:02:06 +02:00

nft_range.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2016-12-03 12:29:53 -05:00

nft_redir.c

netfilter: nf_tables: add register parsing/dumping helpers

2015-04-13 17:17:28 +02:00

nft_reject_inet.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_reject.c

netfilter: nft_reject: restrict to INPUT/FORWARD/OUTPUT

2016-08-25 12:55:34 +02:00

nft_rt.c

netfilter: nf_tables: use hook state from xt_action_param structure

2016-11-03 11:52:34 +01:00

nft_set_hash.c

netfilter: nf_tables: fix race when create new element in dynset

2016-10-27 18:22:02 +02:00

nft_set_rbtree.c

netfilter: nf_tables: fix *leak* when expr clone fail

2016-10-27 18:20:45 +02:00

x_tables.c

netfilter: x_tables: simplify IS_ERR_OR_NULL to NULL test

2016-11-13 22:26:13 +01:00

xt_addrtype.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_AUDIT.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_bpf.c

net: filter: split 'struct sk_filter' into socket and bpf parts

2014-08-02 15:03:58 -07:00

xt_cgroup.c

netfilter: implement xt_cgroup cgroup2 path match

2015-12-14 20:34:55 +01:00

xt_CHECKSUM.c

…

xt_CLASSIFY.c

…

xt_cluster.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_comment.c

…

xt_connbytes.c

netfilter: Convert pr_warning to pr_warn

2014-09-10 12:40:10 -07:00

xt_connlabel.c

netfilter: connlabels: move set helper to xt_connlabel

2016-07-22 17:05:10 +02:00

xt_connlimit.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_connmark.c

netfilter: connmark: ignore skbs with magic untracked conntrack objects

2016-11-08 23:53:36 +01:00

xt_CONNSECMARK.c

…

xt_conntrack.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_cpu.c

…

xt_CT.c

netfilter: cttimeout: add netns support

2015-12-14 12:48:58 +01:00

xt_dccp.c

…

xt_devgroup.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_dscp.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_DSCP.c

netfilter: fix various sparse warnings

2014-11-13 12:14:42 +01:00

xt_ecn.c

…

xt_esp.c

…

xt_hashlimit.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

xt_helper.c

netfilter: Remove explicit rcu_read_lock in nf_hook_slow

2016-09-24 21:29:53 +02:00

xt_hl.c

…

xt_HL.c

…

xt_HMARK.c

net: use reciprocal_scale() helper

2014-08-23 12:21:21 -07:00

xt_IDLETIMER.c

netfilter: IDLETIMER: fix race condition when destroy the target

2016-04-29 14:28:48 +02:00

xt_ipcomp.c

netfilter: xt_ipcomp: add "ip[6]t_ipcomp" module alias name

2016-10-17 17:38:19 +02:00

xt_iprange.c

…

xt_ipvs.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_l2tp.c

…

xt_LED.c

Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net

2014-08-05 18:46:26 -07:00

xt_length.c

…

xt_limit.c

…

xt_LOG.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_mac.c

…

xt_mark.c

netfilter: xt_MARK: Add ARP support

2015-05-14 13:00:27 +02:00

xt_multiport.c

netfilter: xt_multiport: Use switch case instead of multiple condition checks

2016-10-26 16:35:15 +02:00

xt_nat.c

…

xt_NETMAP.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_nfacct.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_NFLOG.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_NFQUEUE.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_osf.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_owner.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_physdev.c

netfilter: physdev: add missed blank

2016-08-12 00:42:14 +02:00

xt_pkttype.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_policy.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_quota.c

…

xt_rateest.c

net_sched: gen_estimator: complete rewrite of rate estimators

2016-12-05 15:21:59 -05:00

xt_RATEEST.c

net_sched: gen_estimator: complete rewrite of rate estimators

2016-12-05 15:21:59 -05:00

xt_realm.c

…

xt_recent.c

netns: make struct pernet_operations::id unsigned int

2016-11-18 10:59:15 -05:00

xt_REDIRECT.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_repldata.h

…

xt_sctp.c

sctp: rename WORD_TRUNC/ROUND macros

2016-09-22 03:13:26 -04:00

xt_SECMARK.c

…

xt_set.c

netfilter: ipset: Improve skbinfo get/init helpers

2016-11-10 13:28:42 +01:00

xt_socket.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_state.c

…

xt_statistic.c

…

xt_string.c

net: Remove state argument from skb_find_text()

2015-02-22 15:59:54 -05:00

xt_tcpmss.c

…

xt_TCPMSS.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_TCPOPTSTRIP.c

net: Change pseudohdr argument of inet_proto_csum_replace* to be a bool

2015-08-17 21:33:06 -07:00

xt_tcpudp.c

netfilter: Convert FWINV<[foo]> macros and uses to NF_INVF

2016-07-03 10:55:07 +02:00

xt_TEE.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_time.c

…

xt_TPROXY.c

netfilter: x_tables: move hook state into xt_action_param structure

2016-11-03 10:56:21 +01:00

xt_TRACE.c

netfilter: xt_TRACE: add explicitly nf_logger_find_get call

2016-06-23 13:26:49 +02:00

xt_u32.c

…