Paolo Abeni
c353e8983e
net: introduce per netns packet chains
...
Currently network taps unbound to any interface are linked in the
global ptype_all list, affecting the performance in all the network
namespaces.
Add per netns ptypes chains, so that in the mentioned case only
the netns owning the packet socket(s) is affected.
While at that drop the global ptype_all list: no in kernel user
registers a tap on "any" type without specifying either the target
device or the target namespace (and IMHO doing that would not make
any sense).
Note that this adds a conditional in the fast path (to check for
per netns ptype_specific list) and increases the dataset size by
a cacheline (owing the per netns lists).
Reviewed-by: Sabrina Dubroca <sd@queasysnail.net >
Signed-off-by: Paolo Abeni <pabeni@redhat.com >
Reviewed-by: Eric Dumazet <edumaze@google.com >
Link: https://patch.msgid.link/ae405f98875ee87f8150c460ad162de7e466f8a7.1742494826.git.pabeni@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2025-03-24 13:58:22 -07:00
Eric Dumazet
c204fef97e
net: move sysctl_mem_pcpu_rsv to net_hotdata
...
sysctl_mem_pcpu_rsv is used in TCP fast path,
move it to net_hodata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240429134025.1233626-6-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-04-30 18:46:52 -07:00
Eric Dumazet
d480dc76d9
net: move sysctl_skb_defer_max to net_hotdata
...
sysctl_skb_defer_max is used in TCP fast path,
move it to net_hodata.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240429134025.1233626-3-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-04-30 18:46:52 -07:00
Eric Dumazet
a86a0661b8
net: move sysctl_max_skb_frags to net_hotdata
...
sysctl_max_skb_frags is used in TCP and MPTCP fast paths,
move it to net_hodata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240429134025.1233626-2-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-04-30 18:46:52 -07:00
Eric Dumazet
71c0de9bac
net: move dev_rx_weight to net_hotdata
...
dev_rx_weight is read from process_backlog().
Move it to net_hotdata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-10-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:42 -08:00
Eric Dumazet
26722dc74b
net: move dev_tx_weight to net_hotdata
...
dev_tx_weight is used in tx fast path.
Move it to net_hotdata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-9-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:42 -08:00
Eric Dumazet
edbc666cdc
net: move netdev_max_backlog to net_hotdata
...
netdev_max_backlog is used in rx fat path.
Move it to net_hodata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-6-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:42 -08:00
Eric Dumazet
0b91fa4bfb
net: move ptype_all into net_hotdata
...
ptype_all is used in rx/tx fast paths.
Move it to net_hotdata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-5-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:41 -08:00
Eric Dumazet
f59b5416c3
net: move netdev_tstamp_prequeue into net_hotdata
...
netdev_tstamp_prequeue is used in rx path.
Move it to net_hotdata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-4-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:41 -08:00
Eric Dumazet
ae6e22f7b7
net: move netdev_budget and netdev_budget to net_hotdata
...
netdev_budget and netdev_budget are used in rx path (net_rx_action())
Move them into net_hotdata for better cache locality.
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-3-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:41 -08:00
Eric Dumazet
2658b5a8a4
net: introduce struct net_hotdata
...
Instead of spreading networking critical fields
all over the places, add a custom net_hotdata
structure so that we can precisely control its layout.
In this first patch, move :
- gro_normal_batch used in rx (GRO stack)
- offload_base used in rx and tx (GRO and TSO stacks)
Signed-off-by: Eric Dumazet <edumazet@google.com >
Acked-by: Soheil Hassas Yeganeh <soheil@google.com >
Reviewed-by: David Ahern <dsahern@kernel.org >
Link: https://lore.kernel.org/r/20240306160031.874438-2-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org >
2024-03-07 21:12:41 -08:00