Skip to content

Commit

Permalink
docs: net: document new locking reality
Browse files Browse the repository at this point in the history
Also clarify ndo_get_stats (that read and write paths can run
concurrently) and mention only RCU.

Cc: Saeed Mahameed <saeed@kernel.org>
Signed-off-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20250305163732.2766420-14-sdf@fomichev.me
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
  • Loading branch information
Stanislav Fomichev authored and Jakub Kicinski committed Mar 6, 2025
1 parent 605ef7a commit cc34acd
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 13 deletions.
65 changes: 52 additions & 13 deletions Documentation/networking/netdevices.rst
Original file line number Diff line number Diff line change
Expand Up @@ -210,49 +210,55 @@ packets is preferred.
struct net_device synchronization rules
=======================================
ndo_open:
Synchronization: rtnl_lock() semaphore.
Synchronization: rtnl_lock() semaphore. In addition, netdev instance
lock if the driver implements queue management or shaper API.
Context: process

ndo_stop:
Synchronization: rtnl_lock() semaphore.
Synchronization: rtnl_lock() semaphore. In addition, netdev instance
lock if the driver implements queue management or shaper API.
Context: process
Note: netif_running() is guaranteed false

ndo_do_ioctl:
Synchronization: rtnl_lock() semaphore.
Context: process

This is only called by network subsystems internally,
not by user space calling ioctl as it was in before
linux-5.14.
This is only called by network subsystems internally,
not by user space calling ioctl as it was in before
linux-5.14.

ndo_siocbond:
Synchronization: rtnl_lock() semaphore.
Synchronization: rtnl_lock() semaphore. In addition, netdev instance
lock if the driver implements queue management or shaper API.
Context: process

Used by the bonding driver for the SIOCBOND family of
ioctl commands.
Used by the bonding driver for the SIOCBOND family of
ioctl commands.

ndo_siocwandev:
Synchronization: rtnl_lock() semaphore.
Synchronization: rtnl_lock() semaphore. In addition, netdev instance
lock if the driver implements queue management or shaper API.
Context: process

Used by the drivers/net/wan framework to handle
the SIOCWANDEV ioctl with the if_settings structure.

ndo_siocdevprivate:
Synchronization: rtnl_lock() semaphore.
Synchronization: rtnl_lock() semaphore. In addition, netdev instance
lock if the driver implements queue management or shaper API.
Context: process

This is used to implement SIOCDEVPRIVATE ioctl helpers.
These should not be added to new drivers, so don't use.

ndo_eth_ioctl:
Synchronization: rtnl_lock() semaphore.
Synchronization: rtnl_lock() semaphore. In addition, netdev instance
lock if the driver implements queue management or shaper API.
Context: process

ndo_get_stats:
Synchronization: rtnl_lock() semaphore, or RCU.
Synchronization: RCU (can be called concurrently with the stats
update path).
Context: atomic (can't sleep under RCU)

ndo_start_xmit:
Expand Down Expand Up @@ -284,6 +290,10 @@ ndo_set_rx_mode:
Synchronization: netif_addr_lock spinlock.
Context: BHs disabled

Most ndo callbacks not specified in the list above are running
under ``rtnl_lock``. In addition, netdev instance lock is taken as well if
the driver implements queue management or shaper API.

struct napi_struct synchronization rules
========================================
napi->poll:
Expand All @@ -298,6 +308,35 @@ napi->poll:
softirq
will be called with interrupts disabled by netconsole.

struct netdev_queue_mgmt_ops synchronization rules
==================================================

All queue management ndo callbacks are holding netdev instance lock.

RTNL and netdev instance lock
=============================

Historically, all networking control operations were protected by a single
global lock known as ``rtnl_lock``. There is an ongoing effort to replace this
global lock with separate locks for each network namespace. Additionally,
properties of individual netdev are increasingly protected by per-netdev locks.

For device drivers that implement shaping or queue management APIs, all control
operations will be performed under the netdev instance lock. Currently, this
instance lock is acquired within the context of ``rtnl_lock``. The drivers
can also explicitly request instance lock to be acquired via
``request_ops_lock``. In the future, there will be an option for individual
drivers to opt out of using ``rtnl_lock`` and instead perform their control
operations directly under the netdev instance lock.

Devices drivers are encouraged to rely on the instance lock where possible.

For the (mostly software) drivers that need to interact with the core stack,
there are two sets of interfaces: ``dev_xxx`` and ``netif_xxx`` (e.g.,
``dev_set_mtu`` and ``netif_set_mtu``). The ``dev_xxx`` functions handle
acquiring the instance lock themselves, while the ``netif_xxx`` functions
assume that the driver has already acquired the instance lock.

NETDEV_INTERNAL symbol namespace
================================

Expand Down
4 changes: 4 additions & 0 deletions include/linux/netdevice.h
Original file line number Diff line number Diff line change
Expand Up @@ -2505,6 +2505,10 @@ struct net_device {
*
* Also protects some fields in struct napi_struct.
*
* For the drivers that implement shaper or queue API, the scope
* of this lock is expanded to cover most ndo/queue/ethtool/sysfs
* operations.
*
* Ordering: take after rtnl_lock.
*/
struct mutex lock;
Expand Down

0 comments on commit cc34acd

Please sign in to comment.