The ‘Bridge’ Between Linux Devices and Drivers | Bus

1. Introduction

In the Linux device driver model, a bus is an abstract concept, a special type of device. In the implementation of the device model, the kernel specifies that each device in the system must be connected to a bus, which can be an internal Bus, a virtual Bus, or a Platform bus. The bus is described in the kernel through the struct bus_type structure, defined in include/linux/device.h.

This article first describes the data structures related to the bus, focusing on the meaning of each element in the struct bus_type structure and the relationships between them. Next, it will describe the initialization process of the Linux device driver model concerning the bus, which is completed by buses_init(), and finally, it will describe several operation interface functions for the bus.

All source code analysis in this article is based on the Linux kernel version: 4.1.15.

• 1. Introduction

• 2. Data Structures Related to the Bus

• (2-1) struct bus_type

• (2-2) struct subsys_private

• 3. Bus Initialization

• 4. Bus Operation Interfaces

• (4-1) Bus Registration

• (4-2) Bus Unregistration

• (4-3) Adding device and device_driver

• (4-4) Driver Probe

2. Data Structures Related to the Bus

(2-1) struct bus_type

A bus is a channel between the processor and more devices. In the Linux device model, all devices are connected together through the bus. Buses can connect to each other; for example, a USB controller is often a PCI device, and the device model represents the actual connection between the bus and the devices it controls. The bus is represented by the struct bus_type structure, which contains the bus name, default attributes, methods of the bus, PM operations, and private data of the driver core. The struct bus_type is defined as follows:

struct bus_type {
 const char  *name;
 const char  *dev_name;
 struct device  *dev_root;
 struct device_attribute *dev_attrs; /* use dev_groups instead */
 const struct attribute_group **bus_groups;
 const struct attribute_group **dev_groups;
 const struct attribute_group **drv_groups;

 int (*match)(struct device *dev, struct device_driver *drv);
 int (*uevent)(struct device *dev, struct kobj_uevent_env *env);
 int (*probe)(struct device *dev);
 int (*remove)(struct device *dev);
 void (*shutdown)(struct device *dev);

 int (*online)(struct device *dev);
 int (*offline)(struct device *dev);

 int (*suspend)(struct device *dev, pm_message_t state);
 int (*resume)(struct device *dev);

 const struct dev_pm_ops *pm;

 const struct iommu_ops *iommu_ops;

 struct subsys_private *p;
 struct lock_class_key lock_key;
};

• name: The name of the bus.
• dev_name: Used for subsystem enumeration of devices, e.g., (“foo%u”, dev->id).
• dev_root: Represents the default device to be used for the parent device.
• dev_attrs: Device attribute group.
• bus_groups: bus attribute group.
• dev_groups: dev attribute group.
• drv_groups: drv attribute group.
• match: A callback function that needs to be implemented by the specific bus driver, which the kernel calls when all devices and drivers belonging to that bus are added to the kernel.
• uevent: Also a callback function implemented by the specific bus driver, which is called when devices belonging to that bus trigger addition, removal, or other actions, allowing the bus driver to modify environment variables.
• probe, remove: These are also callback functions. When a new device or driver is added to this bus, the kernel first calls this bus’s probe, and then calls the specific driver program’s probe to initialize the matching device; when a device is removed from this bus, remove is called, making these two callback functions very important.
• shutdown: Called when a shutdown is needed to stop the device from working. This function is related to power management.
• online: Called when the device comes back online after being offline. This function is related to power management.
• offline: Called when taking the device offline for hot-swapping.
• suspend: Called when a device on the bus wants to enter sleep mode.
• resume: Called when a device on this bus exits sleep mode.
• pm: The power management operations corresponding to this bus, which will call the specific driver program’s pm ops.
• iommu_ops: The IOMMU specific operations for this bus, used to attach IOMMU driver implementations to the bus and allow drivers to perform special setup operations on the bus.
• p: A pointer to struct subsys_private, which is the private data of the driver core, only accessible to the driver core.
• lock_key: This parameter is used for lock validation.

(2-2) struct subsys_private

Both bus_type and class structures contain a pointer to struct subsys_private to store the private data of the driver core part of bus_type and class structures. It may not be easy to understand struct subsys_private from its name, but since both bus_type and class structures have a pointer to struct subsys_private, it can be understood that subsys_private is the upper layer of bus_type and class, containing the bus and class.

The struct subsys_private structure is defined as follows (/drivers/base/base.h):

struct subsys_private {
 struct kset subsys;
 struct kset *devices_kset;
 struct list_head interfaces;
 struct mutex mutex;

 struct kset *drivers_kset;
 struct klist klist_devices;
 struct klist klist_drivers;
 struct blocking_notifier_head bus_notifier;
 unsigned int drivers_autoprobe:1;
 struct bus_type *bus;

 struct kset glue_dirs;
 struct class *class;
};

• subsys: Describes the kset of this subsystem, representing itself.
• devices_kset: Represents the device directory of the subsystem.
• interfaces: Interfaces are a list_head type data used to store related interfaces. In the kernel, interfaces are used to abstract some special functions of all associated devices under the bus.
• mutex: A mutex type lock used to protect the device and interface linked list.
• drivers_kset: Represents the driver-related linked list in the subsystem.
• klist_devices: Device linked list used to save pointers to all devices under this bus for easy lookup.
• klist_drivers: Driver linked list used to save pointers to all device_drivers under this bus for easy lookup.
• bus_notifier: A bus notifier list used to monitor any events occurring on the bus.
• drivers_autoprobe: Used to control whether drivers or devices under this bus have the auto probe attribute.
• bus: A pointer to the associated struct bus_type. Used to save the upper layer bus.
• glue_dirs: Represents glue directories to avoid namespace conflicts between parent devices.
• class: A pointer to the associated struct class. Used to save the upper layer class.

3. Bus Initialization

The bus is part of the Linux driver model, so during the kernel startup process, the driver_init() function calls buses_init() to complete the initialization operations related to the bus:

In the operation logic of buses_init(), the following tasks are completed:

(1) Dynamically create the bus kernel kset, specify its event operation functions, and then add it to sysfs.

(2) Dynamically create the system kernel kset, specifying its parent kset as devices_kset->kobj, and then add it to sysfs.

4. Bus Operation Interfaces

This section describes the API interfaces for bus operations in the Linux kernel:

// Add a device to the bus
extern int bus_add_device(struct device *dev);
// Probe driver for a new device
extern void bus_probe_device(struct device *dev);
// Remove a device from the bus
extern void bus_remove_device(struct device *dev);

// Add a driver to the bus
extern int bus_add_driver(struct device_driver *drv);
// Remove a driver from the bus
extern void bus_remove_driver(struct device_driver *drv);

// Detach the driver from all devices it controls
extern void driver_detach(struct device_driver *drv);
// Attempt to bind the device and driver together
extern int driver_probe_device(struct device_driver *drv, struct device *dev);

// Register a driver core bus subsystem
extern int __must_check bus_register(struct bus_type *bus);

// Unregister a driver core bus subsystem
extern void bus_unregister(struct bus_type *bus);

/* Register a bus notifier. The bus notifier is used to get notifications when devices are added/removed or drivers are bound/unbound with devices. */
extern int bus_register_notifier(struct bus_type *bus,
     struct notifier_block *nb);
         
// Unregister a bus notifier
extern int bus_unregister_notifier(struct bus_type *bus,
       struct notifier_block *nb);

(4-1) Bus Registration

Call bus_register to perform the specific bus registration operation, which is implemented in /drivers/base/bus.c. The specific execution logic is as follows:

• (1) Call kzalloc() to create memory for struct subsys_private, set priv->bus to the bus type to be registered, and then assign bus->p to priv.
• (2) Initialize the bus notifier.
• (3) Reset the name of priv->subsys.kobj, that is, the name of the bus.
• (4) Initialize the kset and ktype fields of priv->subsys.kobj.
• (5) Call kset_register to register private->subsys.kobj in the kernel.
• (6) Call bus_create_file to add a uevent attribute under the bus directory in the sysfs file system:
• (7) Call kset_create_and_add() to add devices and drivers kset to the kernel, making them viewable in sysfs:
• (8) Initialize variables such as mutex, klist_devices, and klist_drivers in the priv pointer.
• (9) Call add_probe_files to add bus_attr_drivers_probe and bus_attr_drivers_autoprobe attributes under the bus:
• (10) Call bus_add_groups to add the bus_groups attribute group.

(4-2) Bus Unregistration

Call bus_unregister to perform the specific bus unregistration operation, which is also implemented in /drivers/base/bus.c:

void bus_unregister(struct bus_type *bus)
{
 pr_debug("bus: '%s': unregistering\n", bus->name);
 if (bus->dev_root)
  device_unregister(bus->dev_root);
 bus_remove_groups(bus, bus->bus_groups);
 remove_probe_files(bus);
 kset_unregister(bus->p->drivers_kset);
 kset_unregister(bus->p->devices_kset);
 bus_remove_file(bus, &bus_attr_uevent);
 kset_unregister(&bus->p->subsys);
}

(4-3) Adding Device and Device Driver

The Linux kernel’s driver model provides two interfaces, device_register() and driver_register(), for various driver modules. From the source code of multiple subsystems in the Linux kernel, it can be found that the registration of various drivers ultimately calls driver_register(). However, the core logic of these two interface functions is implemented by calling the bus’s bus_add_device() and bus_add_driver(): in driver_register(), it calls driver_find() to find a driver with a given name in the specified bus; if the driver already exists, it returns -EBUSY; if the driver does not exist in the bus, it calls bus_add_driver() to register the driver. In device_register(), it first calls device_initialize to initialize the device (essentially assigning values to struct device), and then calls device_add to register the device in the Linux kernel driver model.

The interfaces device_register() and driver_register() are both implemented in /drivers/base/bus.c. Let’s take a closer look at the execution logic of these two interfaces:

1. Execution logic of bus_add_device:

• (1) Obtain the bus_type* pointer from dev->bus; if obtaining the bus fails, the function returns directly; if the bus is successfully obtained, proceed to step (2).
• (2) Call device_add_attrs to add the default attributes defined by the bus->dev_attrs pointer to the kernel; this operation will be reflected in the sysfs file system under the directory /sys/devices/xxx/xxx_device/.
• (3) Call device_add_groups to add the bus_dev_groups to the kernel.
• (4) Call sysfs_create_link to link this device’s directory in sysfs to the devices directory of this bus.
• (5) Then call sysfs_create_link again to create a link in this device’s sysfs directory pointing to the directory of the bus it belongs to, named subsystem.
• (6) The previous operations are essentially registering information about the device in the sysfs file system, providing interfaces to user space. The final step is to call klist_add_tail() to save the device pointer in bus->p->klist_devices.

2. Execution logic of bus_add_driver:

• (1) First call bus_get to obtain the bus_type pointer of the driver. If this pointer is zero (i.e., obtaining fails), return -EINVAL; otherwise, continue with subsequent operations.
• (2) Allocate space for the driver’s struct driver_private pointer (priv) and initialize variables such as priv->klist_devices, priv->driver, priv->kobj.kset, etc., while saving priv in the device_driver‘s p.
• (3) Call kobject_init_and_add, passing the driver’s name as a parameter, to register the driver’s kobject in sysfs. This operation is reflected in the sysfs file under /sys/bus/xxx/drivers/.
• (4) Call klist_add_tail to add the driver to the bus’s klist_drivers linked list; if the driver’s drivers_autoprobe is true, call driver_attach to attempt to bind the driver to the device.
• (5) Call module_add_driver() to add the driver to drv->owner, but we won’t analyze this further.
• (6) Call driver_create_file to create a uevent attribute in the driver directory in the sysfs file system.
• (7) Call driver_add_groups to add the bus->drv_groups attribute group to the driver.

(4-4) Driver Probe

When a driver is probing, most of the logic will depend on the specific implementation of the bus, with core operations being bus_probe_device() and driver_attach(). These two interfaces are declared in drivers/base/base.h and implemented in drivers/base/bus.c.

• Implementation of bus_probe_device() is as follows:

• Implementation of driver_attach() is as follows: From the above diagram, it can be seen that driver_attach will call bus_for_each_dev to traverse devices. This function is used to traverse the device list of drv->bus and calls __driver_attach() for each device, passing drv to __driver_attach. Therefore, it is clear that the operations performed on each device are __driver_attach(), which is defined in /drivers/base/dd.c and its specific execution logic is as follows:

(1) Use driver_match_device() to determine whether the driver and device match (essentially checking if drv->bus->match is specified; if specified, execute the corresponding function; otherwise, return 1). If the driver and device match, return directly; otherwise, continue with subsequent operations.

(2) Call driver_probe_device() to attempt to bind the driver and device together.

Overall, the operation processes of bus_probe_device() and driver_attach() are similar, i.e., search for the bus, compare whether there is a device_driver (or device) with the same name, and if there is and the device is not bound to a driver (note: this is important to allow the same driver to drive multiple devices with the same name), call the probe interface of the device_driver.