NAPI

NAPI is the event handling mechanism used by the Linux networking stack. Today, the term “NAPI” no longer represents any specific meaning [1].

In basic operations, devices notify the host of new events via interrupts. The host then schedules a NAPI instance to handle these events. Additionally, devices can poll for events through NAPI without first receiving an interrupt (busy polling).

NAPI processing typically occurs in the context of a software interrupt, but it can also optionally use a separate kernel thread for NAPI processing.

Overall, NAPI abstracts away the context and configuration information for event (packet reception and transmission) handling from the driver.

Driver API

The two most important elements of NAPI are the struct napi_struct structure and the associated polling methods. The struct napi_struct structure holds the state of the NAPI instance, while the polling method is the driver-specific event handler. This method typically releases transmitted Tx packets and processes newly received packets.

Control API

netif_napi_add() and netif_napi_del() are used to add or remove a NAPI instance from the system. These instances are attached to the network device passed as a parameter (the instance will be automatically deleted when the network device unregisters). The instance is disabled when added.

napi_enable() and napi_disable() are used to manage the disabled state. A disabled NAPI cannot be scheduled, and it is guaranteed that its poll method will not be called. napi_disable() will wait for the release of ownership of the NAPI instance.

The control API is not idempotent. Control API calls are safe when used concurrently with data path APIs, but an incorrect order of control API calls may lead to system crashes, deadlocks, or race conditions. For example, calling napi_disable() multiple times in succession will lead to a deadlock.

Data Path API

napi_schedule() is the fundamental method for scheduling NAPI polling (poll). The driver should call this function within its interrupt handler (for more information, see the “Scheduling and Interrupt Masking” section). A successful call to napi_schedule() will take ownership of the NAPI instance.

After NAPI is scheduled, the driver’s polling (poll) method will be called to handle events/packets. This method accepts a budget parameter—the driver can process the completion of any number of Tx packets but should only process up to budget Rx packets. Rx processing typically incurs much greater overhead.

In other words, for Rx processing, the budget parameter limits the number of packets the driver can process in a single polling. When budget is 0, specific Rx APIs like page pool or XDP cannot be used at all. Regardless of the value of budget, skb Tx processing should occur, but if this parameter is 0, the driver cannot call any XDP (or page pool) APIs.

Warning

If the kernel only attempts to handle skb Tx completions without processing Rx or XDP packets, the budget parameter may be 0.

poll method will return the amount of work completed. If the driver still has work remaining (for example, if the budget has been exhausted), the polling method should return exactly budget. In this case, the NAPI instance will be processed/polled again (without needing to be rescheduled).

If event processing is complete (all unprocessed packets have been handled), the polling method should call napi_complete_done() before returning.napi_complete_done() will release ownership of the instance.

Warning

Care must be taken when handling the case where all events have been processed and exactly budget packets were used. This rare situation cannot be reported to the networking stack, so the driver should either not call napi_complete_done() and wait to be called again, or return budget - 1.

If budget is 0, napi_complete_done() must never be called.

Call Order

The driver should not assume the exact order of calls. Even if the driver does not schedule an instance, the polling (poll) method may still be called (unless the instance is in a disabled state). Similarly, even if the napi_schedule() call is successful, there is no guarantee that the polling (poll) method will be called (for example, if the instance is disabled).

As described in the “Control API” section—napi_disable() and subsequent calls to the polling method only wait for the ownership of the instance to be released, not for the polling method to exit. This means that the driver should avoid accessing any data structures after calling napi_complete_done().

Scheduling and Interrupt Masking

The driver should keep interrupts masked after scheduling the NAPI instance—until the NAPI polling is complete, any subsequent interrupts are unnecessary.

Drivers that must explicitly mask interrupts (as opposed to devices that automatically mask interrupts) should use napi_schedule_prep() and __napi_schedule() calls:

if (napi_schedule_prep(&amp;v-&gt;napi)) {
    mydrv_mask_rxtx_irq(v-&gt;idx);
    /* Schedule after masking to avoid race */
    __napi_schedule(&amp;v-&gt;napi);
}

Only after a successful call to napi_complete_done() can interrupts be unmasked:

if (budget &amp;&amp; napi_complete_done(&amp;v-&gt;napi, work_done)) {
  mydrv_unmask_rxtx_irq(v-&gt;idx);
  return min(work_done, budget - 1);
}

napi_schedule_irqoff() is a variant of napi_schedule() that takes advantage of the guarantees provided when called in an interrupt context (no need to mask interrupts). If interrupts are threaded (for example, if PREEMPT_RT is enabled), napi_schedule_irqoff() will fall back to napi_schedule().

Mapping Instances to Queues

Modern devices have multiple NAPI instances (struct napi_struct) per interface. There are no strict requirements on how these instances map to queues and interrupts. NAPI is primarily a polling/processing abstraction with no specific user-facing semantics. That is to say, most network devices ultimately use NAPI in a fairly similar manner.

The most common mapping of NAPI instances is one-to-one with interrupt and queue pairs (a queue pair is a set of a single receive (Rx) queue and a single transmit (Tx) queue).

In less common cases, a single NAPI instance may be used for multiple queues, or receive and transmit queues may be handled by different NAPI instances on a single core. However, regardless of how queues are allocated, there is typically still a one-to-one mapping between NAPI instances and interrupts.

It is worth noting that the ethtool API uses the term “channel,” where each channel can be rx, tx, or combined. The exact composition of channels is not yet clear; the recommended understanding is to view a channel as an IRQ/NAPI serving a given type of queue. For example, configuring 1 rx channel, 1 tx channel, and 1 combined channel is expected to use 3 interrupts, 2 receive queues, and 2 transmit queues.

Persistent NAPI Configuration

Drivers typically dynamically allocate and free NAPI instances. This can lead to the loss of user configuration associated with NAPI each time a NAPI instance is reallocated.netif_napi_add_config() API avoids this configuration loss issue by associating each NAPI instance with a persistent NAPI configuration based on a driver-defined index value (such as queue number).

Using this API allows for persistent NAPI IDs (and other settings), which is beneficial for user-space programs using SO_INCOMING_NAPI_ID. For other NAPI configuration settings, see the sections below.

Drivers should use netif_napi_add_config() whenever possible.

User API

User interaction with NAPI relies on the NAPI instance ID. Users can only see these instance IDs through the SO_INCOMING_NAPI_ID socket option.

Users can query the NAPI ID of a device or device queue using netlink. This can be done programmatically in user applications or using scripts included in the kernel source tree:tools/net/ynl/pyynl/cli.py.

For example, using the script to dump all queues of a device (which will show the NAPI ID for each queue):

$ kernel-source/tools/net/ynl/pyynl/cli.py \
          --spec Documentation/netlink/specs/netdev.yaml \
          --dump queue-get \
          --json='{"ifindex": 2}'

For more details on available operations and attributes, see Documentation/netlink/specs/netdev.yaml.

Software Interrupt Coalescing

By default, NAPI does not perform any explicit event coalescing. In most cases, batching occurs due to interrupt coalescing performed by the device. However, in some cases, software coalescing can be beneficial.

NAPI can be configured to start a re-polling timer after processing all packets, rather than immediately unmasking hardware interrupts. The network device’s gro_flush_timeout sysfs configuration is reused to control the delay of the timer, while napi_defer_hard_irqs controls the number of consecutive empty polls before NAPI gives up and resumes using hardware interrupts.

The above parameters can also be set individually for each NAPI using netlink via netdev-genl. When using netlink and configuring each NAPI individually, the above parameters use hyphens instead of underscores: gro-flush-timeout and napi-defer-hard-irqs.

These can be configured programmatically in user applications for each NAPI or using scripts included in the kernel source tree:tools/net/ynl/pyynl/cli.py.

For example, using the script:

$ kernel-source/tools/net/ynl/pyynl/cli.py \
          --spec Documentation/netlink/specs/netdev.yaml \
          --do napi-set \
          --json='{"id": 345,
                   "defer-hard-irqs": 111,
                   "gro-flush-timeout": 11111}'

Similarly, the parameter irq-suspend-timeout can be set using netlink via netdev-genl. There is no global sysfs parameter for this value.

irq-suspend-timeout is used to determine how long an application can completely suspend interrupts. It is used in conjunction with SO_PREFER_BUSY_POLL, which can be set individually for each epoll context via EPIOCSPARAMS ioctl.

Busy Polling

Busy polling allows user processes to check for incoming packets before a device interrupt is triggered. Like any busy polling, it trades CPU cycles for lower latency (the use of NAPI busy polling in production environments is not widely known).

It can be enabled by setting SO_BUSY_POLL on selected sockets, or using the global net.core.busy_poll and net.core.busy_read system control parameters to enable busy polling. There is also an io_uring API for NAPI busy polling.

Epoll-based Busy Polling

Packet processing can be triggered directly from a call to epoll_wait. To use this feature, user applications must ensure that all file descriptors added to the epoll context have the same NAPI ID.

If the application uses a dedicated receive thread, it can use SO_INCOMING_NAPI_ID to obtain the NAPI ID of incoming connections and then assign that file descriptor to the worker thread. The worker thread will add that file descriptor to its epoll context. This will ensure that the file descriptors included in the epoll context of each worker thread have the same NAPI ID.

Alternatively, if the application uses SO_REUSEPORT, a bpf or ebpf program can be inserted to assign incoming connections to individual threads, ensuring that each thread only receives incoming connections with the same NAPI ID. Care must be taken in systems that may have multiple network cards.

To enable busy polling, there are two options:

The /proc/sys/net/core/busy_poll can be set to a time in microseconds for busy looping waiting for events. This is a system-wide setting that will cause all epoll-based applications to busy poll when calling epoll_wait. Since many applications may not need to busy poll, this may not be an ideal choice.
Applications with newer kernels can issue ioctl commands on the epoll context file descriptor to set (EPIOCSPARAMS) or get (EPIOCGPARAMS) struct epoll_params, which the user program can define as follows:

struct epoll_params {
    uint32_t busy_poll_usecs;  // Microseconds for busy polling
    uint16_t busy_poll_budget; // Budget for busy polling
    uint8_t prefer_busy_poll;  // Prefer busy polling

    /* Fill the structure to a multiple of 64 bits */
    uint8_t __pad;
};

Interrupt Request (IRQ) Mitigation

While busy polling is typically used for low-latency applications, similar mechanisms can also be used for IRQ mitigation.

Applications that process a large number of requests per second (especially routing/forwarding applications, particularly those using AF_XDP sockets) may not want to be interrupted before completing a request or a batch of packets.

Such applications can promise the kernel that they will regularly perform busy polling operations, and the driver should permanently mask the device’s IRQ. This mode can be enabled using the SO_PREFER_BUSY_POLL socket option. To avoid abnormal behavior in the system, if no busy polling calls are made within the gro_flush_timeout time, this promise will be revoked. For busy polling applications based on epoll, the prefer_busy_poll field in the struct epoll_params can be set to 1, and the EPIOCSPARAMS ioctl command can be issued to enable this mode. For more details, see the section above.

The NAPI budget for busy polling is lower than the default (which is reasonable considering the low-latency intent of normal busy polling). However, this is not the case for IRQ mitigation, so the SO_BUSY_POLL_BUDGET socket option can be used to adjust the budget. For busy polling applications based on epoll, the busy_poll_budget field in the struct epoll_params can be adjusted to the desired value and set in a specific epoll context using the EPIOCSPARAMS ioctl command. For more details, see the section above.

It is important to note that choosing a larger value for gro_flush_timeout will delay IRQs for better batching, but will introduce latency when the system is not fully loaded. Choosing a smaller value for gro_flush_timeout may cause device IRQs and soft interrupt processing to interfere with user applications attempting to busy poll. These trade-offs should be considered when carefully selecting this value. Busy polling applications based on epoll can reduce the impact on user processing by choosing an appropriate value for maxevents.

Users may need to consider another approach—IRQ suspension—to help address these trade-offs.

Interrupt Request (IRQ) Suspension

IRQ suspension is a mechanism where the device’s IRQ is masked when the epoll triggers new API (NAPI) packet processing.

When the application successfully calls epoll_wait to obtain events, the kernel will delay the IRQ suspension timer. If the kernel does not obtain any events during busy polling (for example, due to reduced network traffic), IRQ suspension will be disabled, and the above IRQ mitigation strategy will be enabled.

This allows users to balance CPU consumption and network processing efficiency.

To use this mechanism:

Each NAPI configuration parameter irq-suspend-timeout should be set to the maximum time (in nanoseconds) that the application’s IRQ can be suspended. As mentioned above, this can be done via netlink. This timeout is a safety mechanism to restart IRQ-driven interrupt handling if the application stalls. The value chosen should cover the time required for the user application to process data from the call to epoll_wait, noting that the application can control the amount of data obtained by setting max_events when calling epoll_wait.
The system filesystem (sysfs) parameters or each NAPI configuration parameters gro_flush_timeout and napi_defer_hard_irqs can be set to lower values. They will be used to delay IRQs after busy polling finds no data.
The prefer_busy_poll flag must be set to true. This can be done using EPIOCSPARAMS input/output control (ioctl) as mentioned above.
The application uses epoll as described above to trigger NAPI packet processing.

As mentioned above, as long as subsequent calls to epoll_wait return events to user space, the irq-suspend-timeout will be delayed, and IRQs will be disabled. This allows the application to process data without interruption.

Once a call to epoll_wait does not find any events, IRQ suspension will be automatically disabled, and the gro_flush_timeout and napi_defer_hard_irqs mitigation mechanisms will take effect.

It is expected that irq-suspend-timeout will be set to a value much greater than gro_flush_timeout, as irq-suspend-timeout should suspend IRQs within a user-space processing cycle.

While it is not strictly necessary to use napi_defer_hard_irqs and gro_flush_timeout when using IRQ suspension, it is strongly recommended to use them.

IRQ suspension will alternate the system between polling mode and IRQ-driven packet delivery mode. During busy periods, irq-suspend-timeout will override gro_flush_timeout, keeping the system in busy polling mode, but when epoll finds no events, the settings of gro_flush_timeout and napi_defer_hard_irqs will determine the next action.

Network processing and packet delivery essentially have three possible loops:

Hard interrupt (hardirq) -> Soft interrupt (softirq) -> NAPI poll; basic interrupt delivery
Timer -> Soft interrupt (softirq) -> NAPI poll; delayed IRQ processing
Epoll -> Busy polling -> NAPI poll; busy loop

If gro_flush_timeout and napi_defer_hard_irqs are set, loop 2 can take control from loop 1.

If gro_flush_timeout and napi_defer_hard_irqs are set, loops 2 and 3 will “compete” for control.

During busy periods, irq-suspend-timeout will act as a timer in loop 2, effectively making network processing favor loop 3.

If gro_flush_timeout and napi_defer_hard_irqs are not set, loop 3 cannot take control from loop 1.

Therefore, it is recommended to set gro_flush_timeout and napi_defer_hard_irqs, as otherwise setting irq-suspend-timeout may not yield significant effects.

Threaded NAPI

Threaded NAPI is an operational mode that uses dedicated kernel threads instead of software IRQ context for NAPI processing. This configuration is per network device and affects all NAPI instances of that device. Each NAPI instance will generate a separate thread (named napi/${ifc-name}-${napi-id}).

It is recommended to pin each kernel thread to a single CPU, which should be the same CPU that handles interrupts. Note that the mapping between IRQs and NAPI instances may not be straightforward (and depends on the driver). The order of allocation of NAPI instance IDs is opposite to the order of process IDs of kernel threads.

Threaded NAPI can be controlled by writing 0 or 1 to the threaded file in the sysfs directory of the network device. This feature can also be enabled for specific NAPI using network link interfaces.

For example, using the following script:

$ ynl --family netdev --do napi-set --json='{"id": 66, "threaded": 1}'

Footnotes

[1] In version 2.4 of Linux, NAPI was originally referred to as New API.

Src

https://docs.kernel.org/networking/napi.html

Linux Networking Subsystem – NAPI

NAPI

Driver API

Control API

Data Path API

Call Order

Scheduling and Interrupt Masking

Mapping Instances to Queues

Persistent NAPI Configuration

User API

Software Interrupt Coalescing

Busy Polling

Epoll-based Busy Polling

Interrupt Request (IRQ) Mitigation

Interrupt Request (IRQ) Suspension

Threaded NAPI

Footnotes

Src

Leave a Comment Cancel reply

NAPI

Driver API

Control API

Data Path API

Call Order

Scheduling and Interrupt Masking

Mapping Instances to Queues

Persistent NAPI Configuration

User API

Software Interrupt Coalescing

Busy Polling

Epoll-based Busy Polling

Interrupt Request (IRQ) Mitigation

Interrupt Request (IRQ) Suspension

Threaded NAPI

Footnotes

Src

Related posts

Leave a Comment Cancel reply