About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Sat, 06 Sep 2008

Latencies in netchanneles and sockets in receiving path.

When NIC's interrupt fires in Linux, driver's handler does not process the packet, it either schedules NAPI handler, which will push packet higher to the stack, or submit packet to the software interrupt handler, which will do the same. This is the first queue: interrupt->fotware interrupt (or NAPI, which happens in the same context).

When NAPI polling handler (or networking software interrupt) fires, it searches for the appropriate receiving socket, adds data packet to its queue and wakes up a receiving process. This is second queue.

Netchannels currently work the same way, since its receiving processing happens in netif_receive_skb(), which already may be too late for some low-latency applications.

As was noticed by Salvatore Del Popolo, it is possible to queue packet into netchannel in netif_rx(), but that will limit netchannels to only work with non-NAPI drivers. Instead I think about creating a special helper which will be invoked from the interrupt handler and if there is no appropriate netchannel to queue data into, it will schedule NAPI or network softirq. So far this is in todo list though.

What was really done, its a complete rework of the initalization process, netchannel creation and allocation and its processing. Essentially I rewrote most of the netchannels subsystem for good. It became lockless (RCU protected, there is a hash bucket lock, which is only used when netchannel is added/removed from the bucket, searching is lockless), but allocation process is slower, since netchannel now contains array of the skb pointers, which is allocated at creation time. Size of the array is limited to maximum number of packets netchannel can hold, kind of queue size.

/devel/networking :: Link / Comments ()