About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Sat, 02 Sep 2006

Zero-copy sniffer.


I was wrong about magical ab symbols found in sniffer dump - it is usual sending data, but since sending side reserves MAX_TCP_HEADER bytes, so in my setup sending ethernet header starts with offset of 190 bytes and receiving with offset of 16 bytes from the begining of the allocated buffer.

Sending sequence number graph for tcpdump and zero-copy sniffers.


As you can see there are no gaps in graphs (although it is just scp transfer using 100Mb NIC (3c59x), CPU usage for zero-copy sniffer when data is being written into /dev/null is about two times less then when it is writtend using tcpdump.

I've released new version of network allocator and zero-copy sniffer and sent it to netdev@ with a question about possibility of inclusion into mainline.

Zero-copy sniffer has following overheads:

• several atomic operations (in the worst case one atomic_set(), one atomic_inc() and one or two atomic_dec_and_test())
• one lock (bad global lock per sniffer device), which is held when information about new packet is being put into sniffer's queue when skb is freed
• delayed freeing which can lead to increased memory usage, or (like implemented) if introduced maximum amount of "locked" data by sniffer, some packets can be dropped by sniffer.

• use NTA only for netdev_alloc_skb() and sk_stream_alloc_pskb(), i.e. only for allocations of traffic received by NIC and sent through send() syscall over stream socket.
• always compile zero-copy sniffer in, which increases memory usage and adds described above overhead.
• skb_copy() always allocate data from SLAB allocator, although it could check if original skb's data was allocated through NTA, but I think that skb_copy() is completely incompatible with high performance.
• it is possible to eliminate several atomic operations (I'm lazy).
• debug code (poisoning of the tail of the buffer and additional reference counter) is always compiled in.

/devel/networking/zcs :: Link / Comments ()