About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Mon, 01 Sep 2008

Netchannels strike back.

A while ago I implamented Van Jackobson idea of netchannels - peer-to-peer connection module, which pushed all protocol processing as close to the end peers as possible.
In my first realization, TCP processing was done on behalf of running process (instead of mostly bottom-half context), which resulted in a slightly better performance. Then I implemented userspace network stack as a continuation of this idea. Despite its huge performance improvement, I do not think particul reason is netchannels architecture, but instead amount of syscalls to be made to process bulk traffic flow via small packets. Nevertheless it can also be considered as a netchannels architecture improvement, which resulted in so exceptionally good batching abilities.

Now I want to move further: kernel netchannels side will be made completely lockless and simultaneously very cache-friendly. As in the first implementation, idea is not completely mine, approach I will test is based on Van Jackobson's array design to store network buffers.

During its lifetime, netchannels got NAT support (actually just to show to those people, who do not belive in netchannels architecture, that it is possible to implement filtering and packet mangling), but now I drop it from the project. Netchannels also got tricky multidimentsional trie-based storage, which, after being ported to the socket core, resulted in a noticeable perforamance win, although I did not complete it to support statistics. Actually netchannels implementation of this trie is broken, and it required quite a few steps in socket code to be fixed.
Now I drop it from netchannels patchset too and move to the usual hash tables.
I will make RCU locking for them and make netchannels hash table optionally automatically resizeable. This feature does not exist in socket hash tables, but right now I want to experiment smaller code base, since algorithm I have in mind is a bit tricky.

So, there are lots of interesting ideas, which I've started to work on and plan to finish sooner than later. But since I will move to the USA counsil department for the interview, and then want to finish appartment development tasks, and then, hopefully, move to the Kernel Summit and Plumbers conference, it can take quite long... Please note that I do not forget about other projects.
Code is not dead if not marked appropriately in the TODO list :)
Stay tuned nevertheless!

/devel/networking :: Link / Comments ()