About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Thu, 26 Oct 2006

New combined acrypto release.


It fixes small glitch in OCF <-> acrypto bridge.
Thanks to Yakov Lerner (author of the module) for fixing it.
Patch is available in archive.

/devel/acrypto :: Link / Comments ()

Russian language troubles.


Tonnel - look into 'tunnel'. "Brokgauz and Efron dictionary".
Tunnel - look into 'tonnel'. "Big soviet encyclopaedia."

/life :: Link / Comments ()

Gigabit send/recv benchmark for netchannels and sockets using small packets. Complete rout.




Netchannel with userspace network stack behaves much faster.
Receiving is about 8 MB/sec faster. Receiving CPU usage is 3 times less (90% socket code vs. 30% netchannels+unetstack).

Sending is 10 MB/sec faster. Sending CPU usage is 5 times less (upto 50% vs. upto 10%).

Number of syscalls is about 10 times less for netchannels.

Update: I want to say that with increased reading/writing size netchannels with userspace network stack behave exactly like with 128 byte chunks, and socket test's CPU usage decreases significantly. They become equal with about 1024 byte chunk size.
Userspace network stack does not use hardware checksumm offloading and use C-coded one instead, it also has additional memory copy, which should be eliminated.

/devel/networking :: Link / Comments ()

New blog tag related to upcoming filesystem development.


I've started new fs tags, where I plan to put my thoughs about filesystems, VFS and my own FS development ideas.

Let's talk about several modern filesystem trends: extents, delayed allocation and (de)fragmentation.
Extent is a contiguous block on the storage allocated for one file. I.e. using extents and creating FS based on them, we, roughly saying, just increase filesystem block size (not exactly, since it can be written into later too). Which immediately leads to the problem with small files. I think it is possible with ext4 to reuse extent, i.e. split it back into separated blocks for separate files, so that problem can be eliminated, but using extents we just hide the problem of filesystem fragmentation.
In the thread mentioned above, it is suggested that real-time defragmentation is needed and that it is good feature.
So fs will allocate block, write into it, then later new thread will allocate block in the different place, copy data, swap block/inodes, and then next thread will allocate block in the different place... Does not sound too good.
The real solution for such problems is delayed allocation (as far as I can read, only XFS and ext4 support it (the later only in TODO, or already not?), and Reiser4 has something similar too).
Delayed allocation is a way of writing data to the disk - it is not written immediately when page is committed, but some set of them is collected, and then filesystems decides where to put each page. It is very elegant solution (which is implemented in XFS and Reiser4 (although the latter calls is differently)), but it has some problems too - if there is strong memory pressure, delayed allocation can not work, since there will be no pages in VFS to store additional data, and the whole approach forces memory pressure much more than usual one.

So my thoughs about extents and defragmentation is that it is some kind of workarounds for missing delayed allocation (which in turn should be heavily VFS bound, which is not right now, i.e. be similar to IO schedulers). For example ZFS (Solaris future main filesystem) has not only extents, but also reserves several bytes in the inode to store there part of the file - this greatly speeds up stat() and other benchmarks, which only require several first bytes of the file - yes, it speeds some operations up, but why should new filesystem be implemented in the way, which would require a lot of additional steps to behave fast.

/devel/fs :: Link / Comments ()