About TODO Blog RSS Old blog Projects Gallery Notes

Sun, 14 Oct 2007

Scaling inodes.

I have in mind an interesting design idea of the inode management structure in my (so far theoretical future) filesystem - so called scaling inodes. Basic idea is to have inodes of the several standard sizes like 256, 512, 1k, 2k and 4k, which will contain meta-information about data object and its data embedded at the end of the structure. Such design reduces number of seeks to get data from disk, reduce fragmentation, since bigger inodes will be allocated at the new locations as a new object, allows to reuse old one for new inode, simplifies copy-on-write semantics, greatly reduce directory reading speed (for limited degree though - there is number of factors - for example my current mainline 2.6 git tree contains about 1500 objects, files with smaller than 4k is more than 60%, about the same statistics is for /etc: about 1600 total, where more than 1300 are smaller than 4k).
This is of course a total speculation, but idea worth further investigation...

/devel/fs :: Link / Comments (0)

HAMMER filesystem in DragonflyBSD.

Here are two links about interesting cluster filesystem started by Matt Dillon of DragonflyBSD (desing and performance expectations). By design it is supposed to handle multi-machine installations of the filesystem, online backup, recovery of the failed nodes (because of full replication of the filesystem on different nodes) and eventually multi-master setup (expected to be completed in a year). So far number of design documents were published, I only followed couple of them. Unfortunately they did not contain enough details for deep analysis of the idea, so right now I can not say, how it will look like (except rough words of the underlying structure), but it will be possible to use this filesystem as a usual one, so it guarantees data integrity (compare this with filesystems created for search engines (aka producer-consumer data rings) like googlefs or hadoop filesystem).
I believe this is a good step further for BSD systems to invent new filesystem.
So far the only disadvantage, based on my limited knowledge of design principles, is full filesystem replication (i.e. mirroring mode), which in case of failure of some of the nodes, will require more complex recovery process than block-based replication, but contrary it allows easier snapshot support.
I will try to follow its development a bit more closer to get more knowledge about it, especially if I will work on own distributed filesystem.

/devel/fs :: Link / Comments (0)

Meanwhile in Moscow ...

... there is a snowfall, a 100% humidity, an angry wind and killing cold.
And me of course... sitting and slacking at the office.
I would drink a bit and ate ravioli, I did not eat them for so long already, but can not cook at home obviously. I tried to cook when I was at Marina's birthday - but made a too salty meat - I'm forgetting this bits already, so need to finish faster and start cooking again...
Anyway I will not (at least in the nearest future), so just sending you 'Hello' from this bloody weather, Moscow, Russia (somewhere between Europe and west coast of USA).

/other :: Link / Comments (0)

I live in a great country.

Several years ago one of the former ministers of the tv/radio broadcasting said following phrase (after big license price increase):

We get money not for the paper, but for nature resource usage during transmission of the electro-magnetic waves over the air.

/other :: Link / Comments (0)

I've returned.

Back to drawing board - thinking about distributed storage, new redundancy codes and maybe distributed filesystem...
Enough for now, this is short term tasks.

/life :: Link / Comments (0)