About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Tue, 14 Aug 2007

New release of the distributed storage subsystem.

I've just completed testing of the following complex setup, which I think is a good milestone for the project.

One system has 10 local devices, each of which is a mirror device on top of two remote systems. All 10 devices are combined into single linear device, which is exported as a local target to remote machine, where that device was added into linear device with another remote system, and resulted device was tested by performing different IO tasks.
So, roughly it looks like this:



Main feature of the new release is mirroring to any number of devices with automatic reconnection to remote nodes when it is ready after a crash and resync. Reconnection is a feature of the distributed storage core and does not depend on algorithm being used. Resync bitmap can have various size, so with some compilers it might not compile, I'm considering to move to per-block bitmask in the next release. Because of this feature resync can be inaccurate (i.e. partial if less than a page was written to the device before resync is completed), and thus it will be rescheduled.
Another feature for the next release is documentation (excertpts from blog mostly and overall design, since it looks like (quite big for me amount of) comments in code is not enough).

/devel/dst :: Link / Comments (2)

Strange math in kernel.

During heavy testing of the mirroring code in the distributed storage (200 remote nodes and one local form a mirror, which is pretty slow just because 200 nodes are separated between two physical devices, each one has 100 process accessing each own's file in random order, I can not test 1000 nodes since it will be too slow, try to put 1000 active readers/writes over 1000 5mb files in your desktop, for 100 processes my LA is about 77) I found following problem.
To select an index in the dirty bitmap code gets a sector and devides it to chunk size, so I calculate this:

 request start: 7 segments
 rest of the data: 0 segments
 chunk size: 8 segments
 index: (7 - 0) / 8 = 2305843009213693951

/devel/other :: Link / Comments (0)

Kevent has been removed from kernel summit agenda.

As long as other event delivery mechanisms, seems things settled down and everyone is happy.

/devel/kevent :: Link / Comments (0)

Spam.

I got a mail with following header:

POSIX and up able to recognize all three formats, and let older GNU tar fade out slowly.

/other :: Link / Comments (0)