About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Thu, 18 Oct 2007

Filesystem survey. Take 2.

I've started a short survey between my known admins about filesystem usage statistics they have on different systems, and about how it should be handled in the best case.
To make it clear I asked to specify type of the system (like mail server, e-commerce, database and so on), type of the load (a lot of small/big reads/writes, sequential/random access and so on), and expected results (i.e. what load should be preferred and what performance is expected and would be good to have). With results of the previous survey this should create a clear picture of the usage cases for any filesystem.

Feel free to drop in comments your ideas and experience with different systems.

Since those, who origianlly really wanted to participate in distributed system discussion, keep silence, I will postpone new redundancy code implementation (I started it in userspece already, so maybe I will return to it quite soon).
It also looks like vast majority of the systems of such scale use the simplest mirroring.

For the new release of the distributed storage I will definitely implement ability to store dirty/clean bitmap on the disk, mainly for the following purpose: suppose you have a distributed storage with multiple mirror devices, if one of them fails and you want to replace it with new one, which has new media, but higher layer (dst itself) only knows about pending updates, which were not stored on the media, so when new node 'resurrects' it will read its dirty/clean bitmap from the media, find that it is clear (or its age is not equal to age of the currently used system) and thus force DST core to copy all required blocks from different node.
This is not that complex task so I've started it right now.

Stay tuned.

/devel/fs :: Link / Comments ()