Zbr's days.
←July→
Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    
←2008→
Months
Jan Feb Mar
Apr May Jun
JulAug Sep
Oct Nov Dec

About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Sat, 19 Jul 2008

Disributed storage is dead, long live the Distributed storage!

As you may know, DST project was an attempt to implement redundant, failover resistant, flexible block level storage subsytem. Among other features it supported ability to map multiple remote nodes via linear or mirroring algorithms to single node, reconnect to failed node, reading balancing and parallel writing to multiple nodes (in case of mirroring) and so on.

Now it has gone. There is no more distributed storage you knew before, instead there is completely new project being developed, which main goal is to provide a transport layer for the block requests only. Consider it as Network Block Device on huge steroids. Consider it as iSCSI on huge steroids. Consider it as ATA-over-Ethernet on even more huge steroids.
It is just an example of what all those protocols should have. And only that.
An it does not sound very ambitious, previous DST versions already supported lots of features, which never existed (and in some cases were impossible to be added) in another block level network storages.
DST moves further.

There will be no mirroring and overall ability to map multiple devices into single one, instead one should use Device Mapper for this goal, since its features were simply mirrored (although I tried to optimize them sometimes) in DST, and amount of targets was noticebly smaller.

Now DST is just a simple block device which operates on top of network connection. With just a single exception: its done right.

Features planned for the new Distributed Storage:

  • kernelspace client and server
  • initial autoconfiguration between client and server nodes
  • automatic reconnect to failed target
  • transaction model: resending, timeout error completion, full rollback of the failed transaction
  • wire speed performance
  • data channel encryption, strong checksumming
  • cryptographical authentification
  • ability to work on top of any network protocol
  • barriers support (when, if any, Device Mapper will start support them, DST will not need to be changed)
  • flexible protocol with simple ability to extend it to needed functionality
  • trivial configuration
Project is being written from scratch, but it is actually very simple, and should be quite small, so expect its first release quite soon.
It will be pushed upstream when ready.

/devel/dst :: Link / Comments (8)

anonymous wrote at 2008-07-22 07:46:

Sorry for the silly question, but I really don't know about this and you give me curiosity :). When you said:

"There will be no mirroring and overall ability to map multiple devices into single one, instead one should use Device Mapper for this goal"

what do you refer to ? how would I do to have "mirroring" ability ?

Sorry, I know that probably I don't understand because I don't know enough, and I was really intrested in mirroring ability of DST. But if you please could give an example of how can this facility be achived with the new DST ? I really dont understand :S

"It will be pushed upstream when ready. " --> anyway, I am really glad to hear that :)

Thanks a lot!

Zbr wrote at 2008-07-23 00:12:

I meant that one should use device mapper and/or lvm to implement mirroring, raid, striping or whatever administrator decides to have. DST now looks like simple block device (just like existing local drive), which hides all network communications, failover and recovery there, so you can create a mirror/raid using device mapper on top of it and another similar device (connected to different server) and/or local drives.

anonymous wrote at 2008-07-23 02:04:

you mean for example with "mdadm" ? it should not be problem with timeout and those stuff ? It should be handled in a nice way by DST ?

The difference with NBD in the homepage would be still valid ? or it would be different in an other way ?

Thanks a lot, again!

Zbr wrote at 2008-07-23 02:16:

Yes, mdadm is a userspace utility to configure device mapper. DST lives at lower than device mapper layer, so all its magic is hidden. DST can not be configured via mdadm, since the latter is very different monster implemented to control not target devices, but how they are processed by the selected algorithm. DST instead has own configuration utility.

anonymous wrote at 2008-07-23 05:24:

So to be clear:

Using the new DST and "mdadm" I would have something similar to what is now the mirror alorithm that should work without problems with timeouts and that stuff(because DST manges it with its configuration), or I don't understand ?

Thanks!

Zbr wrote at 2008-07-23 09:37:

Yes, you are right.

anonymous wrote at 2008-07-24 04:57:

Thanks a lot for your patience. Really :-)

Zbr wrote at 2008-07-24 15:56:

No problem :)

Please solve this captcha to be allowed to post (need to reload in a minute): 29 * 42

Name:
URL (optional):
Captcha:
Comments: