|
|
About ::
TODO ::
Blog ::
RSS ::
Old blog ::
Projects ::
GIT ::
Gallery ::
Notes
Wed, 03 Jan 2007
Initial thoughs on the 'true AIO'.
Here was
first announce of the idea, and now I will open it a bit more.
This was written after some studing of Intel Dan Williams' work
on async copy found here,
the whole thread
can be also interested for those who want to know what is AIO developemnt status
and some ideas about its improovement.
A generic solution must be used to select appropriate device to perform
actual data processing.
We had a very brief discussion about asynchronous crypto layer
(acrypto)
and how its ideas could be used for async dma engines - user should not
even know how his data has been transferred - it calls async_copy(),
which selects appropriate device (and sync copy is just an additional
usual device in that case) from the list of devices, exported its
functionality, selection can be done in millions of different ways from
getting the fisrt one from the list (this is essentially how your
approach is implemented right now), or using special (including run-time
updated) heueristics (like it is done in acrypto).
Thinking further, async_copy() is just a usual case for async class of
operations. So the same above logic must be applied on this layer too.
But
layers are the way to design protocols, not implement them.
David Miller on netchannels.
So, user should not even know about layers - it should just say 'copy
data from pointer A to pointer B', or 'copy data from pointer A to
socket B' or even 'copy it from file "/tmp/file" to "192.168.0.1:80:tcp"',
without ever knowing that there are sockets and/or memcpy() calls,
and if user requests to perform it asynchronously, it must be later
notified (one might expect, that I will prefer to use kevent :)
The same approach thus can be used by NFS/SAMBA/CIFS and other users.
That is how I start to implement AIO (it looks like it becomes popular):
- system exports set of operations it supports (send, receive, copy,
crypto, ....)
- each operation has subsequent set of suboptions (different crypto
types, for example)
- each operation has set of low-level drivers, which support it (with
optional performance or any other parameters)
- each driver when loaded publishes its capabilities (async copy with
speed A, XOR and so on)
From user's point of view its aio_sendfile() or async_copy() will look
following:
- call aio_schedule_pointer(source='0xaabbccdd', dest='0x123456578')
- call aio_schedule_file_socket(source='/tmp/file', dest='socket')
- call aio_schedule_file_addr(source='/tmp/file',dest='192.168.0.1:80:tcp')
or any other similar call and then wait for received descriptor in kevent_get_events() or
provide own cookie in each call.
Each request is then converted into FIFO of smaller request like 'open file',
'open socket', 'get in user pages' and so on, each of which should be
handled on appropriate device (hardware or software), completeness of
each request starts procesing of the next one.
Reading microthreading design notes
created by Zach Brown (Oracle), I recall
comparison of the NPTL and
Erlang threading models - they are _completely_ different
models, NPTL creates real threads, which is supposed (I hope NOT)
to be implemented in microthreading design too. It is slow.
(Or is it not, Zach, we are intrigued :)
It's damn bloody slow to create a thread compared to the correct non-blocking
state machine. TUX state machine is similar to what I had in my first kevent
based FS and network AIO patchset, and what I will use for current async
processing work.
A bit of empty words actually, but it can provide some food for
thoughts.
/devel/kevent/aio :: Link / Comments ()
|