| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| 7 | 8 | 9 | 10 | 11 | 12 | 13 |
| 14 | 15 | 16 | 17 | 18 | 19 | 20 |
| 21 | 22 | 23 | 24 | 25 | 26 | 27 |
| 28 | 29 | 30 | 31 |
| Months | ||
|---|---|---|
| Jan | Feb | Mar |
| Apr | May | Jun |
| Jul | Aug | Sep |
| Oct | Nov | Dec |
About
TODO
Blog
RSS
Old blog
Projects
Gallery
Notes
Tue, 30 May 2006
HIFN benchmark continue.
Hardware: VIA C3 Ezra with 256 Mb of RAM.
Big file copy from unencrypted to encrypted partition.
HIFN (7955 in 32/33 pci slot) with acrypto dm-crypt:
687235072 bytes (687 MB) copied, 78.1593 seconds, 8.8 MB/s 0.19user 28.15system 1:18.32elapsed 36%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (3major+210minor)pagefaults 0swaps 687235072 bytes (687 MB) copied, 79.6745 seconds, 8.6 MB/s 0.14user 28.30system 1:20.57elapsed 35%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (3major+210minor)pagefaults 0swaps 687235072 bytes (687 MB) copied, 75.5192 seconds, 9.1 MB/s 0.13user 31.09system 1:16.33elapsed 40%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (4major+209minor)pagefaults 0swaps 687235072 bytes (687 MB) copied, 75.9418 seconds, 9.0 MB/s 0.12user 28.85system 1:16.40elapsed 37%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (4major+209minor)pagefaults 0swapsSW dm-crypt:
687235072 bytes (687 MB) copied, 91.1585 seconds, 7.5 MB/s 0.16user 12.82system 1:31.25elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (2major+211minor)pagefaults 0swaps 687235072 bytes (687 MB) copied, 91.5068 seconds, 7.5 MB/s 0.10user 13.12system 1:32.36elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (4major+211minor)pagefaults 0swaps 687235072 bytes (687 MB) copied, 91.4392 seconds, 7.5 MB/s 0.10user 12.98system 1:32.13elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (4major+210minor)pagefaults 0swaps 687235072 bytes (687 MB) copied, 94.6944 seconds, 7.3 MB/s 0.10user 12.82system 1:35.47elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (3major+211minor)pagefaults 0swapsAs you see HIFN results are definitely better. But CPU usage is higher, but that CPU usage is not 100%-what_user_might_use, but instead it is time which copy itself was performed, so if it is higher it is better.
Hardware: Celeron 1.3 Ghz with 504 Mb of RAM.
HIFN (7955 in 32/33 pci slot) with acrypto dm-crypt:
727478272 bytes transferred in 64.528197 seconds (11273804 bytes/sec) 0.19user 17.22system 1:04.55elapsed 26%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+211minor)pagefaults 0swaps 727478272 bytes transferred in 67.712388 seconds (10743651 bytes/sec) 0.18user 15.33system 1:08.23elapsed 22%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (13major+252minor)pagefaults 0swapsSW dm-crypt:
727478272 bytes transferred in 59.352731 seconds (12256863 bytes/sec) 0.22user 10.50system 0:59.54elapsed 18%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (11major+256minor)pagefaults 0swaps 727478272 bytes transferred in 60.080185 seconds (12108456 bytes/sec) 0.26user 11.07system 1:01.19elapsed 18%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (17major+248minor)pagefaults 0swapsHIFN is slower here. Probably because of PCI bridge.
/devel/acrypto :: Link / Comments (0)
Mon, 29 May 2006
Acrypto hacking.
New version of the HIFN driver has been released.
It introduces watchdog, since 795x chip can just freeze
without any interrupt generation. Likely it is my misconfiguration,
but those datasheet that I have does not explain that behaviour
and sometimes even says wrong things.
New acrypto package and combined patchsets have been released.
Short changelog:
- fixed bug in dm-crypt/ipsec acrypto port in AES-192/256 type detection.
- reduced number of atomic operations in simple load balancer.
- removed reference counter leak for broken sessions.
- small code cleanups.
cp/kjournald starvation,
i.e. CPU usage is about 2-3% and input dataflow for dm-crypt
is very small, so overall performance is small too. But in that case CPU
usage is very small too.
/devel/acrypto :: Link / Comments (0)
Sat, 27 May 2006
I love my country.
Officials here seems to forget who made them.
Those people become mad and sick of the power and the distance
between them and other people becomes incredible.
Today I sat in the traffic jam which was created by those stupid officials,
who just stopped the whole transport flow on parts of Moscow ring road (MKAD)
and some other main roads because they did not want to allow group of motor-car
enthusiasts to show them how big is the gap (one of the form is protest against
blue flashing lights on offcials cars which allow them to break any traffic rules).
Fri, 26 May 2006
Marasmus.
I thought before that there is only one country where
things like theological theory of world creation instead of
probabilistic Darwin theory can be studied in the school
even if 37 (!) Nobel Prize laureates signed against it.
But not. There is Russia, where orthodox believer idiots can burn
the greatest russian poet Pushkin's books just because there is "Tale about priest and it's worker dolt".
That tale is about fun and resourceful worker and stupid and greedy priest.
It is a joke of course, but I will not be that wonder if it happens.
Thu, 25 May 2006
HIFN hacking.
Did I say that HIFN sucks? If you read my blog for a while,
you already know this, but I want to repeat.
HIFN just bloody sucks with theirs horrible specs, at least with that that I have.
And dm-crypt does not work with HIFN adapter yet.
I am a looser.
Crap.
/devel/acrypto :: Link / Comments (0)
Wed, 24 May 2006
Netchannels. Initial benchmark.
Changed processing logic: receiving process context reads as much as possible
(within given limit of 16 MTU sized frames)skbs from netchannel queue thus charging ACK sending and
requeueing them into socket queue, and only then starts sk_receive_queue processing.
Initial benchmark.
Speed is the same, CPU usage is the same
(socket CPU usage is 1% (or 20% :) more than netchannel one, but let's
throw this away as some experimental error).
Netchannel was setup in copy_to_user mode, so it was not expected to be
faster or eat less CPU, but main purpose of this test was to show that
netchannels with all protocol processing moved to process' context can
be at least that fast as when it is splitted into pieces.
I've found some links in the web and blogs where developers completely
disagree with VJ idea of moving stuff into process context...
Well, this should break theirs mostly theoretical arguments.
It is clear that using memcpy setup numbers for netchannel are noticebly
better than for sockets (see previous posted benchmarks).
I will start changing core networking code to accept different copying "actor" methods
which will allow to use netchannels preallocated mapped area instead of
copy_to_user().
Patches and userspace utilities can be found in archive.
/devel/networking :: Link / Comments (0)
Mon, 22 May 2006
Netchannels. Full TCP receiving support.
I've implemented full TCP input processing for
netchannels.
It is based on socket processing code and is fairly hairy for now.
Main idea is to queue skbs into netchannels private queue in interrupt
time and then remove skbs and process them in process' context.
To make TCP works userspace procesing code should only perform several
simple steps similar to how backlog is handled in socket code.
Current state is quite proof-of-concept, since there are some ugliness
in the code and various uninteresting debugs, so I plan to clean this up
and run some tests to show if such approach works or not.
Full patch and userspace application are available from netchannel
homepage.
/devel/networking :: Link / Comments (0)
Sir Arthur Conan Doyle's birthday.
Sun, 21 May 2006
Tea ceremony in chinese restaurant.
I was invited to Tanya Z birthday to small chinese restaurant,
where we had a tea ceremony.
I never drunk real green tea before, and found it to be very tasty,
and process very interetsing itself - conventionalized interior,
music, canary voice...
It was very good time there with my friends. Thank you.
New acrypto release.
This small update brings following items:
- removed connector symbols.
- VIA padlock driver moved outside acrypto tree into separate driver.
- New HIFN driver.
/devel/acrypto :: Link / Comments (0)
- increased number of descriptors, which noticebly improved performance.
- fixed reference counting issue which prevented module unloading.
- removed unneded debug.
2.6.16-1.2069_FC4smp -> vanilla 2.6.16-git: ~11.8 MB/s vanilla 2.6.16-git -> 2.6.16-1.2069_FC4smp: ~13.2 MB/s 2.6.16-1.2069_FC4smp -> acrypto HIFN 2.6.16: ~13.2 MB/s acrypto HIFN 2.6.16 -> 2.6.16-1.2069_FC4smp: ~13.5 MB/sAs you might expect, CPU usage is noticebly less.
Above numbers drift with the time, especially when machine running stock FC4 kernel overheats, and that numbers decrease to 12-13 MB/s.
/devel/acrypto :: Link / Comments (0)
Sat, 20 May 2006
HIFN driver updated.
I've released new version which brings HIFN driver
to the latest acrypto API.
Driver is available in archive.
/devel/acrypto :: Link / Comments (0)
Netchannel.
Initial TCP support. Implementation is fairly proof-of-concept,
since I do not like the idea to bind netchannel to socket.
All TCP state machine is handled inside socket code, so userspace
must create listening socket, wait until new connection is created,
accept it and then bind netchannel to the newly created socket for
established connection. All further data flow is handled inside
netchannels, but actually it is not working as expected yet.
/devel/networking :: Link / Comments (0)
Fri, 19 May 2006
Netchannel.
TCP input processing.
I'm back to drawing board to think about input TCP processing.
In receiving zero-copy
implementation I created tricky schema which uses existing Linux TCP stack.
Skb pages (skb->frag_list) are obtained from preallocated VFS cache pages pool, and then they
are just passed into existing stack as usual. Later in process' context
those skbs are being read using kernel_recvmsg() and discarded
without freeing skb->frag_list pages, since they belong to VFS already.
tcp_recvmsg(), which is called from process' context when
user reads data from TCP socket, copies requested number of bytes from skb queue
and, when cleaning received buffers, sends appropriate ACKs.
ACKs are scheduled while skb is processed in input hot path in softirq context,
which is what netchannels are supposed to eliminate.
Similar to netchannel idea is backlog queue in Linux TCP code - when process
reading from socket is scheduled on the same CPU where softirq has happened, skbs are
added into backlog queue, which is then processed in process' context using the
same mechanism, which is used for usual skb processing in
tcp_v4_do_rcv() and namely tcp_rcv_established() for established
connections.
tcp_rcv_established() is very complex, and actually I never digged
into it's internals quite deep to reinvent input TCP state machine.
So, while thinking about it I've started reading famous Van Jacobson
"30 instruction TCP receive"
and some other interesting Van Jacomson's ideas about networking.
The more I think about TCP processing in netchannels, the more I get close to
the following ideas:
- map netchannel to socket.
- implement own TCP (receiving for now) state machine.
struct sock into netchannel and queue all skbs not into netchannel,
but into appropiate socket. When userspace calls netchannel read for established TCP connection,
tcp_v4_do_rcv() will be called, and generic logic for listening netchannels,
which is quite simple.But idea with own TCP state machine implementation is much more interesting.
I think the right thing is to complete first approach and show generic netchannel improvements over usual socket, and then start own TCP stack implementation.
/devel/networking :: Link / Comments (0)
Climbing.
Grange got his head out of slackass,
so we went climbing. I've finally shinned up, but not run traverses.
I do not see how exactly my wall-runnings helped or not, but I finished
a lot of old interesting traces today. It's time to start new ones on negative slopes.
Thu, 18 May 2006
Rusty Russel's note on netchannels.
The most interesting thing mentioned there is ACK processing,
which, according to VJ should happen in process' context in syscall time,
but articles show that this will slow things down noticebly.
The only thing to finish that mostly theoretical (VJ have not presented the code,
article says that it is wrong) dispute, as I see, is to implement TCP in process' context
in syscall and show if it is fast or not. But yet no one showed the code,
only some empty discussions...
/devel/networking :: Link / Comments (0)
Netchannels.
New update:
- memory limits (soft limits, since update is not protected).
- blocking reading.
- two types of data reading backends (copy_to_user(), copy to (could be mapped) area).
Current implementation allows netchannels to be setup permanently, so process can exit and then new one can bind to existing netchannel and read it's data, but it requires some tricks to create mapping of it's pages into process' context...
Also if netchannel is created, but no process is associated with it, who will process protocol state machine?
Patch is available in archive.
/devel/networking :: Link / Comments (0)
Wed, 17 May 2006
Netchannels.
netchannel.2 patch has been released.
It includes blocking netchannel reading and ability to simultaneously read
the same netchannel for different users.
/devel/networking :: Link / Comments (0)
Climbing.
It was quite hard training today, since I have not fully restored,
so it was shorter than usual and sligtly less aggressive.
It was finished with campus-board straining, sauna, shower and finally
good sleep...
Tue, 16 May 2006
Netchannels.
Completed receiving support. For the proof of concept code
uses copy_to_user() to transfer data to userspace.
I've created netchannel project homepage
where one can find link to project archive, design notes and project status.
First, more proof-of-concept than working, version was
presented in netdev@ as long
as new, which works with UDP.
/devel/networking :: Link / Comments (0)
Mon, 15 May 2006
Climbed a lot today.
Training started with sligtly aching muscles, but after
usual warming things became better. Then a lot of wall runnings,
several campus-board sets, and I'm again tired as hell and completely
wet. Some woman even asked what I'm doing and doesn't I tired to make the same traverse
again and again. Coach commented that I have very coarse technique
with that movings.
But they seem to not understand that it _is_ the main aim - to make
myself very tired as fast as possible and train physical endurance.
Netchannels.
Ok, basic interfaces have been created:
- unified cache to store netchannels (IPv4 and stub for IPv6 hashes, TCP and UDP)
- skb queueing mechanism
- netchannel creation/removing commands
- netchannel's callback to allocate/free pages (for example to get data from mapped area) not only from SLAB cache
- create netchannel's callback to move/copy data to userspace
- create read data command
- test new netchannel interface
/devel/networking :: Link / Comments (0)
Sun, 14 May 2006
Journalism is really menacing weapon in the fact distortion fight.
It can not be confused even if there are no facts at all.
Compared Jenkins hash with XOR hash used in TCP socket selection code.
Tests were performed to determine how fair is hashed results distribution.
I've create simple TCP server simulator, which runs bound to single IP address
and port. Simulator has 64k entry array, each entry contains hit counter. There were
100*64k runs, so ideal hash would result in 64k entries with 100 hits each.
There are two types of hashes used for tests - Jenkins hash:
u32 ports; unsigned int h; ports = lport; ports <<= 16; ports |= fport; h = jhash_2words(faddr, laddr, ports); h ^= h >> 16; h ^= h >> 8; return h;and standard XOR hash, which is used in Linux TCP socket selection code:
unsigned int h = (laddr ^ lport) ^ (faddr ^ fport); h ^= h >> 16; h ^= h >> 8; return h;
Tests were run for different set of remote addresses and ports:
- random remote IP, random remote ports
- random remote IP, linear remote ports
- random remote IP, const remote ports
- const remote IP, random remote ports
- const remote IP, linear remote ports
Horizontal axis contains number of hits, and vertical one shows how many elements of array(64k elements total) have given hit counter.
For random remote IP address distribution and different remote source distribution (first three cases above) both XOR and Jenkins hashes show the same fairness.
But for constant remote IP (the latter two cases above) Jenkins hash shows strange artefacts, which are unacceptible for socket hash function.
So I decided to use simple one-dimensional array with XOR hash for netchannel hash table.
/devel/other :: Link / Comments (0)
Sat, 13 May 2006
Netchannels.
Things go very good as far as I can see - I've created initial
cache for netchannel objects and some interfaces for it.
Cache has been built on top of two dimensional array,
dimensions are selected either through destination
port hash or using source address/port, destination address
and protocol number hash.
Also started interface address changes notification related to
netchannel cache update, similar to how FIB cache is updated
through inetaddr_chain notification chain.
I also plan to gather some statistic on how new cache works
for different set of addresses, ports and protocol numbers.
If I'm lucky I will perform set of some benchmarks today.
/devel/networking :: Link / Comments (0)
Fri, 12 May 2006
Netchannels.
Initial design notes.
First of all, do not use sockets. Just forget that such interface exists.
New receiving channel abstraction will be created by special syscall, which allows
to specify in creation time source IP range, or it can be wildcarded to IPADDR_ANY.
Interrupt handler (or netif_receive_skb()) will check skb
and try to get netchannel, then skb will be linked into netchannel receive
queue if memory limits allow it and that is all. No protocol processing.
If hardware allows to split header from data it will be possible to create
zero-copy technique: when netchannel is selected in netif_receive_skb()
it will have ->alloc_data() callback which would allow, for example, to allocate
data from userspace mapped area.
When receiver is awakened after data has been added to netchannel, it will call
netchannel's ->get_data() callback which would allow to remap
data pages from skb into process' VMA.
Unified network cache will hold information about source and destination IP addresses,
or it's hashes for IPv6, information about ports and well-defined IP protocols.
It will be built on top of simple structure similar to tree, which leafs will correspond
to address' hash, where tables of port hashes will be placed.
Main problem here is routing cache. Linux is great in routing, it supports several
cache algorithms, which are extremely fast in appropriate environments,
but main question is "do we need routing for netchannels"?.
So it looks like netchannel either must include that route interface or,
if it will be designed as userspace-only communication channel, implement
own trivial pseudo-route algorithm.
Most of the drivers do not set skb->pkt_type, so it stays default
PACKET_HOST, which means that every packet, received by interface must be
checked to be valid for the system, which is a point to use standard Linux routing,
but from the other point of view, it is quite easy to create own callback for
IP address setup, which would update netchannel routing table.
No one need an excuse to rewrite anything, so I will create own cache for netchannels,
which will be updated when IP addresses are changed.
/devel/networking :: Link / Comments (0)
Netchannels.
I've decided to implement initial unified protocol socket
cache
and move protocol processing into userspace first.
This will allow easier creation of remapping trick, since it requires
process' context to change VMA.
/devel/networking :: Link / Comments (0)
Climbing.
I've found yet another way to make me even more tired faster -
there is nice traverse in Skala-city
with arm holds on reliefs only. My new favourite wall-running trace
is that traverse from start to the end and from the end to the start.
Last time I was only able to finish it 3 times without the rest, and
after it I was unable to move for several minutes. Training was finished
with some exercises on campus-board. It was really good time there.
Thu, 11 May 2006
Netchannels.
After some testing, benchmarking and thoughts,
I'm going to resurrect zero-copy sniffer project [1] and create special
socket option which would allow to insert pages, which contain
skb->data, into process VMA using VM remapping tricks. Unfortunately it
requires TLB flushing and probably there will be no significant
performance/CPU gain if any, but I think, it is the only way to provide receiving
zero-copy access to hardware which does not support header split.
Other idea, which I will try, if I understood you correctly, is to create unified cache.
I think some interesting results can be obtained from following
approach: in softint we do not process skb->data at all, but only get
src/dst/sport/dport/protocol numbers (it could require maximum two cache lines,
or it is not fast-path packet (but something like ipsec) and can be processed as usual)
and create some "initial" cache based on that data, skb is then queued into that
"initial" cache entry and recvmsg() in process context later process'
that entry.
Back to the drawing board...
/devel/networking :: Link / Comments (0)
Rusty Rassel's "Hanging Out With Smart People".
I especially like this one:
- There are more bad ideas than good ideas.
- You can spend an awful lot of time arguing about "taste".
- Some hackers mistake politeness for uncertainty.
- Finite time constraints: "hang the looters".
- Leaders tend to be dogmatic, black and white.
Wed, 10 May 2006
Minor blog update.
I've started to use categories, so you can find
them at the left side when I actually publish something.
Acrypto.
Fixed bug in fragmented skb ESP processing code,
that tricky thing forced changes in acrypto API, since
when ESP code splits skb into fragments it does not align
it's boundaries to crypto buffer size, but acrypto scatter/gather
processor assumed that scatterlists are always aligned properly.
New combined patches for 2.6.15 and 2.6.16 are available in
archive.
/devel/acrypto :: Link / Comments (0)
Climbing.
I'm climbing alone for a long already, and since I try
to bring some system into the trainings, I decided
to improve physical endurance. And today I again
had tons of wall-runnings, special traverses and couple of
boulderings.
Result was as expected - tired as hell, want to sleep do not want to work.
Mood is very good.
As practice shows tomorrow every piece of body will be in pain,
mood will be still very good and likely I will want to hack some interesting stuff.
Tue, 09 May 2006
Before and After.
I was
wrong that mass-production deprives new talents so they do not try to get out
from swamp. They do.
Only priorities were changed - great people are in math, in physics, in medcine -
just everywhere, and that must be taken into account when question of
mankind as thinking, sentient being is risen.
I congratulate you with The Victory Day!
Thank you, veterans, Thank you for what you did and what you do.
Thank you for that you are!
Mon, 08 May 2006
Initial benchmarks of some VJ ideas [mmap memcpy vs copy_to_user].
I've sligtly modified UDP receiving path and run several benchmarks
in the following cases:
- pure recvfrom() using copy_to_user() with 4k and 40k buffers.
- recvfrom() remains the same, but skb->data is copied into kernel buffer which can be mapped into userspace with 4k and 40k buffers instead of copy_to_user().
- recvfrom() remains the same, but no data is copied at all, and only iovec pointer is increased and it's size decreased.
Receiver is simple userspace application with one thread, which does blocking read from UDP socket with default socket/stack parameters.
Receiver runs on 2.4 Ghz Xeon (HT enabled) with 1Gb of RAM and e1000 gigabit NIC.
Sender runs on amd64 nvidia nforce4 with 1Gb of RAM and r8169 NIC.
Machines are connected with d-link dgs-1216t gigabit switch.
Conclusions:
at least in UDP case with 1gbit NIC performance was not increased, but it can be the result of either NIC speed (I do not entrust to nvidia and/or realtek), or broken sender application.
So the only observable result here is CPU usage changes: it was decreased by 30% for copy_to_user() -> memcpy() changes with 40k buffers. 4k buffers are too small to see any performance changes due to syscall overhead.
Since nocopy is actually equal to dma into mapped buffer, so we get something close to 6 times less CPU usage, and if it can be lineary transferred into performance gain, we found where the most significant part of VJ channels lives. Unfortunately it is not backward compatible with recv() system call, and requires major changes in application to use this advantage.
I've posted results to netdev@, let's see what it will end up with.
Sun, 07 May 2006
Weekly cooking madness.
Today I'm quite lazy so I decided to cook up something simple.
And I wanted this to be quite big, so I would have at least
couple of dinners. Choice was made for meat julien. I slightly modified
recipe so I got bigger pieces of meat and replaced small loam pots
with small frying pan to store more meat and cheese sauce.
Result was excellent - but I've eaten the whole dish for several minutes
again. That is why I cook tasty things only once a week.
I would like to see something like "Man's food. 10kg" bags.
Sat, 06 May 2006
Climbing.
I have not completely restored after previous training,
so today climbing was harder, but that is even better.
I decided to finish each training with several
simple exercises on campus-board - it is time to add
at least some system into my trainings.
Van Jacobson netchannels.
There are at least three different topics in his ideas:
- netchannel is hardware and cache friendly management structure, which is created between two peers (NIC and userspace)
- all protocol processing is moved to the end of the peer, i.e. either into userspace (and libtcp) or at least into socket processing code and process context
- memory buffers are not copied between kernelspace and userspace, but mapped, so userspace has direct access to the data with all it's headers
My personal thought is that second idea of moving as much as possible into process context is the most significant and less intrusive change which can be adopted to usual sockets.
I'm going back to the drawing board to create initial design notes about how I see this to be implemented and start hacking.
Fri, 05 May 2006
LOL.
Billboard in Sevastopol, Ukraine (is going to enter NATO in 2008).
"Hello, NATO. This is my city."
Thu, 04 May 2006
Interesting date: 01:02:03:04:05:06.
Hours:minutes:seconds:day:month:year.
I'm invited to the 2006 Kernel Summit by Program Committee.
Climbed a lot today.
After quite long delay (I did not climb one week) I got to the climbing area.
Four hours of wall-runnings and traverses, that was hard especially after such
laziness period. I even got to sauna after training to relax, although I do not like
dry Finnish bath.I feel myself tired as hell now, and it is excellent feeling.
It was definitely very good training.
Wed, 03 May 2006
I have heavy steel construction to protect myself from external world now.
After more than 5 hours running here and there without the rest
I finally setup it. Only absence of the water stops me from starting the
new development. I've gotten even new book named "Plaster" besides Yaroslav Gashek's
"Stories and feuilletons".
Time, how many things depend on it...
Tue, 02 May 2006
Real hacking.
Did you hear about GSM SIM-card cloning?
There are people who sell (in russian)
SDK and hardware to get IMSI and Ki from old comp128v1 based SIM-card and to reflash new ones.
There is no hardware for COMP128 v3 and v2 though.
And here is an article about
GSM security from Berkely Lab with set of very interesting links.
IMHO, that is a real hacking.
Kevent.
Updated kevent
project - removed direct calls to ext3_get_block() and introduced new generic
address space ->get_block() callback instead.
New patchset kevent_full.diff.3 was relesead which includes above
changes and aio_sendfile() implementation.
It was sent to netdev@ for review.
Mon, 01 May 2006
Acrypto.
Fixed IV update for linux kernel 2.6.15 and new acrypto-combined-2.6.15.diff.4
combined patchset has been released. As usual, patch is available in
archive.
Virtualization.
It looks like virtualization becomes extremely essential in
modern systems. But is it really that cool and is it needed at all?
If you buy one big machine and want to share it's resources between many users
in a way they know nothing about that system is shared between several entities,
you want virtualization control to be flexible enough to allow some users to have
different amount of CPU, memory and other system resources.
But let's more precisely look at how for example CPU could be shared between users.
Let's say you want to limit one user to have no more than 20% of the CPU usage,
obviously it can only be measured in peak, but if your system continuously runs
with 100% CPU usage then it either has too slow CPU or you errorneously disigned it
for given task. Memory is another resource which is frequently shared between many users,
and obviously in well designed environment one may not be allowed to get all memory
from other users. Let's say we want to limit one user to have 20% of the all system
memory. There is a question here, does swap belong to above memory or not, since
access time to swap pages and RAM differs in orders. It is quite easy to see,
that if above 20% limit only limits the whole system's memory including swap,
it is possible to bigger user (which has 80% of the memory) to always be in swap,
which will end up to be pathetically slow which is not the desired behaviour.
So we should limit RAM and swap differently, let's say we want to limit
one user to have only 20% of RAM. RAM is a set of physical pages, each one is
represented by some structure in reserved area of that RAM, so, for the purpose
of right user limitation, we can mark each page to be part of some user's data,
which in turn becomes too big overhead, struct page is too big already.
one can create it's own allocator so each user will only get predefined piece
of physical RAM, but it can end up with DMA problems for example. The simplest way
is to just change vanilla alocator to check if user's quota is exceeded when new allocation
is being performed, and adjust that value when user's pages are moved to the swap.
But here we strike another problem: how should pages be selected to be evicted to swap?
If we will not change swap daemon itself, then it might select pages from different
user, which will end up in not fair distribution, but if we want to evict only
pages that belong to given user, we must mark them specially, which, first,
adds very big overhead, and second, it will hurt swap devices since not the less
active pages were put into the swap, so they will be extracted soon.
I'm quite sure there are some ways to workaround virtualization overhead here and there,
but in my personal opinion, the whole idea, at least for now, does not cost it's price.
I personally like small computers, especially embedded 2" boards with low-power embedded
CPUs with tricky features which will compile the whole kernel several days.
Kevent based AIO sendfile().
Initial commit can be found in archive.
Patch is called aio_sendfile.1 and depends on
full kevent
patchset kevent_full.diff.2, which was recenly
sent to netdev@.
Patch is fairly trivial - just use file->f_op->sendpage() for page
sending, all asynchronous mechanism lives in page propagation into VFS cache.
Patch has been sent to netdev@ for review.