Zbr's days.
April
Sun Mon Tue Wed Thu Fri Sat
           
           
2006
Months
Apr

About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Sun, 30 Apr 2006

Weekly cooking madness.


Today I decided to cook up french meat with mega-tosts.
I got wrong bakeware and placed a lot of meat there, so only part was covered with onion and mayonnaise, and that part is really tasty, while other is similar to usual cooked meat.
For the best result french meat should be cooked in wide not very deep baking tray, so every piece of meat would get a lot of juice from upper tasty layers.

:: Link / Comments (0)


Sat, 29 Apr 2006

I congratulate small Abr with his first birthday "in the wild"!


Although it was yesterday nevertheless I wish him to get a lot and only the best things from this world. He is only one year old, so there are a lot of things to learn!

:: Link / Comments (0)


Acrypto.


I've fixed problem with IV update and released new combined patch for 2.6.16 kernel tree.
Patch is available in archive.
Performance is still not less than in synchronous case.

:: Link / Comments (0)


Kevent.


Some core kernel developrs found kevent to be a strong idea, so I concatenated all kevent based projects (NAIO, aio_sendfile) and have sent full patchset to netdev@.

:: Link / Comments (0)


Fri, 28 Apr 2006

I'm stupid.


I got my climbing shoes from repair, I got all equipment, I bought a water and went to the climbing zone... When I was almost there I found that I forgot my training wear.
Crap.

:: Link / Comments (0)


NetBSD rocks!


Grange has sent me quotation of the day:

	Initial commit of proplib, a library for manipulating property lists and
	converting to/from an XML external representation (based on Apple XML
	property lists).  Works in the kernel and user space.

Even Linux does not have XML in kernel.

:: Link / Comments (0)


Thu, 27 Apr 2006

Acrypto hacking.


I've run several acrypto IPsec benchmarks, and pleased to present my results:

	2.6.16-1.2069_FC4smp -> vanilla 2.6.16-git: ~11.8 MB/s
	vanilla 2.6.16-git -> 2.6.16-1.2069_FC4smp: ~13.2 MB/s

	2.6.16-1.2069_FC4smp -> acrypto 2.6.16: ~12.6 MB/s
	acrypto 2.6.16 -> 2.6.16-1.2069_FC4smp: ~13.5 MB/s
And the more CPUs or hardware accelerators you have, the more you will win from acrypto setup, since it was designed with scalability and fully asynchronous processing in mind.

But it is done without IV update patch, after it is applied performance dropped in 6 times, so you should not use acrypto-combined-2.6.16.diff.2 and acrypto-combined-2.6.15.diff.3 for a while, so I removed them from archive.

:: Link / Comments (0)


Wed, 26 Apr 2006

New street.


It is called in honour of Vilor (Vladimir Iljich Lenin Organized Revolution) Trifonov - nice place to live.

:: Link / Comments (0)


Tue, 25 Apr 2006

Descent.


If Earth and mankind were created by some higher mind (god, extraterrestrial or something), are we just pawns in a game or some part of the positive aim?

:: Link / Comments (0)


Arthur Conan Doyle readers.


Mining robbery attempt (in russian).

:: Link / Comments (0)


Mon, 24 Apr 2006

New acrypto combined patchsets have been released.


acrypto-combined-2.6.16.diff.2 and acrypto-combined-2.6.15.diff.3.
The former contains RFC-compliant IV update after each session completion, and the latter contains the same IV update and error path fix for out-of-memory condition.

:: Link / Comments (0)


Wall-running climbing.


Well, Grange definitely can not arrive in time, so I spent a lot of time climbing simple quite long traverses without the reset for physical endurance improvement, which I call wall-runnings. It becomes quite interesting technique since I started to invent new types of movings which make me tired more and more. I found that after several such trainings I awake still tired and this means that some processes are in flight, so no stagnation which is definitely a good sign.

:: Link / Comments (0)


Kevent based AIO sendfile.


Ok, big step has been done - kevent AIO can propagate pages into VFS cache and then process them in user defined chunks, which means that full aio_sendfile() implementation is not that far away.

:: Link / Comments (0)


Sun, 23 Apr 2006

I saw little Abr today!


Abr's house heir is in Moscow now, so I was invited to Abr's wife Tanya and small Anton. He grew significantly after I saw him last time half of a year ago. Weight was changed noticebly and, while looking from his back, you can not see his cheeks anymore. It was nice time to walk with them, although small Abr showed his character sometimes...

:: Link / Comments (0)


Sat, 22 Apr 2006

The door.


Grange told me someday, that when there is a door, half of the house is ready.
Well, maybe he was right...

:: Link / Comments (0)


Quotation of the day: I claim that Mach people (and apparently FreeBSD) are incompetent idiots.


On Thu, 20 Apr 2006, Linus Torvalds wrote:
>
> I claim that Mach people (and apparently FreeBSD) are incompetent idiots.

I also claim that Slashdot people usually are smelly and eat their
boogers, and have an IQ slightly lower than my daughters pet hamster
(that's "hamster" without a "p", btw, for any slashdot posters out
there. Try to follow me, ok?).

Furthermore, I claim that anybody that hasn't noticed by now that I'm an
opinionated bastard, and that "impolite" is my middle name, is lacking a
few clues.

Finally, it's clear that I'm not only the smartest person around, I'm also
incredibly good-looking, and that my infallible charm is also second only
to my becoming modesty.

So there. Just to clarify.

	Linus "bow down before me, you scum" Torvalds
Linus Torvalds on COW page table manipulations and FreeBSD ZERO_COPY_SOCKET.

When I created zero-copy kernel sniffer, bechmark showed that such page table manipulations can significantly improve performance, but Linus is right that not every usage case will win with such approach.
So in receiving zero-copy support I decided not to play in virtual memory games (like FreeBSD does) and preallocate some VFS pages and DMA data from NIC directly into them. Now it looks like I was right...

:: Link / Comments (0)


Fri, 21 Apr 2006

Climbed a lot today.


Today I held my personal record - 5 vertical traces without rest between them. Well, it was not that hard traces, four of them were just 5c and only one was complex old 7a trace. Combine that with several 15 minutes wall-runnings and now you can imagine how I feel myself.
In case you have not guess, I feel myself excellent.

:: Link / Comments (0)


Acrypto hacking.


Changed IV processing in ESP4 acrypto port - it used the same IV for each message even if it was changed during crypto processing. Such behaviour is allowed by protocol, but it has lesser security protection, so I resolved this issue.

:: Link / Comments (0)


Thu, 20 Apr 2006

To be or not to be? That is a question...


Have you ever asked yourself, is your current lifestyle what you wanted? Maybe something could be changed, just as a new hobby for example.
Only consider, you are getting up, going to shower, get your clothes, going outside and ... plant the tree. Or build a house.
I'm just thinking, what if I start completely new development from scratch? The more I think, the more I want to do the whole that cycle by myself, maybe excluding some really professional tasks.

:: Link / Comments (0)


Wed, 19 Apr 2006

Excellent climbing time.


It was really good, several traces - old and almost new, a lot of wall-runnings and traverses, good weather and, and other good things.
Have a nice day.

:: Link / Comments (0)


Kevent based AIO sendfile.


Noone need an excuse to rewrite something.
So I decided to change aio_sendfile() design. It will not significantly change from what was described before, exception is that repeatedly called from keventd context kevent's main processing function will populate pages into VFS cache, and then send them over the net without temporary caching them in kevent's private structure.

Small note: do not mix kevent and keventd. The former is a mechanims of delivering edge and level triggered events to userspace, the latter is a kernel thread in which context workqueues work.

So with aio_sendfile() I just move the work from userspace process' context to keventd context and slighlty change the way pages are inserted into VFS cache and processed. Yes, I'm lazy boom slacker, but it is the simplest way to combine completely different execution contexts of network, VFS and block layer processings.

:: Link / Comments (0)


Tue, 18 Apr 2006

New friend - Vriesia splendens.


I call this "Vrez".
vriesea.jpg

:: Link / Comments (0)


Mon, 17 Apr 2006

New trace!


There is new interesting trace in skala-city built recently. It is called "do not want", i.e. "ne hochu", and it is disposed in right central sector over black holds.

:: Link / Comments (0)


Sat, 15 Apr 2006

New acrypto release.


New acrypto combined patch for 2.6.16 kernel tree has been released, which includes asynchronous crypto layer acrypto, dm-crypt, ESP4 IPsec processing and software crypto provider ported to acrypto. Patch and other acrypto drivers are available in archive.

:: Link / Comments (0)


Quarter of the century though...

:: Link / Comments (0)


Meet with old friends.


I decided to celebrate middle of the spring and called old friends to "5 oborotov".
It was a good time with Fedor, Ira, Evgeniy, Alexandra, Tatiana and Alexander. Hey, I wish Abr and Mephody were there too, but they will return probably only in May.
Thank you, friends.

:: Link / Comments (0)


Fri, 14 Apr 2006

Climbing.


Grange got his (shaved to bald) head out of slackswamp and we went to Skala-city. Since he does not know how to get to the meeting in time, I climbed one hour along. Several wall-runnings and a bit more complex traverses, that was good warm-up. Then I climbed several old traces, some of them, which are more simple, were done without the rest for endurance improving. I hope my climbing progress does not stay at one place.
It was really good time there.

:: Link / Comments (0)


Acrypto hacking.


I've fully ported 2.6.16 ESP4 IPsec processing to acrypto.
I will stress test it tomorrow and if things will go smooth, I will release new combined patchset and post this news to linux-crypto@ and linux-kernel@ mail lists.

:: Link / Comments (0)


Thu, 13 Apr 2006

I love my country.


There is one official for 100 people in Russia.
Level of income (after all taxes paid) for officials is increased about 30% per year, while for normal people it is about 9% per year.

:: Link / Comments (0)


Wed, 12 Apr 2006

Acrypto hacking.


I've released new acrypto combined patch for 2.6.15 kernel tree, which fixes IPsec ESP4 tunnel mode processing and initialization dependency on connector when acrypto is built statically.
Many thanks to Yakov Lerner for testing.
Patch is available in archive.

New standalone acrypto source released. It is a sync with combined patch, so it only includes resolution of dependency on connector when acrypto is built statically. Tarboll is available in archive.

Main work is concentrated on 2.6.16 IPsec port.

:: Link / Comments (0)


Climbed a lot today.


While Grange is slackassing I went to climbing zone to have several wall-runnings. Several hundreds of meters on holds with only litle time for rest - that was really hard, but it was extremely good.
I tired as hell, every muscle is aching, legs do not bend...
Then hot shower and world becomes almost perfect.

:: Link / Comments (0)


Mon, 10 Apr 2006

Climbed a lot today.


Since Grange is a lazy boom slacker, he did not go climbing for a while already, so I decided to train for alone.
It was day of traverses - several dozens of simple traverses around the climbing zone without a rest, then about ten minutes in the chair and do it again - after two hours of such a training I was completely wet, squeezed and tired as hell. And that was really good. I'm thinking about adding some weight to hands and legs and repeat that wall-runnings regulary.

:: Link / Comments (0)


Before and after.


This essay is being written under impression of past events.
People always like to say that it is much better somewhere where they never were, either it is in different geographical position, or somewhere in the past or maybe future. According to geographical position I always like to live in Russia, although there are a lot of things inside people minds which I would like to see changed.
But I really think that golden era of mankind as thinking, sentient being has gone already.
I never met a man who knows 5 or more foreign languages, but almost every intelligent person knew at least three 100 years ago.
I never read any modern author which would write like Shakespeare (although it was really hard to read for me in original, so I stopped, but even translated it is incredible), Pushkin or Lermontov.
I never read any modern philosopher which had so deep ideas like antic thinker.

But what I see and hear every day is always submitted to one simple law: mass-production.

People produce more and more stuff which has worse and worse quality, so people would throw this and get new one, which will be thrown later for the new one.
No, number of really talented people does not decrease, but it becomes harder and harder to break away from the swamp. So we do not see a man who knows a lot of languages, since this requires a lot of time, which must be spent in mass-production cycle to get some money. We do not have gigantic thinkers, since mass-production does not require this. And new talents see that everything around them is subjected to grey mass-production law, and this deprives of the will.

:: Link / Comments (0)


Fri, 07 Apr 2006

Climbing.


I did not write about my climbing tranings for a while.
There is actually no major progress, and it looks I found why.
I do have not so bad climbing technique, but physical endurance is not that good. So on vertical walls I can climb quite good traces including 7a, but with negative slopes I fail to finish complex traces without handing up. So training, trainig and training again.

I climb in Skala-city, where new sauna was built recently instead of previously used turkish bath. I definitely like vapour bath more than dry sauna, but no filter could handle those quality of water, so turkish bath did not function a lot of time and was finally replaced.

:: Link / Comments (0)


Thu, 06 Apr 2006

Acrypto hacking.


Fixed bug in tunnel mode ESP processing. I'm currently trying to sync asynchronous IPsec processing with 2.6.16 tree, which is quite challenging, since XFRM engine was changed a lot since 2.6.15 time.

There is a problem with asynchronous input IPsec processing in 2.6.16.
Linux had stackable dst entry processing for input and output, which was very convenient to use with dst_input()/dst_output(). Now it is gone.
Output processing of the dst stack was moved into XFRM engine from generic dst_output(), since there is only one in-kernel user. And it is XFRM. So currently asynchronous IPsec processing is not possible with 2.6.16 until some other stackable technique will be reinvented, like it is done for asynchronous IPsec input processing.

:: Link / Comments (0)


Wed, 05 Apr 2006

Netlink performance testing.


For those of you who do not know if it is needed or not to switch to/from netlink, I've created simple set of benchmarks which shows performance of _blocking_ data transfer from kernelspace to userspace in case of using unicast and broadcast.
There are several tests for different number of listening users (1, 2, 4 and 10) and different message size (128, 512, 1024 and 4096 bytes). 10k messages are transferred from process' context (keventd) in one loop. For unicasting this number of messages is multiplied by number of listening users, so each user receives 10k messages.

Messages are sent using blocking technique, since otherwise socket queue with default length of 200k is filled before the first rescheduling, so it is not allowed to send a lot of events without rescheduling using nonblocking methods (netlink_broadcast() for example).

While unicast easily allows blocking send, broadcast does not, so I created simple patch to implement this functionality, which is only usefull for testing, since netlink table is locked during test. Performance testing module and userspace tool are available in archive.

Test was run on 2.4 Ghz Xeon (HT enabled) with 1Gb RAM with 2.6.15-rc6 kernel.
Netlink performance

From this test one can see, that unicasting is faster for 1 and maybe 2 users, and broadcast definitely wins when number of users increases.

:: Link / Comments (0)


Mon, 03 Apr 2006

AIO sendfile.


Let's briefly describe how generic reading is being done in Linux kernel.
When chunk of file data is not in VFS cache, it is requested using block layer, which bio->bi_end_io callback, generally mpage_end_io_read(), is called in hard IRQ context and only marks pages as uptodate or error and unlock the pages.
Generally cold pages from VFS cache, i.e. pages which were not used, but inserted recently, are used for this technique.
Network processing code must wait on those pages before it can start use them until pages are marked as uptodate.
AIO sendfile's state machine is invoked in bio->bi_end_io callback, in which it basicaly does the same as mpage_end_io_read() plus schedules reading of the next chunk of file data.
Let's compare what happens when we call do_generic_file_read() and how kevent based AIO sendfile works.

  • do_generic_file_read() runs through all pages which would contain data from requested region of the file, and if file data is not in VFS cache, it invokes block layer through mapping->a_ops->readpage(), which is likely mpage_readpage() or block_read_full_page(), and then wait on VFS pages to become ready. Selected VFS page is then provided to specified from higher layer actor function, which actually copies data or send it over the net.
  • Current AIO approach does almost the same. It has kevent which has array of preallocated clean pages, which are used either for actor function, which is used for uptodate pages found in VFS cache; or for block layer mechanism, which is very similar to mpage_readpage(), but it's bio->bi_end_io not only marks pages as ready, but also invokes kevent state machine. Then new work is scheduled in callback invoked from kevent state machine, which will process kevent's pages and then will start reading of the next chunk of data.

So, what is the difference between the two approaches?
Basically, synchronous buffered reading handles data in page sized chunks, and if there are no data in VFS cache, it allocates new page, inserts it into VFS cache, and waits until data is read there. Then VFS page is procesed by higher layer actor function. Synchronous buffered reading is never interrupted (I mean do_generic_file_read() does not exit except on errors) between different page processing.
AIO approach works similar, but it does not populate page read into VFS cache, uses bigger set of pages and is interrupted each time given number of pages has been processed, i.e. only predefined number of pages in kevent are processed in a time, while buffered reading process all requested data.

Since AIO processing happens in work queue, we should add here an overhead for process switching after predefined number of pages in kevent has been processed.

:: Link / Comments (0)


New w1 release.


It only includes sync with in-kernel tree, which introduces new locking primitive called mutexes instead of semaphore, so it can only be compiled with 2.6.16+ kernels, which is reflected in README. This release does not contain new functionality.

:: Link / Comments (0)


Sun, 02 Apr 2006

Weekly cooking madness.


Today I decided to cook up some mushroom dish. I've chosen simple recipe of julien of champignons.
Not bad, but I expected more, do not even know why. It has a taste of mushrooms in sour cream sauce with some slight mix of cheese. Nothing outstanding really, but this is matter of taste I think.

:: Link / Comments (0)


Sat, 01 Apr 2006

AIO sendfile support.


Initial support for kevent based AIO sendfile has been implemented.
Although I can not say it is sendfile, since nothing is sent yet, but only the whole file content is being read in preconfigured chunks, which in turn will be sent using network asynchronous IO mechanism.

It is quite a big step, since the whole last week has been spent in suspended state: from one side I have a paid job, where some time might be spent to complete requested features, but that required a lot of steps from quite a few people, so actually nothing was done, and from the other side I have my own projects, like kevent based AIO, acrypto, w1 and others, which require full attention on them, but not just two minutes view.

:: Link / Comments (0)


Interesting language transformations.


"Nihilo" in latin means nothing, while in russian pronunciation it is fun synonym for "a lot".

Russian language belongs to Slavic language group which in turn is part of Indo-European language group, which includes dead latin besides others.

:: Link / Comments (0)