About TODO Blog RSS Old blog Projects Gallery Notes

Thu, 31 May 2007

I've gotten a first Linux dmesg on SuperH board.


What do you expect? Maybe this one:

Starting from MBR
Jumping to second stage
START
input_len: 0x00 10 ba f6
Uncompressing Linux... 

ran out of input data

 -- System halted

/devel/sh :: Link / Comments (0)

What is Linux Kernel Summit in real.


As Jens Axboe (besides other things he is a KS committee member) said to me:

The kernel summit is around 70-80 people, core people is probably around 60-70. The kernel summit is not a conference as such, it's a place to sit and talk about various issues. People don't give slide presentation (the exception here being some corporate sponsors or invited corporate people), the intention is to have a 5 minute intro spark a conversation on the selected topic.

/devel/other :: Link / Comments (0)

My first SuperH patch has been applied.


It fixed most of the warnings/errors I found during custom config, and is pretty trivial. Patch includes spinlock fixes, read/write lock fixes, SMP build fixes and include mess. Well, word 'fixes' likely is not good enough, but it allows to compile the tree.

/devel/sh :: Link / Comments (0)

Wed, 30 May 2007

Climbing evening.


This was easy training - climbing when temperature is around +30 degrees Centigrade is not that pleasant thing, but I do want that. So being choked with a sweat, with melted brain and skin I managed to complete several old traverses and even started a new trace (read: start of the trace, first several holds, which are lower than 3-4 meters), but since it was not finished, it was quite difficult to complete. So most of the training was spent drinking warm water, sitting and siscussing the weather.

Friday-saturday is expected to be interesting - there will be cyclone changes, so there is a possibility to have a tornado in Moscow. And I do not even have a photocamera.

/life :: Link / Comments (0)

2007 Linux Kernel Summit.


I'm about to attend this meeting, although, frankly, I do not know, what to do there.
I do not know anyone there, unlikely that I will participate in discussions about filesystems or virtualization (which, it looks so, are the hottest toppics).
Kevent is in the topic list (several months ago, before eventfd was committed into the tree, it was on the main board), but it is unlikely I will cook up a presentation about it (politics is boring, so I will not dig into that swamp).

Networking presentation might be interesting (at least for me), so I could create something to show how kernel misuse netlink at startup (millions of small messages sent during startup), how I see unified socket storage being a good approach, how memory power-of-two overhead ends up eating tons of memory in network path, I could even recall netchannels and related problems.
Hardware crypto is being in the presentation topics, so I could dig there too, but my opinion will not be 100% without bias :)
I also have quite a few interesting ideas related to filesystem development, but until something is ready, that is not a right way to handwave...

Although I've created a first slide for any presentation, which could be done by me:
... Name of the presentation ...
Run away - author is not a native english speaker.

We will see. I'm gathering info about Cambridge and kernel summit itself.

Thinking more about above stuff, it looks like presentation would be too technical, and thus it is better to handle it via e-mails. What is the hottest topic in the e-mail lists? That should be discussed at kernel summit, not how to create yet another good or bad feature for Linux kernel. So, it looks like politics will be the hottest theme. Another good, from my point of view, presentatino topic is iteraction with userspace, especially if something will be beaten there - people like to see such things, and obviously userbase of such changes is much wider.

/devel/other :: Link / Comments (0)

Tue, 29 May 2007

My small SuperH bootloader.

SH IPL+g version 0.9, Copyright (C) 2000 Free Software Foundation, Inc.

This software comes with ABSOLUTELY NO WARRANTY; for details type `w'.
This is free software, and you are welcome to redistribute it under
certain conditions; type `l' for details.

2002/09/09 Making.  2004/09/08 I-O DATA NSU Update.
266:133:33 on base clock 22.22MHz and SDRAM 4 burst. CF boot.

PCIC initialization done.
MASTER:48bit LBA mode non support
Disk drive detected: LEXAR ATA FLASH V1.00 11014102039199095066 
LBA: 001EBF10
DiskSize: 1031675904Byte
PIO MODE1
Set Transfer Mode result: 50 
> b
Set Transfer Mode result: 50 
Initialize Device Parameters result: 50 
IDLE result: 50 
Starting from MBR
SECOND LOADER

read_sector:
	mov	#2, r0			! READ SECTORS
	mov	r13, r6			! destination address
	mov	#1, r5			! looks like source address (source sector number)
	mov	#0, r4			! device number (?)
	mov	#1, r7			! number of sectors
	trapa	#0x3f

	tst	r0, r0
	bt	second_loader

	! Error
	mova	message_err, r0
	mov	r0, r4
	mov	#6, r5
	mov	#0, r0			! Serial Output
	trapa	#0x3f

second_loader:
	mov	#6, r0			! Cache "on"
	mov	#0, r4
	trapa	#0x3f

	mov	r13, r0
	jmp	@r0
	nop

/devel/sh :: Link / Comments (0)

OpenBSD mbufs and tags.


They have been committed a major change for mbuf PF processing - *BSD stacks used to use tags behind mbuf to store various additional information related to packet processing, that memory is allocated in addition to quite big header, which is very slow. With above change PF related processing information is stored in mbuf structure.
That doubled PF performance.

I'm trying to recall how many years Linux does not use tags in skbs and stores everything in skb structure itself.

/devel/other :: Link / Comments (0)

Beer drinking on streets is officially forbidden in Russia.


Law was accepted in Duma with only one voice against and one refrain. Likely the latter overslept and the former confused the buttons.

You probably know, I live in country, where any government which tried to regulate and limit strong drinks finished its life in pain and sorrow.

/other :: Link / Comments (0)

I have been invited to 2007 Linux Kernel Summit.


It will be hosted in in Cambridge, UK, 5-6 September.

Thinking...

/devel/other :: Link / Comments (0)

Mon, 28 May 2007

Sector reading code.


Ok, I've written a code which resides in MBR and reads a sector into the RAM and jumps there, but it only works if I read the same zero sector - I get recursive call of the same .start routine, which prints "Starting from MBR" to the serial console, but if I change source address, system freezes. Likely this is because of wrong sector->LBA translation (I need second or third sector, but I must provide an LBA address), at least if I blindly use 2 for the third sector, it reads something different than third sector. SuperH LILO source I have use some tricky asm to convert from one system to another, which I do not yet understand, and it also uses some strange constants (likely hardcoded address of the second stage loader), so it is not very useful source of information.
Thinking...

/devel/sh :: Link / Comments (0)

I know kung-fu or how to write your own bootlaoder.


I never did that before, so it is completely new task for me.
My testing system is SuperH board, which already has initial bootloader, which jumps into MBR of the compact flash for execution. I found LILO port for SuperH CPU, but it does not work (and all documentation is in japanese, and google can not translate that page), which required very old LILO versions, since recent ones (22 and higher) just write x86 boot sector into MBR no matter what you ask it to do (i.e. what boot.b is being used). So obvious step from my point of view is to write own bootloader.
"How is it supposed to work?" - I asked myself and found an answer - it will be pretty small stuff without several stages like in LILO or GRUB, my bootloader will just read fixed size Linux kernel image from fixed offset into memory and jump into it, it is quite simple task since the whole initialization and helper code is already written and stored on the NAND flash in IPL (Initial Program Loader).
My code, which starts from MBR and prints greeting message into serial port for that board is very simple (not including initialization of the base registers):

	mova	message, r0
	mov	r0, r4
	mov	#17, r5
	mov	#0, r0
	trapa	#0x3f

/devel/sh :: Link / Comments (0)

OpenBSD c2k7 hackathon. Day two.


BBQ day or playing "shake and suck" as Grange describes - this is a game where each player (can be infinite number) gets a bottle of some crappy beer and starts to shake it trying to suck at least something. There is no winner, but everyone becomes filled with beer and very happy.
BBQ was cooked (as expected) by Bob Beck (This time without "Will cook for sex" motto on the apron).

One can check second day photos.

/devel/other :: Link / Comments (0)

My first boot on SuperH CPU.


Serial console output:

SH IPL+g version 0.9, Copyright (C) 2000 Free Software Foundation, Inc.

This software comes with ABSOLUTELY NO WARRANTY; for details type `w'.
This is free software, and you are welcome to redistribute it under
certain conditions; type `l' for details.

2002/09/09 Making.  2004/09/08 I-O DATA NSU Update.
266:133:33 on base clock 22.22MHz and SDRAM 4 burst. CF boot.

PCIC initialization done.
MASTER:48bit LBA mode non support
Disk drive detected: LEXAR ATA FLASH V1.00 11014102039199095066 
LBA: 001EBF10
DiskSize: 1031675904Byte
PIO MODE1
Set Transfer Mode result: 50 
> b
Set Transfer Mode result: 50 
Initialize Device Parameters result: 50 
IDLE result: 50 
I'm booting from MBR!

/devel/sh :: Link / Comments (0)

Do you know that Einstein was offered to become an Israeli president in 1952?

/other :: Link / Comments (0)

More OpenBSD c2k7 hackathon photos.


Here.

/devel/other :: Link / Comments (0)

Sun, 27 May 2007

Linux SuperH SMP support.


I do not know how it can ever be compiled:

...
arch/sh/kernel/smp.c:182: undefined reference to `__smp_call_function'
...

$ grep __smp_call_function -r include/asm-sh* arch/sh* | egrep -v "Binary|~"
grep: warning: include/asm-sh/mach: recursive directory loop

arch/sh/kernel/smp.c:void __smp_call_function(unsigned int cpu);
arch/sh/kernel/smp.c:                   __smp_call_function(i);

/devel/sh :: Link / Comments (0)

Sat, 26 May 2007

OpenBSD drink^Whackathon. Day one.


Grange's first day there.
First photos.

As far as I can tell, Linux does not have such friendly meetings. Maybe netconf is similar, but not Linux kernel deveopers summit for sure.

/devel/other :: Link / Comments (0)

Fri, 25 May 2007

Climbing evening.


It was OFP (in russian) or GPT (in english) - general physical training (obshaya fizicheskaya podgotovka). I do it in a bit differnet manner, instead of fixed amount of simple exercises I try to comple as much as possible in fixed time interval. This training was second one devoted to GPT, but I think result is good. For 1.5 hours I managed to complete quite a lot of exercises, but each one was completed about 80 times (8 rounds of 10 in each turn, which contains of 7 different exercises with small weights (or without) for different groups of muscles), although I wanted to complete 100 (10 rounds).
Main goal of such exercises is to increase my power endurance, since on negative slope I behave too miserable (well, not that bad, but I do want to do better).
I also bought myself new shoes, which exeeds my budget a bit, but I expect a good result from them. This time it is 'La Sportiva Viper'.

/life :: Link / Comments (0)

Another idea about captcha solving.


And this time it is not based on any graphic metrics.
All previous methods are similar to how small child plays with a cube, which has inlets of different forms and set of objects, each one matches only one form of inlet. Child gets objects and tries to push them into the cube via different inlets until succeded. But reading from the paper is different, instead let's recall how we were trained to draw letters and digits - draw line from one point to another, then rotate it to some degree and draw another line or arc and so on.
That is a base for my new method - each image is converted into set of control points (start of the lines, crosses (questionable though), rotates) and set of rules of how line between control points must be drawn. If that set of rules matches in some metric to database set of rules, then letter is found.
It is purely theoretic so far, but I already see how it can solve some issues with all methods I used before, but it is possible, that there will appear complex problems. For example it is not 100% clear to me how to create set of control points from given image automatically, but I have some ideas.

/devel/captcha :: Link / Comments (0)

Thu, 24 May 2007

A very small progress of SuperH booting today.


I've compiled RedBoot and about to compile SuperH Linux kernel, but since I do not have Compact Flash reader I can not reflash an image, so I've stopped for now.
Tomorrow I will get a flash reader and continue, likely (unfortunately) it will not be that hard to start a linux kernel on that board, since it is well supported.

Update: SuperH kernel for Hitachi SuperH SH7751R CPU can not be compiled with SMP support. This CPU can not be put into SMP system, but it is not a reason to fail building, isn't it? There is a default config for landisk board in Linux kernel, but it is not interesting to use it, so I'm playing with own set of options.
Update2: SuperH really lacks SMP support. Check this atomic increment instruction:

static inline void atomic_add(int i, atomic_t *v)
{
	unsigned long flags;

	local_irq_save(flags);
	*(long *)v += i;
	local_irq_restore(flags);
}

static inline void __raw_read_lock(raw_rwlock_t *rw)
{
	__raw_spin_lock(&rw->lock);
	atomic_inc(&rw->counter);
	__raw_spin_unlock(&rw->lock);
}

/devel/sh :: Link / Comments (0)

OpenBSD firmware on landisk SuperH board.

SH IPL+g version 0.9, Copyright (C) 2000 Free Software Foundation, Inc.

This software comes with ABSOLUTELY NO WARRANTY; for details type `w'.
This is free software, and you are welcome to redistribute it under
certain conditions; type `l' for details.

2002/09/09 Making.  2004/09/08 I-O DATA NSU Update.
266:133:33 on base clock 22.22MHz and SDRAM 4 burst. CF boot.

PCIC initialization done.
MASTER:48bit LBA mode non support
Disk drive detected: LEXAR ATA FLASH V1.00 11014102039199095066 
LBA: 001EBF10
DiskSize: 1031675904Byte
PIO MODE1
Set Transfer Mode result: 50 
> b
Set Transfer Mode result: 50 
Initialize Device Parameters result: 50 
IDLE result: 50 

OpenBSD MBR

OpenBSD/landisk Primary Bootstrap
>> OpenBSD/landisk BOOT 0.99
boot> 
booting cf:/bsd: 2690340+264524 [72+128848+115558]=0x30d35c
[ using 244972 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2006 OpenBSD. All rights reserved.  http://www.OpenBSD.org

OpenBSD 4.0-current (GENERIC) #9: Thu Nov  9 14:49:28 MST 2006
    root@landisk.openbsd.org:/sys/arch/landisk/compile/GENERIC
I-O DATA USL-5P
real mem = 67108864 (65536K)
avail mem = 58761216 (57384K)
using 844 buffers containing 3457024 bytes (3376K) of memory
mainbus0 (root)
cpu0 at mainbus0: HITACHI SH4 266.666 MHz PCLOCK 33.333 MHz
cpu0: 8KB/32B direct-mapped Instruction cache.
cpu0: 16KB/32B direct-mapped Data cache.
cpu0: P0, U0, P3 write-through; P1 write-through
cpu0: full-associative 4 ITLB, 64 UTLB entries
cpu0: multiple virtual storage mode, SQ access: kernel, wired 61
shb0 at mainbus0
scif0 at shb0
scif0: console
rsclock0 at shb0: RS5C313 real time clock
shpcic0 at mainbus0: HITACHI SH7751R
pci0 at shpcic0
re0 at pci0 dev 0 function 0 "Realtek 8139" rev 0x20: irq 5, address 00:a0:b0:6c:d0:eb
rlphy0 at re0 phy 0: RTL internal PHY
ohci0 at pci0 dev 2 function 0 "NEC USB" rev 0x43: irq 7, version 1.0
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: NEC OHCI root hub, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1 at pci0 dev 2 function 1 "NEC USB" rev 0x43: irq 8, version 1.0
usb1 at ohci1: USB revision 1.0
uhub1 at usb1
uhub1: NEC OHCI root hub, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
ehci0 at pci0 dev 2 function 2 "NEC USB" rev 0x04: irq 5
usb2 at ehci0: USB revision 2.0
uhub2 at usb2
uhub2: NEC EHCI root hub, rev 2.00/1.00, addr 1
uhub2: 5 ports with 5 removable, self powered
obio0 at mainbus0
wdc0 at obio0 port 0x14000000-0x1400000f irq 10
wd0 at wdc0 channel 0 drive 0: 
wd0: 16-sector PIO, LBA, 983MB, 2014992 sectors
wd0(wdc0:0:0): using BIOS timings
boot device: 
rootdev=0x1000 rrootdev=0x1000 rawdev=0x1002
WARNING: / was not properly unmounted
Automatic boot in progress: starting file system checks.
/dev/rwd0a: 7631 files, 139253 used, 355342 free (54 frags, 44411 blocks, 0.0% fragmentation)
/dev/rwd0a: MARKING FILE SYSTEM CLEAN
setting tty flags
starting network
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 1
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 1
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 2
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 2
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 2
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 2
DHCPDISCOVER on re0 to 255.255.255.255 port 67 interval 2
DHCPOFFER from 192.168.0.1
DHCPOFFER from 192.168.0.1
DHCPOFFER already seen.
DHCPREQUEST on re0 to 255.255.255.255 port 67
DHCPREQUEST on re0 to 255.255.255.255 port 67
DHCPREQUEST on re0 to 255.255.255.255 port 67
DHCPREQUEST on re0 to 255.255.255.255 port 67
DHCPREQUEST on re0 to 255.255.255.255 port 67
DHCPACK from 192.168.0.1
duplicate IP address 192.168.0.205 sent from ethernet address 00:11:2f:d3:ed:6a
bound to 192.168.0.205 -- renewal in 3600 seconds.
starting system logger
starting initial daemons: ntpd.
savecore: /bsd: kvm_read: version misread
checking quotas: done.
building ps databases: kvm dev.
clearing /tmp
starting pre-securelevel daemons:.
setting kernel security level: kern.securelevel: 0 -> 1
preserving editor files
/etc/rc[515]: /usr/libexec/vi.recover: No such file or directory
starting network daemons: sendmail inetd sshd.
starting local daemons:.
standard daemons: cron.
Thu May 24 12:00:53 EDT 2007

OpenBSD/landisk (landshark.westerback.to) (console)

login: 

/devel/sh :: Link / Comments (0)

My workplace things.


PPC32 and SuperH boards.



Closer look and my workplace photo with descriptions can be found in album.

Direct link (1 Mb) (preview) to dual-head workplace. Actually I do not see that two monitors help me more than one in development, but I needed to get something from my boss, when we made a bet if it is possible to write a ppc 405 gpr watchdog driver in one day instead of one week with full testing and 'soft' reset configuation. It is possible, task was pretty simple, so now I have two monitors, although only see a gain when watching a movie on one desktop and browse or readmail on another...

Anyway, I always liked to see different dmesgs and workplace photos, so feel free to drop a link.

Update: First time I booted 2.6 kernel on above PPC board almost two years ago.

/devel/other :: Link / Comments (0)

Wed, 23 May 2007

I am about to become a SuperH hacker.


While Grange moved to OpenBSD Drink^WHackathon I got his Landisk SuperH board with Hitachi/Renesas SH-4 CPU based IO-DATA USL-5P board.
Hitachi SH4 SH7751R processor is well supported in Linux (at least according to linux-sh page), but I want to install it myself. This board has RedBoot preloaded, which should be enough to boot Linux kernel.
There are interesting features not being used in kernel so far, so I have some bits to play with.

/devel/sh :: Link / Comments (0)

Tue, 22 May 2007

Climbing evening.


This was quite simple training, I completed several old traverses, then couple of boulderings and starts, eventually after 2.5 hours Grange arrived, so at the end I tried one or two interesting traces, but my shoes are not supposed to anyhow help in climbing, so it was complex and I failed. For reference: my shoes.



And bigger photos: one and two.

I definitely need new pair...

/life :: Link / Comments (0)

Preliminary HIFN 795x (7955/7956) crypto accelerator chip driver for mainstream 2.6 tree.


I've just released a preliminary HIFN 795x driver for mainstream crypto api stack. Driver is based on my work for acrypto asynchronous crypto stack, where it worked correctly, but this one was only compile teted, since I do not have hardware right now. Will do all tests when it arrives.
I've sent to linux-crypto@ for comments mostly about cryptoapi bindings, since there are several issues (I was not agree with when Herbert designed them), which I might implement wrong in the driver.

Due to problems with interrupt storms and possible adapter freeze (sorry, but HIFN spec I have really sucks, so likely it is programming error, but who knows) I added special watchdog, which fires if after predefined timeout sessions which are supposed to be completed are not. In that case callback is invoked with -EBUSY error. Neither implementation I checked (OpenBSD, Linux OCF) does not support it, but it helped me greatly in acrypto days.

Patch is quite big (61k) and contains a lot of debug stuff, but it is only preliminary work.

So far, patch against 2.6.21 can be downloaded from archive.

/devel/acrypto/hifn :: Link / Comments (0)

Mon, 21 May 2007

Climbing eventing.


It has happend. I feel myself like a man with capital letter M.
I climbed on the walls and not with boring traverses, since lazy slackass Grange finally took himself out of laziness swamp and moved to climbing zone.
That was really good trainig - I started with old traverses - I managed to complete about 5 or 6 before he arrived (yes, punctuality is not main goal for some people), so I started to climb up. I completed two new 6b traces, although I failed them on-sight, but I think only because I'm still not in the perfect shape and my shoes are so broken, that big finger appears in the wild.
I even managed to complete old 7a, but with fails too.
Give me a week and all that traces will be finished correctly.

/life :: Link / Comments (0)

Sun, 20 May 2007

First Bezier curve.


Here is a result:



Bezier curve construction is very sensitive to the order of the control points, since resulted Bezier curve is a weighted sum of Bernstain polynomials, which do not depend on control points, but each polynomial is multiplied with the single control point, so mixing them results in completely different picture, while pixel image will still be the same.

As a conclusion, it is not possible to use raw pixel data obtained from captha images, instead they must be converted into ordered set of points, which represents control steps: end points, crosses and rounds.
When this task is solved, it will be quite simple to construct a letter using several Bezier curves and start working with it.
Back to drawing board...

/devel/math/bezier :: Link / Comments (0)

Where is a bug in this code?

#include 

int main()
{
	float u;
	for (u=0.0; u <= 1.0; u += 0.1) {
		printf("%f ", u);
	}
	return 0;
}

0.000000 0.100000 0.200000 0.300000 0.400000 0.500000 0.600000 0.700000 0.800000 0.900000

0.000000 0.200000 0.400000 0.600000 0.800000 1.000000

	for (u=0.0; u <= 1.0;) {
		u += 0.1;
		printf("%f <= %f : %d\n", u, 1.0, u<=1.0);
	}

0.100000 <= 1.000000 : 1
0.200000 <= 1.000000 : 1
0.300000 <= 1.000000 : 1
0.400000 <= 1.000000 : 1
0.500000 <= 1.000000 : 1
0.600000 <= 1.000000 : 1
0.700000 <= 1.000000 : 1
0.800000 <= 1.000000 : 1
0.900000 <= 1.000000 : 1
1.000000 <= 1.000000 : 0

...
0.063200 <= 1.000000 : 1
0.063300 <= 1.000000 : 1
0.063400 <= 1.000000 : 1
0.063501 <= 1.000000 : 1
       ^

/devel/other :: Link / Comments (0)

Sat, 19 May 2007

Bezier curves.


For those, who wonders (like me) about vector transformations, I can recommend to start learning Bezier curves math. It is pretty simple, but too powerful to believe. For example I always wondered how to approximate a circle or ellipse, using Bezier curves and derivatives it becomes quite usual task of approximation.
This is only because of two things, first one is that Bezier curves are parametrized curves and not functions in usual presentation (but superposition of basis set of usual functions called Bernstein polynomials), so it can have several results for the same argument (with different parameter value). Second feature of this curves is that they are fully defined via its control points, so scaling and other affine transformations become trivial.
Building a result for given inputs is a bit more complex than with usual functions, but using de Casteljau's algorithm it is not a big problem too.

I will use b-splines to approximate database letters and then transform them to match requested image to solve captcha problems. Probably I will create a math ground for such comparison, but maybe I will just limit myself with simple 'brute force'.
Even if it will fail, I will create a eye-candy library to create vectorized images (pretty simple, but that is enough for now).
Stay tuned, maybe tomorrow I will have something to play with, since I devoted the whole day reading various math theories behind Bezier curves and b-spline approximation, so I have some (quite small of course, but yet powerful) ground to start with.

/devel/math/bezier :: Link / Comments (0)

Fri, 18 May 2007

Climbing evening.


I decided to devote this day to power endurance trainig, and tried to achieve all parts being aching. I only partially succeeded, although spent about two hours trying hundreds of various exercises. Since I did it in small quantities, eventually it bacame boring, and I started to add weights, so at the end I Was quite tired, but forearms were still not loaded enough, so main aim was not reached, I could even complete a traverse.
Well, it was first attempt, next one will be harder.

/life :: Link / Comments (0)

Thu, 17 May 2007

Captcha problems.


For the last three days I tried different algos to solve this captcha:



As I described previously, first I tried to find number of crosses on the letter, but then found, that it is wrong approach, since frequently letter is crossed with small noise lines, so essentially number of crosses becomes completely unuseful information.
The last approach I decided to try is to present a letter as set of extrapolation functions, each one approximates points which are not placed closer than median error for previous functions. So it is somehow similar to how wavelet transformation works, where each new layer adds details to the picture.
But this approach fails miserably, the best thing I could get from it is to return that the closest to letter 'S' is '8', which is only remotely correct, but obviously wrong in general case.

So far I can not solve above captcha, but I have some additional idea, which is based on letter transformations I decribed previously. It will based on vector 'image' of the letter, i.e. I will create database of images created of lines only, each line will be a vectorized set of points, which can be moved. Searching algorithm will try to transform each database letter into requested one and check how many transformations required. It is possible that letter 'I' can be marked as similar to letter 'W' for example, since the latter can be created from the former using above transformations, but it will require more 'moves' than letter 'W' transformations.

This approach I believe is similar to how our brain works, and since there are no comments on my unified socket storage, I have some time to work with it.

For this topic I created captcha blog tag.

/devel/captcha :: Link / Comments (0)

Wed, 16 May 2007

Climbing evening.


That was great trainig, mainly because of two moments:

• I'm very tired, since intrusctrors removed several holds in the way, I used to make a traverse (second level of shields about 1-1.5 meters high), so not that simple one became quite complex, which sucked quite big part of my power.
• I've made a decision. It is always good to draw the line, all doubts go away, goal becomes clear and way to achieve it apeears immediately.
  Anyway, I decided to devote each third trainig to improving general physical endurance - I will make a lot of simple exercises like bobs, tightenings, liftings and others in small sets. I expect to be at least half-alive after completed, so I will start it Friday to have two days for the rest. Let's see, how this will end up.

/life :: Link / Comments (0)

Tue, 15 May 2007

Kevent/eventfd/pollfs discussion thread.


Here one can find a discussion I referred a week ago.
Actually I do not understand Davide Libenzi's claims that kevent is:

Monolitic and interface-centric solutions, or better, solutions in search of a problem

Yes, of course. If we're heading to yet-another monolitic interface, we're heading with no valid reasons given if other than some handwaving. While there are quite a few (modularity, compatibilty, plus the other ones that came in my mind and that I explained in the way-too-many emails) to back a file-based approach.

That's my point. I think we ultimately have to have something like kevent and then all this *fd() work is unnecessary and just adds code to the kernel which has to be kept around and which might hinder further work in this area.

/devel/kevent :: Link / Comments (0)

Mon, 14 May 2007

Stupid day...


Instead of going climbing I sat in office, I would understand if I did somthing interesting, but not I slackassed the whole day. Found that after service my laptop became more silent, but its wireless light works somehow broken (it never turns off, although bluetooth and wireless are turned on and off using the same key). I woke my laptop up after several month hibernate and ethernet stops working and can not be turned on even with power off/on cycle.
Tried to implement cross-detection logic, but found that to break for example Yahoo captcha protection cross dection is not enough, since image is randomly crossed with noise, which introduce additional crosses. As a side note created a bit more advanced detection of pattern in letters and crosses, which is based on approximation of the lines, which form the letter (or cross), maybe this will be completed tomorrow.
When tried to implement this idea, I found that a lot of my backups do not exist anymore, so I will try it from scratch. Although flushed away two days of work, since eventually I found that implementation of idea I worked on was not correct, actually this was the worst part of the day - throwing away something which has part of your time is not very pleasant step, but this is a ground for even better results.

Anyway, that was not that bad day - I printed Hoking's book about universe - will have something to read next couple of days.

/life :: Link / Comments (0)

Sun, 13 May 2007

Solving captcha problems. Rotation.


Two complex letters have been solved (letter 'N' was solved previously though) with addition of rotation around central point.




Complex letter 'F' failed:



As you can see, computer thinks that letter 'W' is the most similar letter, since it covers the maximum amount of letters, which is obviously incorrect from human point of view. Letter 'W' frequently wins just because it has too many pixels.

To solve remaining issues I plan to introduce algorithm, which will calculate number of crosses in the image. Letters with different number of crosses will be removed from the further calculations.
Actually this is simplified part of the more generic (and more complex) algo, which I have in mind. The latter should detect if it is possible to get resulted image with contiguous transformation of the lines, which form original one. It is absolutely sure that 'F' can not be obtained from letter 'W' without crossing its line with itself. But for now I will limit myself with only simplified part.

Cross-detection logic I will get from my old project to analyze pixmaps, find crosses, build a tree and then find the shortest path from one point to another.
Here is one screenshot of the middle stage of the development (will be opened with real sizes (1024x768) on-click):

/devel/captcha :: Link / Comments (0)

Sat, 12 May 2007

Some time ago we could fly...




Some time ago we could fly.
We just forgot it... But we will recall...
We will recall it absolutely...

Listening Mongol Shuudan - Moscow (Sergey Esenin's words).

/other :: Link / Comments (0)

TCP congestion avoidance algorithms.


Stephen Hemminger ran a simple tests to show how modern Linux congestion algorithms work, his graphs cane be found here.
He tested 1Mbit DSL link with a 100ms RTT.
As we can see from graphs, all they follow RFC, which forces to halve sending window in case of congestion (duplicate ack received by sender), created more than 10 years ago.
Speeds have changed hugely since that times...

So, I'm thinking about high-speed congestion control suitable for small RTT in modern ethernet networks, which will not halve sending window, but decrease it exponentially until decrease rate reaches 50% from current window size. Maybe word 'exponentially' scares a bit, so let me describe idea in more details.
Let's say sender has received a duplicate ack, which means that some segment was arrived not in order (either there is a misordering in network or likely some segment was lost), fast retransmit suggests to resend missed segment and then to halve a window, but what if sender will slightly decrease window, say for 10%, if rate is still very high, and new duplicate acks are being received, window is decreased by 20% and so on, until we reach 50% limit dictated by old RFC.
Actually such congestion control algorithm is implemented in my userspace network stack, but it is much smaller, since my stack does not support extended states like Linux kernel has (and a lot of congestion controls behind old Reno use them).

Needs to think about...

/devel/networking :: Link / Comments (0)

Strange looking man.


I've bought myself a big wrench (my dictionary suggests that word for device, used to twist nuts, although it also means dislocation, distortion and torture). Another word for that device is spanner, which is likely more appropriate.
Anyway, I've bought mysef that device to twist 'split terminal' (copied from dictionary, hope it does not have aditional meanings) for mettalic-plastic tubes. It is quite big (about 20-30 santimeters, its weight is about half of a kilogramm) steel device, which looks like heavy stick.
I do not know why, but people do look very strangely to high man in black clothes with such device.
Do I look like a criminal in the middle of the day in Moscow if I walk with big spanner?
Very strange. Especially getting into account, that I'm very kind, can not hurt even a fly or any other animal (until it is really bad), and wear a glasses.
Strange.

/life :: Link / Comments (0)

Fri, 11 May 2007

Solving captcha problems. Scaling.


I've added scaling into my algo of detecting matched images and changed part which detects how similar images are - now I use simple cover algo, i.e. the smaller amount of uncovered by tested letter pixels of the unknown symbol are, the better letter is being checked, this can lead to situations when black square will have the best match, but older algo (sum of the shortest distances between pixels of the tested image and database letter) is bad too - it fails when database letter crosses studied symbol in many places, so sum of distances becomes extremely small.
For example this quite complex test has been passed successfully:



But things are not too good for other a bit rotated cases. Since main symbol's axis does not match for database symbol (where it is vertical in standard font) and sudied symbol (where is can be arbitrary), amount of covered symbols does not represent matching order.





So, next task is to setup rotating of the letters to find the best match and use it the same way shifts are currently used (if various shifts of the letter in horizontal and vertical directions produce better match, new position is saved).
As you can see on the above passed test, letter 'N' is not placed to the left upper corner, where it was placed initially.

/devel/captcha :: Link / Comments (0)

Climbing evening.


That was quite usual trainig, although I tried some new starts and as usual did quite a few old boulderings, traverses and trace starts. Couple of times I made instructurs feel nervous - they did not allow me to climb higher than 3 meters without ensurance, but eventually that level was changed to about 5 meters for me. This time I wanted more, but scary screams from far ground stopped me from doing it, but my time will come, I know, and I will complete all those complex and not traces I started for the last several month...
At the very end I even climbed high on the wall with insurance, but I was already too tired to complete something interesting, so I just made several moves over interesting holds and that was all. The lesson: try complex traes early while you have a power to complete them.
Nevertheless, that was very good training.

/life :: Link / Comments (0)

Thu, 10 May 2007

Solving captchas.


I've (I think so) a brilliant idea of how to solve simple (with text only) captcha problems on comupter.
I thoughts about how human does it and found, that people frequently decide meaning of some letters and words based not on absolute knowledge, but 'this looks like A' and so on.
Computer, from another side, does not have such tristate logic, it can only compare and decided equal or not.
So I want to create a trivial application, which will select a letters from the picture (just by having a threshold of the colour difference), scale it to proper size (known to application), rotate to get main axis of the letter be vertical and then find a difference between image's letter and database letters using some kind of the smallest square interpolation metrics. Database letter with the smallest difference will be a winner.

It is quite simple task, I wanted to complete it just to show my idea is right or wrong, nothing more, and I even started to read GTK2 tutorials to get my first gtk application to read images, but then I found that in my mixed Ubuntu-Debian setup I do not have gtk development package and it can not be installed because of some problems in dependencies.
Crap.

Did you read Goncharov's 'Oblomov'? He failed to start completely new life just because when standing from the bed his feet missed the shoes.
I feel myself the same now...

P.S. No, I'm not Oblomov, I installed gtk2 and devel libs on my backup server.
Likely I will not go to development shop today...
P.P.S. Crap, I forgot that backup server has different arch, but I have not gave up yet. Thinking...
P.P.P.S. Solved: I've installed xserver on backup server in addition to gtk2 devel libs and use ssh X forwarding. Backup server has plenty of space, so it should not be a problem.

/devel/captcha :: Link / Comments (0)

First captcha solving results.


Problem has not been solved easily. Only the simplest case works:



Captchas from real site ends up with wrong results:








Code includes minimal affine transformation to find the best match, but does not scale images yet, which is a major drawnback, which ends up with above results.
Note, that it is proof of concept code, so I manually cut symbols from real-life captcha images into per-symbol files.

/devel/captcha :: Link / Comments (0)

Kevent strikes back?


Xavier Nicollet has sent me LWN article about yet another round of discussions about event delivery mechanisms in kernel.
After Davide Libenzi started his signalfd/timerfd/*fd patchset after eventfs and kevent, there were quite a bit of discussion about it, and eventually something new called pollfs (I did not follow that threads, since I'm not subscribed to linux-kernel@) appeared and started to fight for the place under the sun. Andrew Morton highlighted that likely eventfs by Davide will be included into 2.6.22, but strange things started to appear here...
Ulrich Drepper, glibc land monarch, said that he opposes against eventfd and similar patchsets because of lack of functionality for high-performance servers (mainly because of absence of special-purpose ring buffer implemented in kevent, there are also words about kevent possibility to carry more information and other small bits) and wants something similar to kevent to go in. Davide seems to be opposed to add such functionality (and actually it is not that simple task with old poll design).
As before - there is no active discussion and support developers for Ulrich's positions, but there are no developers against it too, so likely nothing will be included into mainline in 2.6.22 round.

Frankly, I removed kevent git tree from my machine, but I have backups...
So, I do not know should I dust off kevent patchset for yet another round of (empty as I predict) discussion or just forget it completely.
Let's see if new patchsets will appear in a near future, I will follow LWN kernel line, if nothing will be changed/added I will have a talk with Andrew about needs for this step at all.
We will see...

/devel/kevent :: Link / Comments (0)

Wikipedia computer science logo.




Author of this algo is not a hacker. Absolutely.
In hacker's mind there is no even remotely stage called 'buy new lamp' or 'drop the problem'.

/devel/other :: Link / Comments (0)

Appartments.


University of Minnesota study suggests that ceiling height may affect problem-solving skills.
According to study, high ceiling activates idea of freedom, while low ceiling more constrained, confined concepts are activated.
The concept of freedom promotes information processing that encourages greater variation in the kinds of thoughts one has, said Meyers-Levy, professor of marketing at the University of Minnesota. The concept of confinement promotes more detail-oriented processing.

I knew my idea of having higher ceiling in the working area of the room (where table is going to be installed) and lower hinged ceiling with dotty lights in the rest area was scientifically correct.
Btw, I started to install dotty lights in my ceiling.

Some notes on perfectionism.
Some times you start to change good for the best, you fix small details, you redo and redo something, which is good already just to achieve perfect state. You do it in every single detail, in every single invisible part. You continue until result is really perfect.
Well, that's not about me, but nevertheless I started yet another round of cleaning my hinged ceiling. This time it will be the last one. Likely today I will by the last portion of the colour, will paint selected bits of ceiling this weekend, which were filled with plaster yesterday, and finally complete this fucking ceiling.
You can not even remotely imagine amount of knowledge (and problems) I got during its installation.
But it will be completed.
Actually, it is completed already, it just does not know this yet.

/devel/flat :: Link / Comments (0)

Wed, 09 May 2007

I congratulate you with The Victory Day.


Thanks a lot, veterans, for everything you did!

/other :: Link / Comments (0)

Tue, 08 May 2007

Unified socket storage.


I've just released a second patchset which implements unified cache of sockets for network instead of old hash tables. It stores all types of sockets (although I only implemented af_inet, unix, netlink and raw ones for now) in single object structure called multidimensional trie (which is similar to judy array in some way).

I performed simple performance test with handmade client and httperf. The former is just epoll driven client which issues requested number of requests one-by-one (or with some concurrency, which does not yet proven to work correctly). With mpm apache on test machine I got sustained 2k/s requests for mdt and about 1200/s for (untuned) hash. With lighttpd and httperf (10k max, 1k rate) I got sustained 1k/s for mdt and 550-1000/s for untuned hash. With tuned hash (thash_entries=1000000) I got both 1k/s, with 30k max, 3k rate httperf I got 1650 for mdt and 1k for tuned hash. Server was with lighttpd 1.4.13. (handmade server as long as 'echo -en "GET / HTTP/1.0\n\n" | nc server 80' does not work due to unknown reason, I did not investigate). Results are quite small for that machine (amd athlon64 3500+ with 1gb of ram and gigabit r8169 adapter), but I have all debug options turned on (including heavy slab/vm).

Patch has been sent to @netdev for review. I asked about discussion about future of this project before making any further steps (mainly statistics code).

/devel/networking :: Link / Comments (0)

Linux and pornography.


Short translation of the msot interesting parts of Moscow militia cief officier lieutenant-general Vladimir Pronin speech (in russian and can be a hoax).

... if country officials decided to make users of unlicensed software and murders and violators equal ...
... fact of the usage of Linux says that organisation has something to hide and beware of checks ...
... Unfortunately, the fact of Linux installation itself is not a crime according to russian laws...

... Linux usage is absolutely the same as pornography storing, which is not a crime, but highly related areas like spreading ... and making are absolutely illegal.

/devel/other :: Link / Comments (0)

Mon, 07 May 2007

Climbing evening.


That was a really good trainig, although my shoes are broken, I did not climb up on the walls, I managed to damage my knee and shoulder (something was dislocated both in right knee and right shoulder, so there is quite a bit of pain when I open a heavy door for example). But I did a lot of starts on negative slope and negative horizontal part, I also created several boulderings with jumps and wild screws. It was excellent time.

/life :: Link / Comments (0)

Where I could be Sunday...


Grange invited me to "Pobeda" rally near Mojaysk, but instead of watching fast cars and good driving I hacked a remote flash reprogramming utility to fix some manager's error, which ended up selling a board with wrong firmware, which could end up with problems for collegues and serious one for manager. Luckily it was completed quite fast during Sunday, so no one noticed and no one got problems today.

Grange took several rally shots.

/life :: Link / Comments (0)

Sun, 06 May 2007

Unified socket storage testing.


I've written an utility which managed to heavily crash my testing system. I expected that so currently that is being investigated. Utility is quite small and simple web request generator, which can work with different concurrency level and maximum number of requests. I write my own instead of using Apache benchmark of httperf because both are quite heavy and do not allow to fairly test remote side (ab is client limited, since when I tested kevent it used 100% of the client's CPU, which should not happen, httperf uses old poll, so it does not scale to thousands of simultaneous requests). My application uses epoll and is quite small and is not intended to replace any of the above, but created only for my own tests. So far it is not 100% completed, but already crash unified socket trie cache.

/devel/networking :: Link / Comments (0)

Van Jacobson's talk about modern networking and related problems.


@Google.video.

/devel/networking :: Link / Comments (0)

Fri, 04 May 2007

Andrew Morton talks about Linux challenges.


I'm about to be quite enthusiastic about his words.
I would really like if major kernel developers and maintainers posted theirs views of the future plans, ideas and TODO of the areas they are working on.
I'm only aware of David Miller's network TODO list, which is quite obsoleted right now because none seems to be intresting in some areas, but quite a few bits are being done in another directions, which are not mentioned.

A call for 'papers'?

/devel/fs :: Link / Comments (0)

Thu, 03 May 2007

Great filesystem contest. Results!


Enjoy graphs of ext2/3/4, reiserfs, reiser4, jfx, xfs in postmark, iozone, maildir pop/smtp, dbench and file creation benchmark fights.

Winners everyone declares him/herself.

/devel/fs :: Link / Comments (0)

New wine.


I got a bottle of "Bastardo" from Inkerman factory from 2005 harvest - quite tasty, but I would set strong 4 out of 5 mark.
Caberne from Massandra factory is not that good - it is tasty, but very sour from my point of view, so I sip slowly it with small unfrequent gulps.
Found that Coca-Cola after being opened and halved does not lose its taste after several month "keeping", so I have a mix for the last bits of "Baccardi" rum, Mephody - irish bastard - killed my "Capt'n Morgan", so the most tasty things are over...

/life :: Link / Comments (0)

Wed, 02 May 2007

Climbing evening.


That was quite interesting trainig, although as usual I did not climb high, only on the lowest 3 meters. After warming traverses I started to climb quite old complex 7a trace over massively passive yellow holds in the central sector - scratched fingers and damaged a shoulder, but I completed first 5 holds (3-4 meters) of that trace, but I can not do it on demand, so this needs more traning.

Found, that moving without phone has some positive effects - I do not know current time and thus do not hurry up, and, what is the main puzzle, I arrive in-time (although I have no places where I'm not allowed to be late).

/life :: Link / Comments (0)

Tue, 01 May 2007

Ceiling has been painted.


It is not yet aboslutely perfect (and likely will not be), but I still want to put the last layer of colour, and then it will be done.
I also need to create dotty lights to finally draw a line.

There is one more thing I really do not like in this world (besides running) - long-hair roller, it requires to get a colour even more frequently than for each square meter and to push quite hard, which is a bit unpleasant and unconvenient task when painting a ceiling.
It is also not the best decision to paint several square meters of vertical borders and parts for the hinged ceiling using 3 sm-width ... (here would be a name for the device used to paint, it is small and is being held by one arm, but since I do not have access to the dictionary, I encrypt it this way), but in that case at least one can feel himself as an artist...

/devel/flat :: Link / Comments (0)

Does Ubuntu 2.6.20 kernel suck? Or why file creation on that kernel is so slow.


I've ran a simple test, which creates huge amount of files in selected partition. This is a part of the my filesystem contest started on local desktop (Ubunty 7 with 2.6.20-15 kernel) and backup server (Debian Etch with 2.6.18-3 kernel). Machines are different a bit (Intel(R) Pentium(R) D CPU 3.40GHz desktop and Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz server, different disks, desktop: ST3120023A udma(100) on NFORCE-CK804 chipset, server: ST3160811AS udma(133) ncq on ICH8 chipset), desktop machine runs Gnome desktop, server one has lvm setup. Both have 2gb of ram and use the same mkfs.ext3, and I expected results to be different. But not that much:



I completely understand that desktop hardware is a bit worse, but not 4 times!
Explaination of the picture is very welcome.
Likely Debian kernel has some ext3 feature turned on, while Ubuntu one does not.

P.S. In case you wonder why I do not perform promised trie vs. hash socket lookup algo tests, I can answer, that my test machine is being used by filesystem tests, when it will be free, I will continue to work on that problem (although do not expect major progress - it was met quite cool in netdev@, so I will just clean it up to the stage where I can run a web server with a big load on it, and if results will be good, I will continue).

/devel/fs :: Link / Comments (0)

Distributed RAID.


As a part of my filesystem feature set, I thought of distributed facilities, mostly distributed raid system.
Let's think about it as a raid5 for example - there are 5 targets where 4/5 of the storage is used to store information and 1/5 is used to store redundant data used for recovery. Such setup can be distributed - several targets can be accessed over network. Probably (although I'm not sure and did not even tried to setup) it is possible to do such things even in current linux kernel with network block device, which can be setup as part of the device-mapper software raid. But it is not possible to have, for example raid15 or any other raid system except raid0,1,5,6(testing).
It is quite strange there is no such device so far, since it is quite straightforward system on top of Reed-Solomon encodyng, which is even part of the Linux kernel.
So, either I completely do not understand why there is no tool to create huge distributed raid systems, or something else. Whatever answer is, I will create userspace tool for exactly that purpose to test my ideas. If it will work (is it ever possible to have another case? :) it will be either part of my filesystem, or part of the device-mapper tool, but it is too far prognosis...

/devel/fs :: Link / Comments (0)

Maildir pop/smtp benchmark has been completed.


The simulation works on each filesystem in the following stages:

1. The empty filesystem is created and mounted.
2. The directory structure is created, with no files.
3. A single delivery simulator and retrieval simulator are run simultaneously. The script waits for each of the simulators to finish, and then runs the sync command before proceding to the next step.
4. The above step is repeated with 2, 4, 8, and then 16 delivery simulators.

1. It writes a file with a unique file name to the tmp subdirectory.
2. It fsyncs the newly written file.
3. It renames the file into the new subdirectory.
4. It fsyncs the new subdirectory (to ensure that directory is actually on disk, as most Linux filesystems don't automatically perform this action during the rename).

/devel/fs :: Link / Comments (0)