About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Sun, 14 Jan 2007

NPTL thread stack usage.


NPTL (as of glibc-2.5) uses following tricky stack usage model: when new thread is created some stack must be allocated for it, so NPTL stores cache of stacks, each of which has different size, so when thread is created that cache is searched for the entry with the nearest but bigger cache, if cache is found and it is not that bigger than requested (it is less than 4 times bigger than requested), it is returned to the user (and global variable which holds size of the cached stacks is reduced). If there is no appropriate stack, it is allocated from anonymous memory using mmap() with at least MAP_PRIVATE | MAP_ANONYMOUS flags, which in particular means that pages are not allocated at all, but copy-on-write mechanism is used (i.e. real allocation happens only when user writes to allocated pages).
Maximum size of the thread stack is 40 Mb, default value is taken from rlimits.
Stack is guarded by some pages (if required) and part of it is used as control block.
Simple and good model.

Exactly the same model will be used for stack allocation for M-on-N threading model.

There is another interesting memory issue in NPTL - so called thread-local storage (aka TLS).
It allows to have for example global errno variable to be per-thread. This requires to extend programming language (C and C++ have __thread variables) and ELF header. Reasons to have thread-local storage are obvious and the main one is performance, but it has too many problems with symbol lookups, object allocation and other issues (more details can be found in this article), so I will not implement it.

Furthermore, my library will be actually _trivial_ layer on top of glibc, which will just provide _new_ symbols for programmers, like new_write(), and if approach will be considered as good, glibc maintainers (namely Ulrich Drepper, with whom I had some discussion about kevent (it looks like we still not 100% agree on all issues, but there was a major progress)) could adopt given design.
My library will use makecontext() and friends functions from glibc as a basis, which are actually just wrappers, which work with registers and other meaningfull for process running information.

/devel/threading :: Link / Comments ()