Zbr's days.

About :: TODO :: Blog :: RSS :: Old blog :: Projects :: GIT :: Gallery :: Notes

Mon, 28 Jul 2008

Distributed storage development progress. Thread pools.

Today I implemented simple thread pool subsystem, which allows to create set of threads, to add/remove them them from this set in run-time, and to schedule a work to be done by them. Work is specified as to functions: setup() - it is called when system has selected a thread for execution, so caller can setup needed data, and action() - it is called by thread itself, it has access to the data, provided at initialization time.
Work scheduling has a timeout parameter, which corresponds to time system will wait for free thread, otherwise error is returned.
System is generic enough not to contain any notion about DST or crypto, only two new data types: struct thread_pool and struct thread_pool_worker, only the former is visible to the user.
API looks like this:

void thread_pool_del_worker(struct thread_pool *p);
struct thread_pool_worker *thread_pool_add_worker(struct thread_pool *p,
	char *name,
	int (* init)(void *private),
	void (* cleanup)(void *private),
	void *private);

void thread_pool_destroy(struct thread_pool *p);
struct thread_pool *thread_pool_create(int num, char *name,
	int (* init)(void *private),
	void (* cleanup)(void *private),
	void *private);

int thread_pool_schedule(struct thread_pool *p,
	int (* setup)(void *private, void *data),
	int (* action)(void *private),
	void *data, long timeout);
init() and cleanup() callbacks above are used after new thread is created, so that user could initialize per-thread data, for example it is used to allocate some cached pages and initialize crypto algorithms.

This thread pool system is used by the crypto processing code in the distributed subsystem: when block io request is about to be sent, or when system has received reply for the read request, it schedules crypto processing work to the pool, initialized at DST node setup time.

Crypto processing does not yet work in DST as long as some other bits, so far I only played a bit with its initlialization sequence, so it was split to network, crypto, security initializations and node start, which registers new storage in the block layer subsytem. This steps allow to introduce later additional initialization steps if needed without breaking backward compatibility.

Next steps include proper network initialization and processing and transaction management helpers. Then I will combine all existing code and make a first renewed release.
Stay tuned!

/devel/dst :: Link / Comments ()