There is an application that processes TCP/IP requests according to some logic, and pretty always it connects to DB.
Architecture is very common: one client -> one processing thread -> one DB connection -> one DB thread to process a query(DB thread exists inside DBMS application and mentioned here to show how insufficient current performance is).
The system is based on recent Linux kernel, GCC used is of 6.2 version.
Problem is that DB query can last, and so application's thread is sleeping and clients with tasks which require possibly much faster queries have to wait for a thread which is actually do nothing.
I'm thinking of different possibilities making application asynchronous to some point. The problem is in size and age of application - there is no way I'll find time to fully rewrite each "service" to support some async model (like FSM objects able to store their state while waiting for DB to respond and so able to be moved from stack to some storage). Yet each "service" is a class itself with one function which calls DB function(always same) at some/few point/s inside, so all processing state is on stack(inside this overriding functions), no single class member that can be stored exists, and as mentioned above rewriting it to store something inside "service" object to allow continuation using straight C++ techniques (I know about :-) ) is a half-year job at its best.
Last idea was to provide each thread with custom stack using pthread_attr_setstack(), and manipulate this data manually after DB function call(i.e. DB connection(socket) and stack data can wait in some queue, while current thread can jump to higher stack address(assuming x86 stack grows down)) restoring it from memory and setting stack pointer to appropriate place(asm esp and etc). While manipulating stack data doesn't seem to be a problem(currently), I don't know how to bulk-restore other valuable CPU registers and possibly OS specific thread-related structures, to allow problem-free continuation from some stack address, am I right that restoring stack memory and the stack pointer is not sufficient to continue execution?
I'm looking for some info on how such problems can be/were solved and if an idea of mine can be realized. Any links are appreciated.
Another idea is to have a large pool of threads where most of them are in dead sleeping. If system can normally handle 128 simultaneous threads (128 physical CPU cores) then we can create 1280 threads, but there should be a schedule which will allow only 128 from them to execute at the same time, thus if 1279 threads have been blocked while waiting for DB response, next user request will be processed in thread #1280, and if that thread hasn't been blocked yet further request can be processed in another free thread from, this will continue while we will not get 128 running threads. If DB response has been successfully "epolled" then we can wake up waiting for this response thread. This is not fully asynchronous model as in worst scenario all 1280 threads will be sleeping while waiting for DB, but it will provide an opportunity to have 10 times pending DB requests.