There is an application that processes TCP/IP requests according to some logic, and pretty always it connects to DB.

Architecture is very common: one client -> one processing thread -> one DB connection -> one DB thread to process a query(DB thread exists inside DBMS application and mentioned here to show how insufficient current performance is).

The system is based on recent Linux kernel, GCC used is of 6.2 version.

Problem is that DB query can last, and so application's thread is sleeping and clients with tasks which require possibly much faster queries have to wait for a thread which is actually do nothing.

I'm thinking of different possibilities making application asynchronous to some point. The problem is in size and age of application - there is no way I'll find time to fully rewrite each "service" to support some async model (like FSM objects able to store their state while waiting for DB to respond and so able to be moved from stack to some storage). Yet each "service" is a class itself with one function which calls DB function(always same) at some/few point/s inside, so all processing state is on stack(inside this overriding functions), no single class member that can be stored exists, and as mentioned above rewriting it to store something inside "service" object to allow continuation using straight C++ techniques (I know about :-) ) is a half-year job at its best.

Last idea was to provide each thread with custom stack using pthread_attr_setstack(), and manipulate this data manually after DB function call(i.e. DB connection(socket) and stack data can wait in some queue, while current thread can jump to higher stack address(assuming x86 stack grows down)) restoring it from memory and setting stack pointer to appropriate place(asm esp and etc). While manipulating stack data doesn't seem to be a problem(currently), I don't know how to bulk-restore other valuable CPU registers and possibly OS specific thread-related structures, to allow problem-free continuation from some stack address, am I right that restoring stack memory and the stack pointer is not sufficient to continue execution?

I'm looking for some info on how such problems can be/were solved and if an idea of mine can be realized. Any links are appreciated.

Another idea is to have a large pool of threads where most of them are in dead sleeping. If system can normally handle 128 simultaneous threads (128 physical CPU cores) then we can create 1280 threads, but there should be a schedule which will allow only 128 from them to execute at the same time, thus if 1279 threads have been blocked while waiting for DB response, next user request will be processed in thread #1280, and if that thread hasn't been blocked yet further request can be processed in another free thread from, this will continue while we will not get 128 running threads. If DB response has been successfully "epolled" then we can wake up waiting for this response thread. This is not fully asynchronous model as in worst scenario all 1280 threads will be sleeping while waiting for DB, but it will provide an opportunity to have 10 times pending DB requests.

  • What if you wrote an adapter library that exposed async methods which return std::promise objects? May 16, 2017 at 21:31
  • Are you able to make changes to the database? It might be less intrusive to try that rather than a complex software solution. You might be able to get significant gains from techniques such as in-memory tables, indexing, views, and database caches.
    – William L
    May 17, 2017 at 1:05
  • @WilliamL I'm not a DB developer, I know by seen their faces every day that DB team already does it best :-) May 17, 2017 at 20:59
  • @RobertHarvey all this amazing stuff from <future> header is just syntactic sugar for threads (yet very usefull). By this I mean that those "async" calls are async by user-logic but concurrent by implementation, so they will create threads. And I don't want more threads, the problem is having 1000 threads where every thread waiting 100ms for something leads to ~25% CPU load while no further requests can be processed, if I will start adding new implicit threads by using <future> stuff, I will end up in CPU doing it best (~100% load) switching threads. May 17, 2017 at 21:00
  • Use whatever underlying implementation you want. The concept I'm trying to convey here is adapter. May 18, 2017 at 3:27

3 Answers 3


There is more than one way to add asynchronicity. You have explored one option, to make the single threaded application more asynchronous. Perhaps there is another option, to make a wrapper application that calls multiple copies of the original application.

Here I am assuming that somehow your original application has an API, or you can create an API, so you can run that application with no user interface. The user then makes a request, the request is passed to one instance of the application using inter-process communication, and the wrapper application resumes running. When the result is computed, the wrapper application shows the result.

If while waiting for the response, another request comes in, you will maintain an "application process pool", consisting of a limited number of instances of the original application process. The next request goes to the next instance in the pool that is not occupied. And so on. The instances can be dynamically started.

  • Thank you for your answer. It seems I didn't understand your idea. Do you mean writing dispatcher/consumer where "worker" will be entire existing application? If that is what you meant, than I can't see any advantage or asynchronicity there, because system will end up in N blocked threads waiting for DB responses. Even good servers with TBs of ram and >2 highend CPUs are unable to effectively manage hundreds of threads, it will always end up with CPU handling context switching only. I'm looking for some solution to switch from one running function to another inside one particular thread. May 17, 2017 at 20:33
  • Are the threads blocked on IO or consuming CPU? May 19, 2017 at 5:37
  • They are blocked on IO (DB communication using libpq-fe) May 19, 2017 at 9:24
  • Blocked threads can consume memory resources, but they won't increase context switching overhead. May 19, 2017 at 14:44

Depending on your setup and requirements, if queries are mostly reads and not much writes (such as a reporting system), you might be able to replicate the data to a database of your own which supports SQL calls from multiple clients without the same limitations. The one DB thread in the current database application is just used to extract data and update your own DB which the clients will query.

In my case for a similar problem, it was an IBM DB2 database with restrictions, so the idea was to replicate to a MySQL database using a custom Java application to read from DB2 regularly and write to the MySQL database which we could query without any limitations.


IMHO, if the code is too bad to be refactored into async behavior, there is no hope that it could be refactored into threaded one. So available options are:

  1. Add a front end to start a new process for every heavy request.
  2. Cache database data.
  3. Bite the bullet and refactor the code into async behavior.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.