3

I'm in the process of designing a new desktop application which is very different from other stuff I did before, and so I'll be happy if I could be pointed towards the right direction regarding the basic building blocks of it.

The application should read a binary file, process it "line by line" and after some chunk of data has been read and processed should write it back to disk. The raw data, i.e. the original binary files, are usually too large to load into memory, so I have to processes them bit by bit. The second phase (the processing) isn't too computationally intensive, and from previous experience I'm sure that the writing-back-to-disk part will take the most time.

What I currently have in mind are three threads (and not processes) - one is in charge of reading chunks of data to disk, the other does the processing and the latter does the writing back to disk. The main application (Python or Rust, not sure yet) will allocate the memory buffer for the first thread and will be in charge of scheduling the three threads in general.

Does this make sense? I'm aware that my requirements are very similar to that of standard async web apps, so I might be missing some important tools here that could help me avoid writing all of this from scratch.

  • 3
    Is there a particular reason you think you'll be needing threads at all, rather than a normal loop? Are your data so huge that not even a useful fixed-size chunk fits into memory? – Kilian Foth Jan 21 at 7:22
  • I know that writing to disk will take time (it can be over some LAN network) so that can potentially block my loop for a quite a while. – HagaiH Jan 21 at 7:35
  • 1
    What Kilian is trying to tell you is that if writing the results is the bottleneck, being able to make them faster is not going to accomplish anything. – Blrfl Jan 21 at 10:48
  • I see. It's a fair point. – HagaiH Jan 21 at 14:13
3

Using multiple threads is most helpful when your program is CPU bound and you want to parallelize your program to use multiple CPU cores.

This is not the case for I/O bound problems, such as your scenario. Multiple threads will likely not speed up your system at all.

In I/O bound settings having multiple threads may still be beneficial because using threads is one way to write concurrent/asynchronous codw. These threads are just a way to keep your code organized, but most of the time these threads will be sleeping and waiting for some event, e.g. that a new chunk of input has been loaded.

Instead of using threads, many programs prefer to use an event loop instead: you get all the benefits of concurrency except parallelism, but don't have to deal with the headaches that multiple threads bring. Python and Rust both have suitable libraries available, in particular asyncio (Python) and Tokio (Rust).

Especially for I/O, parallelization might be more harmful than helpful. A lot here depends on the physical storage hardware that you are reading and writing. Especially hard disk drives can be a lot faster when you let them do sequential reads of large blocks. Asking them to do reads or writes at a different location at the same time can end up making everything slower.

Even for pure CPU bound problems, threads are not always the best solution. Operating system threads are fairly large memory structures. The threads will be interrupted and scheduled by the operating system, which requires a context switch Thread A → Kernel → Thread B. That is slow compared to a native function call. Threads may need to synchronize with each other if they share any data structures. Such locks or mutexes may or may not be slow, but having to use them is usually slower than not having to use them at all. Finally, software performance depends dramatically on how efficient the software can use the CPU caches. If two threads work with the same data structure (e.g. a buffer) and one thread changes this memory, any caches of this memory on other CPUs have to be invalidated so that subsequent accesses will have to go to much slower RAM.

A caveat specific to Python is that the CPython interpreter has a Global Interpreter Lock. At most one thread can interpret Python code at the same time. Unless you are using C extensions that manage their own threads, multithreading in Python can never parallelize the code but may be useful for concurrent code. For new code, asyncio can do the same more elegantly.

  • Thank you for the detailed writeup. I will dig further into event loops. – HagaiH Jan 21 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.