4

I am making a program that primarily deals with heavy data mining and analysis. It is primarily a back-end type algorithm, with no GUI elements or console windows seen as it runs. Occasionally though, it will need to graph something or get very special input from the user. However, I initially didn't want to build the application integrated into the GUI as I feel that would be overly complicated. Instead I had wanted to just make my algorithm separate from any GUI elements, and if they were needed, then spawn them in, they do their job, and then exit; they are additions to the base program, not necessary for it to work.

So, the logical conclusion is to make my data processing happen in the main thread, and my GUI occupy another thread when necessary. Why do I need two threads? Because the GUI requires me to run an event loop so everything stays in check.

The specific graphing / GUI utility I have picked for this task is PyQt5, but that should be besides the point (most of them have the same basic mechanics).

It turns out though, that GUI elements do not like being run in a non-main thread (see here). It is possible, as you can see here, but the communication from the main thread and the secondary thread is nasty. I don't have to pass information as much as I have to actually interact with the GUI object's methods. I don't have to do any actual GUI widget manipulation from another thread, just access non-GUI methods belonging to the GUI object.

This must be a common design, but I have no idea how to implement it. Keep in mind, when I say 'data processing' section, it is vastly more complicated than that. Thus, building a GUI around it, and having everything being done inside of signals and slots seems to make it a massive mess.

So here are the general questions. I am not looking for code, but rather general pointers.

  1. How would you design a program like this?

  2. What would each thread look like?

  3. How do they interact in a way that doesn't anger the GUI?

  • You are going to have a hard time trying to make qt gui work in another thread than the main thread. Why not just put your work in a separate thread instead? – Teimpz Aug 16 '16 at 8:01
  • Because the program is spawned as a non gui application, and only in cases where it is necessary will it bring the gui up. It would be inefficient to create my whole application inside of the gui object, and then not use it 90% of the time. – Nick Pandolfi Aug 16 '16 at 15:25
  • Does your main program need to keep running while the gui is active? Can you just halt the main program, spawn gui (decoupled via some kind of interface or callback), collect data, close gui, continue? – Teimpz Aug 16 '16 at 16:27
  • @Teimpz Yeah it has to keep running. The gui will produce real time data graphs – Nick Pandolfi Aug 16 '16 at 16:30
  • How about starting a separate process for the gui? This will make the communication between the two a lot more complex tho. I still think you should put the work in a separate thread that is really how everyone does it. – Teimpz Aug 18 '16 at 6:44
1

First things first. Modern OSs and even programming languages allow the manipulation of GUI only from the main thread. For many reasons this is an absolute must and in fact it is hardly an option.

Then, the concerns you are describing are being solved by developing applications under design patterns. There are many approaches here with the most popular the MVP, MVC and MVVM patterns. They all allow you to separate the model (your data fetching and manipulation) from what and how you present things to the user.

  • Thats interesting, I wasnt aware that there were classifications for design patterns. I have to check this out. Ive never heard this in my years of Python. – Nick Pandolfi Aug 16 '16 at 16:35
  • @NickPandolfi: you probably have not heard of this because neither MVP, MVC and MVVM has directly something to say about multithreading. Where this answer is correct, however, is that these patterns might help you by keeping the UI logic separated from the analysis code, which might indirectly help you to solve your multithreading problems. – Doc Brown Aug 16 '16 at 21:15
  • @John Kouraklis I put the model and view in completely separate processes, and the controller is in a thread belonging to the model. Thanks! – Nick Pandolfi Aug 20 '16 at 14:02
2

There are several approaches possible, and to get the GUI/calculation with multithreading right, there is be more necessary than a few paragraphs here on "Programmers" can teach your.

However, the first thing I would typically ask here is: do you really need multithreading? Since you are writing a program that primarily deals with heavy data mining and analysis, I guess there is database available. You data mining program may run in one process, without any GUI, and your GUI program in a separate process. The interprocess communication might be done by using the DB. That way, it might be possible to avoid any need for multithreading in the GUI at all.

That way, you can keep a strong separation between the processing part and the GUI part, your processing part won't get intermixed with any signal/slot mechanics, and it is not even mandatory to use Python for the GUI part (but if you like it, you still can.)

  • The communication has to be done by one object in one process by calling another object's method in the other process. It isnt really 'communication' as much as it is 'temporary integration' – Nick Pandolfi Aug 16 '16 at 15:27
  • @NickPandolfi: clearly, your current approach looks like that, thus there can't be no other solution, because you say so ;-) Of course, to realize what I suggested you may have to solve some parts differently from the multithreading approach. – Doc Brown Aug 16 '16 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.