0

This is a concept related question.

I'm writing a program polling for alerts on various devices, there are two different type of alerts i need to poll, reds and oranges.

My approach is to create two threads, each polling their type of alerts independently. I thought about using threads because both polling functions have to be designed to be completely independent and have different time intervals for execution, for example:

  • Red: query API 3 times, waits at least 30 seconds between polls ( try to avoid false positive by letting them time to clear out before doing stuff with the alerts )
  • Orange: Very passive, poll once, wait 10 minutes, then poll again, compare and do stuff ( same principle as reds )

Here comes the question:

Each step of alert processing is handled via multiple classes and various methods, ie: APIQUERYCLASS class calls DATAPROCESSINGCLASS (do stuff with alerts) -> DATAPARSINGCLASS (do stuff with alerts) -> ANALYSISCLASS (do stuff with alerts) -> ESCALATIONCLASS and so on...

Each thread is going to have an object APIQUERYCLASS which will create other objects from the chain of classes and use methods.

Is that a "sane" approach ? Will threads be truly independent ? Is there a better way to make this if so how ?

EDIT:

Thanks for the answers guys To add a bit more of clarity:

######Worker.h #######
 // Worker.h is the top file where i'm thinking about creating the threads
#include "ToolName.h"
class Worker
{
    RedHandler()
    {
        Tool t_red;
        t_red.GetRedAlerts();
    }
    OrangeHandler() // exact same thing as RedHandler()
    {
       Tool t_orange;
       t_orange......
    }
}

##### ToolName.h #####

#include "Dataprocessing.h"
class Tool
{
    GetRedAlerts()
    {
        // query API , if query result is OK
        DataProcessing d;
        d.ProcessData();
    }
}

##### DataProcessing.h ######

#include "DataParser.h"
class DataProcessing
{
    DoStuff()
    {
     ....
    }
}

The chain is working pretty much that way all the way to escalation where it breaks ( unless exception ) and goes back to the polling in Worker.h The exact same exists for Orange alerts given that the API query returns the exact same json structure.

And i'd like to put RedHandler() and OrangeHandler() in their own threads

I hope this clarifies a bit my question

2 Answers 2

0

If these API calls are blocking calls to a third-party library you can't control, this threading model is roughly the best you can do.

If the API library is yours and is just doing socket read/select or whatever, you don't really need multiple threads, and can use a single-threaded Reactor. Even if you really need to poll something rather than waiting on a socket, if the poll is fast or non-blocking, you can still do it on a timer callback in a normal single-threaded Reactor.

It doesn't matter in that case that the threads aren't independent, as the handlers are - unless your processing is so time-consuming it's likely to delay the next timer. For your case, processing an event would presumably have to take ~30 seconds to stand any significant risk of delaying the next event.

APIQUERYCLASS -> DATAPROCESSINGCLASS -> DATAPARSINGCLASS -> ANALYSISCLASS ... -> ESCALATIONCLASS

Apart from the horrible naming (which I assume is a placeholder), this deep callback chain seems odd. You definitely don't want all these classes to be co-dependent, so need at least an abstract Observer or something to break the dependency. Still, it's easier to test and debug code like

raw_event = APIQuery.getEvent();
parsed_event = DataParser.parse(raw_event);
DataProcessor.process(parsed_event);

than the chained version

APIQuery.getEventAndThenParseProcessAnalyzeAndEscalateIt();

or even

EscalationClass ec;
AnalysisClass ac(&ec);
DataParsingClass dpc(&ac);
DataProcessingClass dproc(&dpc);
APIQueryClass aqc(&dproc);

// ...

aqc.getEventAndTellObserversAboutIt();

The Observer pattern also becomes brittle if you have relative ordering dependencies on your observers.

2

There is no simple answer to your question. It depends on the details of your classes. Before giving the answer, you have to clarify some points:

  • Do the threads share data with other threads? If so, you have to take care that the data is only accessed by one thread at a time. This normally is done by synchronisation objects (semaphores, locks, etc.).
  • Is the API used by APIQUERYCLASS thread-safe? Can it be called by multiple threads at the same time?
  • Are all APIs used by data processing thread-safe?

You could use a much simpler approach:

  • In most runtime environments you can define timers for specific time intervals. If the time of a timer is elapsed, it will issue a timer event. The event handler then can query your API and start the data processing.

  • API query and data processing for both alerts types then would be implemented in the same worker thread. So you can avoid possible multi-threading issues of your APIs. As your polling intervals are quite long, it should be no problem using one worker thread for both.

Technically, a timer normally is implemented as a separate thread which calls the related event handlers if the time is elepsed. If your runtime environment does not support timers, you would have to implement this by yourself.

1
  • Thank you for answering, regarding the API and threads, we're ok, i already tried with a python prototype and so have my colleagues. The idea about timers is very interesting but i need both threads to live in infinite loops and resume their polling once the data has been processed, the way i see it, both thread would be absolutely separated data wise so they should never access the same thing, as for the ESCALATIONCLASS API used, i honestly don't know yet how i'll do it, i thought of creating a message queue based on alert escalation priority because i don't think it's thread safe. Thoughts ?
    – Ben
    Commented Oct 19, 2018 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.