So I have some naive implementation for the following problem:

  1. We have a list of objects with action methods that have to be trigger at some value of interest.
  2. This value is polled (or is streamed) from a remote server and is constantly changing.
  3. Let's say we have tens of thousands of such objects in the list, and the remote value changes every couple of tens of milliseconds.

What could be a good scalable alternative to such code in order to handle the requirements?

Pseudo-code code looks as follows:

while true:{
   //get some data from continosly changing source
   responseData = getRequest('http://some-remote-endpoint')
   //check our all objects of interest
   for object in list {
       /* if the object has to be triggered at a 
       specific remote source value, trigger an action on it. */
       if object.triggerAtValue == reponseData.value {
  • Is the remote service yours or a third party? Can you change it? Why are you using polling to begin with?
    – John Wu
    Oct 14 '19 at 6:51
  • @JohnWu no it cannot be changed, it's 3rd party, or could be multiple ones if we talk scale. Some of them provide sockets, some just REST endpoints.
    – user59271
    Oct 14 '19 at 6:55
  • Will one responsedata only ever trigger one object or can one responsedata trigger multiple?
    – Pieter B
    Oct 14 '19 at 7:21
  • @PieterB it can be multiple and there are no foreseeable limits really, it can be thousands and more objects have to get triggered by a single value(event for that matter).
    – user59271
    Oct 14 '19 at 7:45
  • Since you can't change the polling endpoint, you can at least insulate yourself from it. Consider having your "poller" feed into one or more broadcast message queues and having your interested objects subscribe to them. Oct 14 '19 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.