Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [queue]

The tag has no usage guidance.

-1
votes
0answers
23 views

Multiple containers and a job queue

I'm planning a web service for some computational models. When trained, the models are usually pretty light, but sometimes a heavy calculation has to be done to update models that the light ...
0
votes
2answers
53 views

How do job queues return results to UI?

I am working on an app that will run queries and return results to UI. However, these queries can take a long time to finish so I should not wait for the result. As far as I know, people use task ...
-3
votes
2answers
142 views

Why are multiple backends in this system? [closed]

I am trying to understand the architecture of the system described in this patent about aggregating and analyzing confidential data: https://patents.justia.com/patent/20180089196. The general ...
0
votes
0answers
32 views

Aggregate in memory and update records with workers in queue

Imagine a 'views' column on database table. Each user that views a page like a video or similar run a query to database to increment the field value. To reduce the database writes, I want to ...
6
votes
2answers
343 views

Fair distributed task scheduling with RabbitMQ

Problem: A single client of our system can completely flood all available resources with a massive workload. You can consider we have only one queue, and anyone can schedule any amount of work in it. ...
1
vote
2answers
300 views

Pull important messages from DLQ queue and save them in a relational database, to be analyzed and sent back later. Is it a good idea?

My project has a integration with a external system. We need to send some important informations to this system. For this, we create a micro service to connect to this external system. This micro ...
0
votes
1answer
45 views

Modified priority queue (with “disabled” elements)

I need something like a priority queue but with the following modification: Sometimes I need to mark some elements in the queue as "disabled". Disabled elements: temporarily (until marked non-...
1
vote
2answers
155 views

Are there any use cases for List when Deques and Arrays are available?

I've been thinking about this over the past few weeks, and I've come up with no good arguments. My perspective is from Java, but if anyone has any language-specific cases outside of this language, I'...
3
votes
1answer
172 views

How to name two functions that could be named the same?

I have a node app with a queue for processing jobs in the background. I have a file that exports a function which when run, creates a job in my queue. In that file, I also have the handler for this ...
-3
votes
1answer
67 views

Best Data Structure for ordered list of connected users

I have three attributes for a user: connectionId (e.g. "sd4-h4sdg-u4j-a2rr3"), userName, and placeInLine. I can't think of a single data structure that will handle my scenario of only allowing a ...
1
vote
3answers
210 views

Why would I use a queue to list all files and subfolders recursively

In Robert Sedgewick's Algorithms book there's this exercise: A folder is a list of files and folders. Write a program that takes the name of a folder as a command-line argument and prints out all ...
1
vote
0answers
62 views

Designing a configurable delay pattern on JMS (tibco maas) stack

My java app is a consumer to maas topic(say PRIMARY) on which upstream systems publishes their messages. Some of the pushed messages now needs some delay(say 10 min) before my app starts processing it....
-1
votes
2answers
732 views

Best way to design a billing system for a SaaS

i am working on a SaaS project that provides some services that people can call via API. My problem is how do I build the billing algorithm. I want to charge on every call to the service. Now I'm ...
-2
votes
2answers
121 views

Priority Queue and Set in Java or general

why do we need priority queue when a set always keeps its object sorted, I think we can always remove the first/last element of a set and it would work as priority queue because priority queue offers ...
0
votes
1answer
98 views

The right way of redundant task queuing?

I'm currently working on refacting an existing project. Basicaly, I need to perform requests to remote services. Current solution is to write tasks to a database and then poll them using cron jobs. ...
-1
votes
1answer
78 views

Moving XML data into DB [closed]

Is this a "overkill" design to store XML data in a relational database? One point is that these web API REST services can later be used by other applications.
5
votes
3answers
335 views

Large doubly-linked-list (or other) storage on disk for queue system; options for how to store

I'm looking at building a message queuing library in Go, which will be used as part of a larger application. A doubly-linked-list seems like a sensible approach for an in-memory data structure, but ...
0
votes
1answer
722 views

Scheduling notifications reminders for users

I am trying to design the best way to send reminders to users for events they are registered for: reminders should be sent 72, 48 and 24 hours before the event reminders cannot be sent twice (so user ...
3
votes
4answers
950 views

Designing an efficient implementation of a random access queue based on a linkedlist

I'm currently working through Robert Sedgewicks "Algorithms in java" (3rd edition, german) on my own and am currently at one of the more complicated questions there. I think I may have the starting ...
2
votes
1answer
140 views

How to detect end of queue in a parallelized web crawler?

Let's say I want to program a parallelized web crawler which has a shared FIFO queue (multi consumer/multi producer). The queue only contains URLs. How do I detect the end of the queue? A worker ...
7
votes
0answers
168 views

I need to write my own version of a bounded queue [closed]

I need to write my own version of a bounded queue. It must be thread safe. The consumption of this queue is based on a priority system, where a consumer makes a request for an object in the queue. ...
2
votes
2answers
737 views

Queue how to preserve requests?

I'm working with Laravel queues lately. I've all set up and running fine. What I see is that I perform a http request to my system, the controllers manage the request issuing a Model having request ...
6
votes
3answers
742 views

Slow throughput - still worth using a dedicated message queue?

Here are some requirements for a queue: Every few days add ~100k tasks with various priorities Workers will pull tasks at typically less than 10 / second Tasks need to be completed by ~2 unique ...
-2
votes
1answer
59 views

Name for object to re-order events received in arbitrary order?

I have a program which generates events. They are ordered with a Lamport clock and sent over the network. They may arrive in a different order. On the receiving end I have an object which re-orders ...
8
votes
2answers
1k views

How to best design a job queue with constraints?

Consider the following situation: You have a program which creates numerous 'jobs' that need to be processed and places them into a queue. You have other worker programs which grab the next 'job' in ...
0
votes
2answers
739 views

Ordering 'Tasks' for execution based on their dependencies

Let's say I have a set of Tasks that they have dependencies. These tasks are not in order, but the execution of tasks should be in the proper order. Each task has two properties: Before and After. ...
4
votes
3answers
3k views

Queue vs Threads

I'm implementing a data processing software. The software gets form the network thousands of events that must be processed in according to rules. I implemented a multi-thread service, which receives ...
1
vote
1answer
105 views

Process synchronization over Internet

I'm trying to create an IOT hub that can be controlled locally (Intranet) or from Internet. The problem I'm facing is of state synchronization. The following points depict what's happening at my local ...
5
votes
1answer
184 views

Querying large amount of data for parallel processing [closed]

I have a dataset containing list of users (around 50M). Each user has an email address, name, and some more data columns. I want to send a weekly email to those users, and the content of the email ...
1
vote
0answers
298 views

Better choice of Queue then a ConcurrentLinkedQueue if I have low number of threads in Java

I have two threads, one that is reading messages from the web and putting them into some sort of a queue. The other is suppose to read it. I looked into a ConcurrentLinkedQueue and it looks good for ...
1
vote
1answer
75 views

Threads Inside Full Duplex Device Library

This is the second project I'm working on which will use a design that I'm not 100% confident about. I'd like some feedback, maybe recommendations for a better design, or verification that this would ...
2
votes
2answers
91 views

Operations Job Queue Strategy

I am working on an application that requires manual approval of requests put in by customers. Now lets say there are 10 people involved in this operation and each of them has a webpage in front of ...
1
vote
2answers
2k views

Creating a queue for web services

I have a program running that collects metadata from the internets. Every so often a summary of this data is written up (every 30 min or so) and the summary is posted to a database. The summary is ...
2
votes
1answer
121 views

Queue rescheduling naming convention (Command pattern)

For my project I'm creating a queueing mechanism based on the Command Pattern, which can execute one of a number of jobs. Jobs can add other jobs to the queue or can be added by a human. Most of the ...
1
vote
1answer
16k views

How to remove a middle element from queue?

I had a question, about how to remove an element from middle of the queue , without disturbing the order of other elements. E.g. Queue before operation:- <| a b c d e |< Queue after ...
9
votes
1answer
682 views

Message Queue for RTOS for microcontrollers

I'm currently writing an RTOS for microcontrollers. The whole thing is written in C++11 - if anyone is interested, and the link to the repository is at the bottom. Currently I'm writing a class that ...
-4
votes
1answer
3k views

Reverse a queue using an extra queue [closed]

You are given 2 queues, Q1 and Q2. Q1 is already filled with n elements. Q2 is empty You have to reverse all the elements in Q1. You cannot use any other space apart from Q2. only operations ...
1
vote
2answers
2k views

Terminology: queue versus fifo

When referring to or documenting a typical lock-free circular data structure (or class or object) used to communicate between arbitrarily long asynchronous threads or processors: "queue", "fifo" or "...
3
votes
1answer
326 views

What happens differently when you add a task Asynchronously on GAE?

Google's doc on async tasks assumes knowledge of the difference between regular and asynchronously added tasks. add_async(task, transactional=False, rpc=None) Asynchronously add a Task or a list ...
0
votes
1answer
338 views

I'm making a report generator, how should I run it?

So I've started a project to generate reports for our system. These are reports that we deliver to our end customers and they are so specialized that no existing system can generate them. This is not ...
0
votes
2answers
244 views

Queues and threading

I am developing a new project where I will be constantly checking a webpage for data and adding this data to a queue for processing. This data will then be removed from the queue and added to a list ...
3
votes
2answers
7k views

How to increase the efficiency of an Immutable Queue in java?

I have been developing an Immutable queue in java, but the version I have developed is supposedly slow and I have been suggested to create a faster version of the same and I have no idea as to how to ...
6
votes
3answers
3k views

Use Queue<T> or stick to native f# lists

I need something that represents a queue in F#. The queue will be filled with items once, and (as the application progresses) items will be taken from this queue and moved to other lists. Should I ...
7
votes
3answers
2k views

When designing a job queue, what should determine the scope of a job?

We've got a job queue system that'll cheerfully process any kind of job given to it. We intend to use it to process jobs that each contain 2 tasks: Job (Pass information from one server to another) ...
9
votes
13answers
33k views

What are some good, simple examples for queues? [closed]

I'm teaching CS2 (Java and data structures), and am having some difficulty coming up with good examples to use when teaching queues. The two major applications I use them for are multithreaded message ...