Questions tagged [real-time]

Real-time processing means processing data as it is created or modified. This is in contrast to batch processing, wherein data is processed in large chunks at a specific interval (e.g. every night).

Filter by
Sorted by
Tagged with
0 votes
0 answers
35 views

Best practices to deal with realtime data in REST API, Postgres and Redis

Overview Today we have a monolithic service that deals with realtime time data. The monolithic are composed by follow stack: Django + AuroraDB Cluster (Postgres AWS) + Varnish cache and cloudfront at ...
user avatar
1 vote
1 answer
146 views

Performing many SQL inserts quickly without using BulkCopy

I have a C# system that reads thousands of messages per minute from a Kafka topic. The messages are filtered and I need to insert the filtered messages into a MSSQL table. A normal insert statement is ...
user avatar
0 votes
1 answer
73 views

Solution architecture for a Kafka streaming to website system

Not sure if this is the correct forum for this question - but I could really use some advice. I need to design a system (within certain constraints). The system must implement the following logic: A ...
user avatar
0 votes
0 answers
35 views

Handling realtime notification receivers in a paginated application

I am working on an application right now, which is fully paginated. The application basically providers the client (a browser SPA) with a list of all data items other users added, and it does that in ...
user avatar
0 votes
1 answer
127 views

Architecture for alerts system based on API - would this be too slow a design?

(Hi I’m new here - I believe this question suits here from the guide saying “ architecture, and design” - however if there is a better place, just pop me a comment. As there’s no actual code it doesn’...
user avatar
  • 101
0 votes
0 answers
69 views

Time Complexity help for Hashmaps using AVL Trees and separate chaining

Cheers, I am implementing a hashmap using separate chaining and AVL Trees as buckets but I am having some trouble proving some time complexities. I have to find the worst and average time complexities ...
user avatar
1 vote
1 answer
48 views

Architecture for an app that shows posts in real time in a map based on location

I would love to hear some more expert opinions on a problem related to bulk real time information, here some details: Use Case The users view in real time a map with determined places. The users can ...
user avatar
1 vote
0 answers
77 views

Near Real-Time Data Aggregation using Complex Queries

We are working on a multi-tenant SaaS product that has a ledger of all tenant customer transactions, which we use to track invoice states on a line-item level. This particular application will result ...
user avatar
-1 votes
2 answers
155 views

Need help with thinking about design to build bidding service [closed]

I have a legacy service written in a procedural style which I am rewriting. I want to improve the maintainability first and foremost; the code gets tweaked often as the business rules change over time....
user avatar
  • 175
-1 votes
4 answers
273 views

How to avoid the accident that an item price is updated right after a user clicks purchase?

This is not a real scenario. This is just my assumption that such an accident may happen if the the following events really occur in this chronological order First, a user clicked "puchase" ...
user avatar
0 votes
2 answers
61 views

Do 1:1 mobile video call face filters belong on Client or Server?

I am drafting architecture for 1:1 video calling mobile app with face filters (face recognition). I could use p2p e.g. WebRTC because more savings less complexity. However, continuous face rec during ...
user avatar
  • 103
2 votes
1 answer
44 views

Need advice on approach for IOT project

I need some advice on an IOT project I am working on. Here are some details about what I am trying to do: Use Case: Periodically receive data from the 3rd party vendor API in real time and allow users ...
user avatar
1 vote
0 answers
77 views

FreeRTOS based application architecture

I am going to use the FreeRTOS on my hardware platform. My plan is to model individual tasks as C structs containing task execution period, task priority, task stack size, task name and maybe another ...
user avatar
  • 141
0 votes
1 answer
151 views

Integrating with real-time data from multiple devices with accelerometer

I am designing a MVP for a simple gamification system for a trampoline park(s). External company is providing bracelets for customers, then collecting this data in their own web app, where they ...
user avatar
  • 67
1 vote
1 answer
250 views

Instant Messaging with WebSocket

I am trying to develop an Instant messenger using WebSocket. I have multiple instances of my servers running (say server1 , server2). Two users(say userA , userB) who wants to chat with each other. ...
user avatar
  • 149
1 vote
2 answers
154 views

Architecture to populate graphs from real-time data

I'm developing an application where graphs need to be populated from realtime data. The Real-Time data comes from a Kafka Queue. How should I send this real-time data to the front-end which is in ...
user avatar
1 vote
1 answer
115 views

Develop Windows based application with 1/10th milisecond precision

I want to develop a Windows based application that would interface with external hardware using serial communication. The hardware would in turn relay back the feedback to the application. This entire ...
user avatar
0 votes
0 answers
109 views

How to design realtime deeplearning based application for robotics using python?

I have created a machine learning software that detects objects(duh!), processes the objects based on some computer vision parameters and then triggers some hardware that puts the object in the ...
user avatar
  • 67
0 votes
0 answers
28 views

The optimal way to simulate vision detection from limited visual field

I need to build an experiment based on a scientific paper in which a high number of agents have a divided visual field (with limited detection distance) and have to detect other agents in a 2D ...
user avatar
0 votes
2 answers
345 views

Should I separate send and receive methods into tasks for a CAN-BUS reader on ESP32?

I am developing a CAN-BUS reader (logger) with an ESP32 microcontroller. This sample uses RTOS and several task to send and receive CAN-BUS messages. https://github.com/espressif/esp-idf/tree/master/...
user avatar
  • 293
0 votes
0 answers
172 views

Real-time application and RDBMS bottleneck

I'm currently developing a real-time application using .NET Core 2.2. Many IOT devices (could be up to 100) are concurrently sending messages via MQTT. These messages are parsed, elaborated to extract ...
user avatar
2 votes
1 answer
220 views

What is a good technique to process an event stream to ensure sequential or transactional consistency

I am trying to improve an event-driven processing system, which is having a few problems because the events are not guaranteed to arrive in the correct chronological sequence. This is due to batching ...
user avatar
  • 131
-5 votes
1 answer
42 views

Easiest way to add tracking of interactivity (mouse clicks, scrolls, multi-touch gestures) to a React Web App already developed?

Context: we have a responsive web application developed in React.js, which we have successfully used a few times to run user studies on Amazon Mechanical Turk. Now, we would like to add interactivity ...
user avatar
1 vote
1 answer
72 views

Displaying periodic simulations via web interface

The gist of my problem is that I'm trying to make a basic simulation for periodically collected data. More specifically, given the position of a radio antenna, the RF patterns would be simulated ...
user avatar
1 vote
1 answer
102 views

Backend architecture for a datastore with real-time events?

I need to build a data store for transactions that meets the following requirements: A 'transaction' is effectively a state machine which moves through a number of statuses during its lifecycle and ...
user avatar
0 votes
2 answers
441 views

copying of memory vs pointer to memory in real time systems

While working in a multi-threaded environment, then data sharing seems to be a good option to pass data between threads. While working with real time systems it is apparently advisable to go with ...
user avatar
1 vote
2 answers
4k views

Why is Modbus/TCP not considered a real-time protocol?

It is often stated that Modbus/TCP isn't a real-time protocol. It is also "known" that Modbus isn't a real-time protocol. This despite the fact that the original Modbus protocol is highly ...
user avatar
2 votes
1 answer
81 views

Realtime Live Auto updates for web applications

It's about trying to understand the architecture behind how updates to web application is done. I've been working on a non-public website that has bunch of records displayed in a UI, now the ...
user avatar
  • 129
1 vote
1 answer
478 views

How to write integration tests for a distributed system highly dependant on current date and flow of time?

My team and I are trying to set up some integration tests of a distributed system that is highly dependant on the flow of time and events coming from external sources. Just to give you an idea about ...
user avatar
  • 241
2 votes
2 answers
6k views

Build a realtime chat app which stores messages in a MySQL database

I'm thinking of making a realtime chat app that would allow me to store user messages into a MySQL database. So far these are the two ideas I have. 1) Create and API which the message is sent to and ...
user avatar
0 votes
2 answers
188 views

Trigger no more than M events per second, simulate N increments per second

What algorithm should I use, to simulate a continual stream of N increments each second — not writing a loop, but instead by timed-interval events, no more than M events per second? I am implementing ...
user avatar
  • 191
-1 votes
1 answer
548 views

Real time Querying on Real-time Data

I have data flowing from Kafka into MongoDB real-time. This is just raw data and various APIs are served using this data in real time. The APIs respond using aggregation queries. However when data to ...
user avatar
  • 101
3 votes
0 answers
1k views

Why does clock() function in C from <time.h> header only returns clock ticks in increments of 10's of thousands?

I am curious to know why does the function clock() only returns values in increments of 10,000 and is not able to return values in between. I have the following code: #include<iostream> #...
user avatar
  • 2,197
4 votes
1 answer
852 views

What is so different about RTOS compared to normal programming?

Classical programs run a loop with prioritized ISRs. How do RTOS do this differently of better?
user avatar
1 vote
3 answers
198 views

Strategy to maintain data efficiency in real time application

Lets say there is a real time application where a list is shared among multiple people and updates done to the list must propagate to all the subscribers of the list. Like in trello. The list is ...
user avatar
1 vote
2 answers
2k views

Realtime statistics of last 60 seconds with O(1) time and memory [closed]

What would be a clean design for an app that is to store transactions and return a overview of the ones received in the last 60 seconds, operating in constant time and memory (O(1))? The app should ...
user avatar
  • 21
4 votes
1 answer
3k views

Architecture for real-time application, avoiding duplicate messages for queues with no active consumers

I am designing the architecture of an application which has your typical backend and frontend components. The frontend application must be semi real-time. For this requirement, the backend is pushing ...
user avatar
-1 votes
1 answer
81 views

Is this a named pattern: Injecting specialized objects to detect fault conditions

I've got an idea, but I wonder whether somebody else already has named it. I'm writing code to control a machine with actuators, motors, etc., and software will be responsible for preventing ...
user avatar
  • 1,193
3 votes
0 answers
1k views

pinging vs socket vs push notifications

I develop mobile apps, now im working on a very big project building a tool for the employees to be heavily used. Now the app is on development mode, and testing with few users, but will need to be ...
user avatar
0 votes
1 answer
129 views

Real-time application : Is this a good way of communicating with my server?[very frequent transmission]

I'm making an application that collects timestamp and GPS data from a user's phone in intervals along with some other information. Currently, every tuple generated is sent via TCP to my application ...
user avatar
  • 3
1 vote
1 answer
68 views

Looking for best way to implement BFD module

I'm going to implement a BFD module (for MPLS LSPs) in VxWorks and was wondering the following: Is it better to create and use a separate timer instance (<2048) per BFD session at the specified Tx ...
user avatar
  • 139
1 vote
4 answers
3k views

Web application Architecture for real time app

Usecase: End user places an order on the company website, the order needs to be pushed to company admin's screen with small latency.The company has multiple branches(identified by compId and branchId ...
user avatar
  • 119
1 vote
1 answer
153 views

Save realtime data on client vs on server

I need to implement a site with a real-time graph. I'm currently using WebSockets and chartjs.org for displaying the values. Now I'm not sure whether I should send all data points in every message ...
user avatar
  • 229
9 votes
3 answers
7k views

How to modify software to become real-time? [closed]

For the first I would like to mention that I'm newbie in real-time systems programming That's why I'm not sure if my questions are correct. Sorry for that But I need some help Question in short: How ...
user avatar
4 votes
1 answer
343 views

Can heavy use of the service pattern substitute pure functions without losing benefits?

There are huge benefits to pure functions in functional programming, but can the same benefits be obtained in imperative programming with heavy use of the service pattern? I ask because I want to ...
user avatar
  • 287
0 votes
1 answer
123 views

Where should I store stream wide metadata in stream processing?

I am building a stream processing architecture and was wondering what to do with metadata about the stream. For instance, every message of data coming from a source has the same attribution, as it ...
user avatar
1 vote
3 answers
264 views

how to calculate communication waste in Scrum ?

communication waste is the time spent in communication between team members in the following cases: delivering features which are not needed by the customer delivering partial features (analysis, ...
user avatar
1 vote
3 answers
2k views

How to detect real time change in API response

I have an API endpoint written for version and health of tomcat applications. It returns a json response with details. I wanted to know, how can i monitor any change in the json response in real time. ...
user avatar
3 votes
0 answers
764 views

How do I implement Socket.io Leaderboard using AWS Dynamo DB cost-effectively?

Every time a user submits a score, Socket.IO causes all the users get their leaderboard updated. This causes every single user connected through Socket.IO to read from the Dynamo DB database (every ...
user avatar
  • 31
1 vote
1 answer
1k views

How to test software with near-real-time requirements?

So, I've been refactoring my videoconferencing application for a while, covering it with unit and general tests. Finally, I got to the point where I need to write a loopback test with 1 client sending ...
user avatar
  • 155