Questions tagged [real-time]
Real-time processing means processing data as it is created or modified. This is in contrast to batch processing, wherein data is processed in large chunks at a specific interval (e.g. every night).
120
questions
0
votes
0
answers
42
views
System Design for filter and notify user
I'm currently trying to design a notification system to notify users about data that match with they interests. Today we have a similar system running in production, but doesn't have a good ...
-2
votes
1
answer
48
views
System design to present live query results over realtime data
Question
What is a good storage layer, coding paradigm, and query language for computing over realtime data?
Use-case
For example, stock and options prices are (essentially) realtime data streams. I ...
- 157
0
votes
0
answers
41
views
Design efficient real-time data subscription to reduce service workload
I am designing a service for a hobby app in JavaScript and looking for feedback on best way to design the service.
TickerService
Accepts an array of stock tickers. eg: ['AAPL', 'AMZN', ...]
...
- 1
0
votes
1
answer
188
views
Best practices to deal with realtime data in REST API, Postgres and Redis
Overview
Today we have a monolithic service that deals with realtime time data. The monolithic are composed by follow stack: Django + AuroraDB Cluster (Postgres AWS) + Varnish cache and cloudfront at ...
1
vote
1
answer
308
views
Performing many SQL inserts quickly without using BulkCopy
I have a C# system that reads thousands of messages per minute from a Kafka topic. The messages are filtered and I need to insert the filtered messages into a MSSQL table.
A normal insert statement is ...
0
votes
1
answer
173
views
Solution architecture for a Kafka streaming to website system
Not sure if this is the correct forum for this question - but I could really use some advice.
I need to design a system (within certain constraints).
The system must implement the following logic:
A ...
0
votes
0
answers
41
views
Handling realtime notification receivers in a paginated application
I am working on an application right now, which is fully paginated. The application basically providers the client (a browser SPA) with a list of all data items other users added, and it does that in ...
0
votes
1
answer
302
views
Architecture for alerts system based on API - would this be too slow a design?
(Hi I’m new here - I believe this question suits here from the guide saying “ architecture, and design” - however if there is a better place, just pop me a comment. As there’s no actual code it doesn’...
- 101
1
vote
1
answer
56
views
Architecture for an app that shows posts in real time in a map based on location
I would love to hear some more expert opinions on a problem related to bulk real time information, here some details:
Use Case
The users view in real time a map with determined places.
The users can ...
- 137
1
vote
0
answers
139
views
Near Real-Time Data Aggregation using Complex Queries
We are working on a multi-tenant SaaS product that has a ledger of all tenant customer transactions, which we use to track invoice states on a line-item level. This particular application will result ...
- 227
-1
votes
2
answers
171
views
Need help with thinking about design to build bidding service [closed]
I have a legacy service written in a procedural style which I am rewriting. I
want to improve the maintainability first and foremost; the code gets tweaked
often as the business rules change over time....
- 175
-1
votes
4
answers
277
views
How to avoid the accident that an item price is updated right after a user clicks purchase?
This is not a real scenario. This is just my assumption that such an accident may happen if the the following events really occur in this chronological order
First, a user clicked "puchase" ...
- 113
0
votes
2
answers
62
views
Do 1:1 mobile video call face filters belong on Client or Server?
I am drafting architecture for 1:1 video calling mobile app with face filters (face recognition).
I could use p2p e.g. WebRTC because more savings less complexity. However, continuous face rec during ...
- 103
2
votes
1
answer
54
views
Need advice on approach for IOT project
I need some advice on an IOT project I am working on. Here are some details about what I am trying to do:
Use Case:
Periodically receive data from the 3rd party vendor API in real time and allow users ...
- 129
1
vote
0
answers
107
views
FreeRTOS based application architecture
I am going to use the FreeRTOS on my hardware platform. My plan is to model individual tasks as C structs containing task execution period, task priority, task stack size, task name and maybe another ...
- 141
0
votes
1
answer
158
views
Integrating with real-time data from multiple devices with accelerometer
I am designing a MVP for a simple gamification system for a trampoline park(s).
External company is providing bracelets for customers, then collecting this data in their own web app, where they ...
- 67
1
vote
1
answer
269
views
Instant Messaging with WebSocket
I am trying to develop an Instant messenger using WebSocket.
I have multiple instances of my servers running (say server1 , server2). Two users(say userA , userB) who wants to chat with each other. ...
- 149
1
vote
2
answers
177
views
Architecture to populate graphs from real-time data
I'm developing an application where graphs need to be populated from realtime data.
The Real-Time data comes from a Kafka Queue.
How should I send this real-time data to the front-end which is in ...
- 105
1
vote
1
answer
121
views
Develop Windows based application with 1/10th milisecond precision
I want to develop a Windows based application that would interface with external hardware using serial communication. The hardware would in turn relay back the feedback to the application. This entire ...
- 147
0
votes
0
answers
110
views
How to design realtime deeplearning based application for robotics using python?
I have created a machine learning software that detects objects(duh!), processes the objects based on some computer vision parameters and then triggers some hardware that puts the object in the ...
- 67
0
votes
0
answers
28
views
The optimal way to simulate vision detection from limited visual field
I need to build an experiment based on a scientific paper in which a high number of agents have a divided visual field (with limited detection distance) and have to detect other agents in a 2D ...
0
votes
2
answers
431
views
Should I separate send and receive methods into tasks for a CAN-BUS reader on ESP32?
I am developing a CAN-BUS reader (logger) with an ESP32 microcontroller.
This sample uses RTOS and several task to send and receive CAN-BUS messages.
https://github.com/espressif/esp-idf/tree/master/...
- 293
0
votes
0
answers
172
views
Real-time application and RDBMS bottleneck
I'm currently developing a real-time application using .NET Core 2.2. Many IOT devices (could be up to 100) are concurrently sending messages via MQTT. These messages are parsed, elaborated to extract ...
- 101
3
votes
1
answer
424
views
What is a good technique to process an event stream to ensure sequential or transactional consistency
I am trying to improve an event-driven processing system, which is having a few problems because the events are not guaranteed to arrive in the correct chronological sequence. This is due to batching ...
- 141
-5
votes
1
answer
50
views
Easiest way to add tracking of interactivity (mouse clicks, scrolls, multi-touch gestures) to a React Web App already developed?
Context: we have a responsive web application developed in React.js, which we have successfully used a few times to run user studies on Amazon Mechanical Turk. Now, we would like to add interactivity ...
- 101
1
vote
1
answer
72
views
Displaying periodic simulations via web interface
The gist of my problem is that I'm trying to make a basic simulation for periodically collected data.
More specifically, given the position of a radio antenna, the RF patterns would be simulated ...
1
vote
1
answer
119
views
Backend architecture for a datastore with real-time events?
I need to build a data store for transactions that meets the following requirements:
A 'transaction' is effectively a state machine which moves through a number of statuses during its lifecycle and ...
- 111
0
votes
2
answers
626
views
copying of memory vs pointer to memory in real time systems
While working in a multi-threaded environment, then data sharing seems to be a good option to pass data between threads.
While working with real time systems it is apparently advisable to go with ...
1
vote
2
answers
6k
views
Why is Modbus/TCP not considered a real-time protocol?
It is often stated that Modbus/TCP isn't a real-time protocol. It is also "known" that Modbus isn't a real-time protocol. This despite the fact that the original Modbus protocol is highly ...
- 131
2
votes
1
answer
235
views
Realtime Live Auto updates for web applications
It's about trying to understand the architecture behind how updates to web application is done.
I've been working on a non-public website that has bunch of records displayed in a UI, now the ...
- 129
1
vote
1
answer
497
views
How to write integration tests for a distributed system highly dependant on current date and flow of time?
My team and I are trying to set up some integration tests of a distributed system that is highly dependant on the flow of time and events coming from external sources. Just to give you an idea about ...
- 251
2
votes
2
answers
7k
views
Build a realtime chat app which stores messages in a MySQL database
I'm thinking of making a realtime chat app that would allow me to store user messages into a MySQL database. So far these are the two ideas I have.
1) Create and API which the message is sent to and ...
0
votes
2
answers
227
views
Trigger no more than M events per second, simulate N increments per second
What algorithm should I use, to simulate a continual stream of N increments each second — not writing a loop, but instead by timed-interval events, no more than M events per second?
I am implementing ...
- 191
-1
votes
1
answer
555
views
Real time Querying on Real-time Data
I have data flowing from Kafka into MongoDB real-time. This is just raw data and various APIs are served using this data in real time.
The APIs respond using aggregation queries. However when data to ...
- 101
4
votes
0
answers
1k
views
Why does clock() function in C from <time.h> header only returns clock ticks in increments of 10's of thousands?
I am curious to know why does the function clock() only returns values in increments of 10,000 and is not able to return values in between.
I have the following code:
#include<iostream>
#...
- 2,267
4
votes
1
answer
907
views
What is so different about RTOS compared to normal programming?
Classical programs run a loop with prioritized ISRs. How do RTOS do this differently of better?
1
vote
3
answers
204
views
Strategy to maintain data efficiency in real time application
Lets say there is a real time application where a list is shared among multiple people and updates done to the list must propagate to all the subscribers of the list. Like in trello.
The list is ...
- 121
1
vote
2
answers
2k
views
Realtime statistics of last 60 seconds with O(1) time and memory [closed]
What would be a clean design for an app that is to store transactions and return a overview of the ones received in the last 60 seconds, operating in constant time and memory (O(1))?
The app should ...
- 21
4
votes
1
answer
4k
views
Architecture for real-time application, avoiding duplicate messages for queues with no active consumers
I am designing the architecture of an application which has your typical backend and frontend components. The frontend application must be semi real-time. For this requirement, the backend is pushing ...
- 41
-1
votes
1
answer
82
views
Is this a named pattern: Injecting specialized objects to detect fault conditions
I've got an idea, but I wonder whether somebody else already has named it.
I'm writing code to control a machine with actuators, motors, etc., and software will be responsible for preventing ...
- 1,213
3
votes
0
answers
1k
views
pinging vs socket vs push notifications
I develop mobile apps, now im working on a very big project building a tool for the employees to be heavily used. Now the app is on development mode, and testing with few users, but will need to be ...
- 830
0
votes
1
answer
130
views
Real-time application : Is this a good way of communicating with my server?[very frequent transmission]
I'm making an application that collects timestamp and GPS data from a user's phone in intervals along with some other information.
Currently, every tuple generated is sent via TCP to my application ...
- 3
1
vote
1
answer
70
views
Looking for best way to implement BFD module
I'm going to implement a BFD module (for MPLS LSPs) in VxWorks and was wondering the following:
Is it better to create and use a separate timer instance (<2048) per BFD session at the specified Tx ...
- 139
1
vote
4
answers
4k
views
Web application Architecture for real time app
Usecase: End user places an order on the company website, the order needs to be pushed to company admin's screen with small latency.The company has multiple branches(identified by compId and branchId ...
- 119
1
vote
1
answer
157
views
Save realtime data on client vs on server
I need to implement a site with a real-time graph.
I'm currently using WebSockets and chartjs.org for displaying the values.
Now I'm not sure whether I should send all data points in every message ...
- 352
9
votes
3
answers
7k
views
How to modify software to become real-time? [closed]
For the first I would like to mention that I'm newbie in real-time systems programming
That's why I'm not sure if my questions are correct. Sorry for that
But I need some help
Question in short:
How ...
- 215
4
votes
1
answer
378
views
Can heavy use of the service pattern substitute pure functions without losing benefits?
There are huge benefits to pure functions in functional programming, but can the same benefits be obtained in imperative programming with heavy use of the service pattern?
I ask because I want to ...
- 287
0
votes
1
answer
128
views
Where should I store stream wide metadata in stream processing?
I am building a stream processing architecture and was wondering what to do with metadata about the stream. For instance, every message of data coming from a source has the same attribution, as it ...
- 113
1
vote
3
answers
276
views
how to calculate communication waste in Scrum ?
communication waste is the time spent in communication between team members in the following cases:
delivering features which are not needed by the customer
delivering partial features (analysis, ...
- 37
1
vote
3
answers
2k
views
How to detect real time change in API response
I have an API endpoint written for version and health of tomcat applications. It returns a json response with details. I wanted to know, how can i monitor any change in the json response in real time. ...
- 119