Questions tagged [redis]

open-source, networked, in-memory, key-value data store

Filter by
Sorted by
Tagged with
-1
votes
0answers
21 views

Help to structure a machine learning application

I have a recommendation machine learning API that recommends products to clients. The trained model is stored in memory machine for fast access when the API is called. The problem is: we want to ...
4
votes
3answers
267 views

Best way to reduce database writes?

I couldn't find a similar enough thread, so i'm making a new one. My question is about reducing database writes. Context is that I have an application which increments a number in a database every ...
0
votes
1answer
70 views

Design review for a Cache aside system

I am maintaining a system where a list of string ids are stored against a container (using the term here as a generic entity) object. The database used is MySQL and the table structure looks like this....
0
votes
0answers
56 views

Best Architectural practices to combine Elasticsearch and Redis with MySQL

Idea: Multi-tenancy in 3 different layer Database: MySQL with multi database (database per tenant) Indexer: Elasticsearch Queue and Database load helper: Redis Problem: We have chosen to work with ...
0
votes
1answer
39 views

How do I decide an initial in-memory cache size given my DB size and expected load throughput?

(Purely for learning purposes) Say the DB contains 1 billion rows with 200 bytes per row = 200 GB of data. The traffic at peak is 1000 requests/s, with each request asking for one DB row. What cache ...
0
votes
1answer
50 views

Caching or in-memory table in Azure for performance

I am building an Angular web application that retrieves part of its data from a Azure SQL Database table via APIs developed in Azure Functions (with Azure API Management at the API Gateway). The data ...
1
vote
0answers
42 views

Architect Java Hashmap Hierarchy to its equivalent in Redis Cache?

I have the following Job class representing runs of a job. I have a list of start and end times in Job class because the same Job can be rerun. public class Job { private final List<...
0
votes
1answer
84 views

Key-value database for dynamic/changing keys

I have the following dataset: [234,565,678,90,66,7,8,44,5] [275,23,54,34,5] [4745,23,54,4,556,65,89,5,4,569,87,412] [75,273,59,5,4,567,412] [44,34,556,69,5,4,569,812] Each array represents a web ...
0
votes
0answers
34 views

Recommendations for data clustering in an application

I am making a social media app. I have a use case in which i have to cluster similar type of data. For example, take Instagram. When a user likes a post, we get a notification that 'x likes your post'...
0
votes
1answer
76 views

Design decision of reading XLSX file at once or intermittently

I am working on converting an existing python based monolith solution to a microservice. The current flow is pretty straight forward: Accept XLSX as input -> Run some complex algorithms based on ...
1
vote
1answer
69 views

Uncertain of cache solutions for CAD models

I am running a personal project of offering 3D model generation as a service (in this case CAD model). Since most of the generation of 3D models take a long time, I thought about using cache to ...
0
votes
0answers
40 views

Is it a good practice to store session in two different places?

I have a use case like this and I am wondering if this solution is a good practice or not. Say I have a website called dashboard.com and this is only for US region. When users login here, I am ...
-3
votes
1answer
58 views

is using redis correct to hold a dataset?

[Have restructured the question] We have a lot of DB readers and few writers where writers may perform longer operations. This is a real-time system so the read has to be performed as efficiently as ...
2
votes
0answers
49 views

Distributing tasks to several clients

I describe a very very simple scenario. I try to develop a system where many machines can help another one (server). I tried RabbitMQ and Redis, but both don't fulfil all requirements. I have 1-...
4
votes
1answer
2k views

Using Redis as a cache in a Multi-Tenant application

I'm working on building a multi tenant application which relies on redis as a cache. I'm currently researching a feasible way to achieve multi tenancy with redis. I see that redis enterprise offers ...
4
votes
0answers
657 views

Using distributed locks in microservice environment

I'm developing distributed system and trying to use best practices of microservice architecture. I was faced with a situation when I think I need something like distributed locks. Since I have not so ...
0
votes
0answers
58 views

How to cache timestamp based key values with a distributed cache?

I have a dataset in a key value store (similar to Cassandra). The data model is - Key - ServiceName / timestamp_in_milliseconds value - operation In the write side I get some operations for a ...
2
votes
0answers
56 views

Event Sourcing & Redis

I'm using StackExchange.Redis library, it seems perfect. Hard-coded events related BankAccount with ID 1: // UNSORTED COLLECTION public static List<Event> DomainEvents = new List<Event>()...
0
votes
0answers
207 views

How to handle server side pagination + processing large sets of data

So lets say I have a SQL query made by a server that fetches up to thousands of records at a time and I need to display those records in a grid(similar to AG Grid) on the client for the user to review....
4
votes
3answers
2k views

Different Redis instances for different microservices?

Say that we have Kubernetes running with n pods, each being a separate microservice. Now, some of those services need a caching layer. When we take into consideration: the fact that Redis was tested ...
0
votes
1answer
173 views

How to solve cache eviction when not using distributed cache in microservices?

Let's say we will not use distributed caching because we want that our microservices are independent and they do not have many single point of failures. Either we will have in-memory cache (simple ...
0
votes
1answer
301 views

Robust locking mechanism between n service instances

I have a job that runs every hour. There can be n identical Node.js microservices running in a stack. The job must be executed only once per hour, though, so I've implemented the following locking ...
-2
votes
1answer
140 views

Do I need a redis for caching object while MongoDB already cached frequently accessed items

I'm using mongodb for our web application. Recently I'm think of adding redis for caching objects which usually is the setting/configuration. But the mongodb already cached the frequently accessed ...
1
vote
1answer
988 views

Pros and Cons of using ASPNET.Core In-Memory Caching instead of Redis? [closed]

I know redis is a very robust caching solution and scales great, but when it comes to simpler non-enterprise websites I feel as if it's a bit too expensive (Azure Standard/C1: $100/m). I'm ...
5
votes
1answer
830 views

How is caching used within messaging apps?

I'm actually studying about web development. I was just asking why a lot of web apps and chats(Whatsapp, Telegram, Discord, and a lot, a lot more!) are using cache. I mean, after learning cache ...
1
vote
1answer
866 views

How to store counters on Redis while minimizing potential data loss

The application I'm working on has a need to create a counter representing the number of times each piece of content has been viewed. There are two goals here: Update the counter in real time ...
2
votes
1answer
277 views

How to better scale reads in a website: to CQRS or not to CQRS?

The company i'm working for it's rewriting a legacy web application and we are evaluating the better architecture to get the job done. The business domain is quite simple: there are few entities, ...
2
votes
1answer
459 views

How to think about queries in NoSQL

I'm writing a project for school - Tinder-like app. I thought of implementing database entirely in RDBMS (PostgreSQL). But now I think it's a good opportunity to get to know NoSQL systems a bit. I'm ...
4
votes
1answer
3k views

Microservices - sharing data through distributed in-memory cache?

I'm fairly new to both microservices and shared in-memory databases like Redis. I'm wondering though if it would be a good idea to combine the two to address the challenges of data duplication, ...
-3
votes
1answer
292 views

Design the cache and search feature like hotel booking?

I need to design the feature like in encommerce(consider hotel booking site) we can enter the keyword/partial-keyword and need to return the relevant results. For Example :- If site admin or hotel ...
1
vote
1answer
232 views

How to architect master/slave store/messages and de-duplication?

To clarify, as I'm not entirely sure how to summarise in the title. I have a redis cluster and a redis-backed scheduler (similar to sidekiq/resque, etc). My redis cluster has a master and a couple ...
0
votes
1answer
115 views

Architecture for distributed real time traffic capturing/viewer system

I have a scenario as following (you can think of it as a distributed Wireshark system): For a single capturing session, there are about 1 ~ 10 traffic capturing nodes distributed in a LAN network. ...
0
votes
1answer
122 views

Where should I store statistics about sent emails?

my application sends a batch of custom emails every day using mailgun (an email API service). I have a producer worker scheduled to run daily, and all does is query the database for the unique ...
0
votes
1answer
151 views

Removing Redis Entries from Multiple Redis Instances

I have an interesting problem that I'd like to solve. Let's say I have 2 deployments, US Data Center and Europe Data Center each containing Web Servers and a Redis Instance (replicated, but could ...
8
votes
2answers
6k views

Why is Redis called an in memory database when it provides persistence?

I would like to understand why is redis called an in memory database when it provides persistence similar to databases like MySQL or Postgres: http://oldblog.antirez.com/post/redis-persistence-...
3
votes
0answers
337 views

Redis set of sets - Is there a better Design?

I have a telegram bot that alerts whenever a price of certain asset reach a certain price. I have the following table in a DB: AlertId ChatId Price Asset So each Asset will have several ChatId (...
1
vote
1answer
799 views

Should I partition my messages with channels or with content?

I'm building a system using redis as a pub-sub system. I'm thinking about ways to represent multiple groups that clients can subscribe to. I could do this with one channel per group, but this could ...
3
votes
1answer
389 views

Is my use case for Redis appropriate?

I'm writing a web app that is basically facebook events, but more impromptu and real time. A user can create an event, and other online users in the area (30 mile radius) should immediately see the ...
-1
votes
2answers
238 views

Using NoSQL as MQ and for maintaining state

Currently we have an application that doesn't scale well because of multiple concurrent updates to a table which gets locked. The tech stack is ASP.NET and backend is in SQL Server. To avoid database ...
-2
votes
1answer
268 views

How to implement a scalable exists in large set query

I need to design a system where I need to check if a key exists or not. Currently I am already using aerospike for the cause but my use case is slightly different. I will have a key like this ( A ...
2
votes
1answer
319 views

Exploiting Apache Spark Data

and sorry if the question seems a bit naive. I'm currently reading tutorials about Kafka & Spark and there's something I can't figure out : how to exploit / expose the data Spark received. Here'...
3
votes
1answer
1k views

How does Redis (or any typical distributed cache) handle replication conflicts?

Suppose you set up a Redis cluster with one master and two slaves. Two clients are connected to each of the slaves. Both clients make conflicting changes at the same time: What happens if these ...
0
votes
1answer
219 views

What is the meaning of sharing state between processes?

The author here states that: Share state between processes. Run a long running batch job in one Python interpreter (say loading a few million lines of CSV in to a Redis key/value lookup table) and ...
0
votes
2answers
639 views

Algorithm for searching a domain name in list of wildcard

I have a list of domain name that contains wildcard ('*' character) Example: *.google.com *.domain.com ... It contains nearly 1 million domains. The data will be stored on a mongodb or redis ...
3
votes
0answers
1k views

How should I store user activities in ElasticSearch and figure out popular searches?

I've got java logging user activities to Fluentd, Fluentd is then writing these activities into an elasticsearch index. Every user activity is recorded, some include: User1 follows user2 User1 likes ...
1
vote
3answers
2k views

Is it ok to wrap a singleton?

I'm using SignalR with StackExchange Redis. What I gather from the basic usage doc is the way to implement this is using a singleton. I've never used this patter before. My class looks like this: ...
1
vote
1answer
355 views

How to implement searchable persistent queues?

We will implement a system , which will receive quite a lot of requests through web service interface. We will have to queue the requests and process them asynchronously. Consequently we will have to ...
2
votes
2answers
274 views

Web App Authentication's Architecture Related Query

I have a web app that makes use of node.js in the backend and angulajs on the front end. Let's say I have a user who has the ability to login via multiple systems; I must allow multiple logins. I ...
3
votes
1answer
419 views

Distributing work between TCP Clients

I have an application (run by a Windows Service) which connects to a TCP server (let's call the service 'Listener' from here on). [It's important to note that the TCP server is out of my reach, and ...
0
votes
0answers
401 views

how to create messaging storage structure in redis?

I am developing Node.js, Redis based chat application in php. I had an idea about chat logger structure but I required suggestions that how and what to choose proper data structure to create Redis ...