Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

Filter by
Sorted by
Tagged with
-2
votes
1answer
20 views

Data structure to store user data and sort on multiple fields

What would be the best data structure to use if I want to store user data for a large scale project. For example, I want to store student data, I have a Java object model for each student with ...
-4
votes
0answers
30 views

Most efficient database for the location based statistics [closed]

I would like to create a simple application where I will show the tourists counts of a particular location with the tourist origin. For example Spain got 20 million visitors from which 5 million of ...
0
votes
3answers
77 views

Need advice on reporting with big amounts of data

I have a monolith application in .net core 3.0 with entity framework core 3.0. using: a table with ~3 million records. Its structure is BusinessUnitId | ProfileId | Amount(it has more fields, but ...
17
votes
2answers
19k views

Best method to implement a filtered search

I would like to ask you, your opinion when it comes to implement a filtered search form. Let's imagine the following case: 1 Big table with lots of columns It might be important to say that this SQL ...
-3
votes
0answers
8 views

Most efficient approach for syncing message read and message posted data between apps on desktop and iOS app?

I have a basic CRUD app that has a desktop client and an iOS client. Users can post and update their data on either one. However, once they have posted something using either phone/desktop, I want ...
0
votes
2answers
124 views

What's the best way to save/track change requests on multiple objects in a database?

I have a schema containing contracts, sub-contracts, services offered by each of those and sub-services. I also have a application that allows to extend the duration of the contract and to modify ...
1
vote
1answer
55 views

Is my server design safe regarding multiple threads and concurrent database reads/writes?

I'm making a chat server using sockets and a MySQL database, and after it's working I want to expand it to become a more complex game server. I want to know whether my design is missing anything. ...
113
votes
3answers
140k views

Creating database connections - Do it once or for each query?

At the moment I create a database connection when my web page is first loaded. I then process the page and run any queries against that conection. Is this the best way to do it or should I be creating ...
-1
votes
1answer
45 views

Data sharing on a multi-instance application

I would like to know which architecture is more suitable when considering data sharing between tenants: a Multi-instance (Single-tenant) or Multi-tenant architecture with a database by tenant. ...
1
vote
2answers
64 views

Separate database in microservice architecture

I'm trying to create project siteWeb and mobile app with clean Architecture and Microservices. create identity service with separate database A and make another api service with other database B. for ...
1
vote
1answer
58 views

how simulate sequential consumers in Kafka

I have kafka consumer that get data from kafka and do some process if some circumstances meet and send it for another consumer to do it's job. How can i build something like this? One way I think ...
2
votes
2answers
88 views

DB design while designing the system

Recently we had a discussion if rough DB schema (high level tables and columns) design should be part of system design phase or not. We have two confronting approaches in the company. Let's assume ...
0
votes
1answer
39 views

Ensure that Room No. of a Course's Section is within the permitted Rooms for that Course with some Constraint

I am facing difficulties establishing constraints in the following situation. This is a hypothetical scenario for practicing DB designing. Imagine, a rule in a School states that the Courses there ...
-4
votes
3answers
123 views

Why local database should be the only source of truth for mobile application?

Being mobile developer for quite some time (ios/android) I've learnt that local database is very rarely needed. Mobile application are mobile by definition, they usually serve only as clients to ...
3
votes
2answers
96 views

Is Excel data model as good as or better than Microsoft Access?

I'm working with several Excel worksheets and workbooks that need to consolidated or linked to each other. These workbooks will track client interactions for different team members. I did the initial ...
4
votes
1answer
104 views

Solving file system dependency with database storage

I have a software project A which makes API calls to a third-party software B that is heavily based on data stored on the file system. Also, those software and file systems are distributed on servers ...
2
votes
1answer
47 views

From Oracle to Apache Parquet : how to handle eventual consistency?

I have an existing production Oracle Database. However, there are performance issues for certain kind of operations, because of the volume of the data, or the complexity of queries. That's why I ...
-5
votes
0answers
25 views

How to load initial data, except SQL

Could someone please tell about ways to load initial data? Most obvious way is to use SQL script. But in case if ORM used in project, is it will be better to use Dao to write to database(or initial ...
116
votes
3answers
56k views

How is localStorage different from indexedDB?

localStorage and indexedDB are used for offline storage of data in HTML5. What are their key differences and which one is preferable in what situations?
0
votes
2answers
281 views

Unit Test or Integration Test

I recently asked a question about design and got suggestion about how to structure my code. I'm still working on design so I only have pseudo code, but this is what I had in mind. class TableManager(...
1
vote
1answer
27 views

Pattern for schema split across sources

I was not sure how to title this question, but bear with me. My company is building a new product and for it we will use a third-party service (let's call it ENB for short) to be responsible for many ...
4
votes
2answers
810 views

Tracking online users with a stateless architecture

I own a stateless architecture, exchanging a JWT token between client and server at authentication time. I want to store in database (once per day) a timestamp related to the user access. Indeed, I ...
0
votes
1answer
50 views

MVC database connection .NET

Introduction I intend to create a .NET WinForms application (this will be a toy application) which connects to a SQL Server database backend. I have designed and implemented an object-oriented model ...
1
vote
1answer
78 views

How to keep user inputs consistent with assumed inputs to slow computations at the backend?

Background: I'm writing a clinical trials simulator. The user defines future trial options, eg a trial with 100 placebo patients, 200 treatment patients, "optimistic" outcome scenario, etc. There can ...
1
vote
1answer
74 views

Write complex query OR write simpler queries and combine the results in Backend service?

Take the following statement for example: 'Get results for ALL Active students of a Course'. Naturally, this query has 2 parts, Get all active students of the course. Get results of those ...
2
votes
0answers
102 views

Is it possible to implement insert file operation in modern extent-based filesystems?

The problem was already discussed here. But there was not consensus on this topic. I have some thoughts on how insert operation can be implemented for some popular file systems. If FS has extent-...
1
vote
1answer
46 views

Structuring a db to allow multiple products to hit a select component list

I am designing my first ever database architecture, no real framework to build up on. So I am working on my own! Terrifying. A huge question I have is for having two separate tables, one being ...
1
vote
1answer
62 views

Best approach to handle user statuses and keep their history

I am developing a Spring application which has a few different modules. There will be a bunch of users added in the database. I want to add a feature, which will allow me to track users' availability,...
0
votes
2answers
98 views

Is database linked list a good architecture for replies and reviews system when using mongodb?

I want to create a system of user reviews and replies to the reviews in a website. There can be replies to replies. I'm using mongodb database which I think is an important detail. The review ...
0
votes
3answers
146 views

How to append a chunk of fixed size data to a file and make sure this chunk doesn't get fragmented on disk?

So i want to understand how DBMS implementation works To give an example : MySQL implements each tables with its own pages, which are 16KB so each table is a file, and is a multiple of 16KB, ...
2
votes
1answer
45 views

Data modeling user-object permissions, between claims and metadata

I've been thinking about how to map user claims to permissions in a way that scales from a read and write perspective, probably using a stand-alone service. What we have today doesn't scale for ...
20
votes
5answers
10k views

What is the proper way to synchronize data across microservices?

I am relatively new to microservice architecture. We have a moderately sized web application and I am weighing the pros and cons of breaking it out into microservices instead of a monolithic system we ...
-2
votes
1answer
75 views

Elasticsearch and PostgreSQL combination

I have a marketplace application and I stored data in a PostgreSQL. I have performance problem for product search, I know I can improve search performance if I use Elasticsearch instead of PostgreSQL ...
0
votes
0answers
25 views

When should I run my database migrations on a multi-server php app deployment thas uses a legacy database that previously had no migrations?

I have a multi-server laravel php application running behind a load balancer and I deploy the code using aws codebuild and codepipeline. The approach is to deploy my application one server at the ...
0
votes
1answer
53 views

Should I skip some migrations on production in case of migrating a legacy database that has not been generated via migration script?

I have a complex PostgreSQL database structure that is consisted with views, materialized views and foreign wrappers. The database schema is manually updated without any sort of migration script, ...
1
vote
2answers
106 views

which layer must check if something(like account etc) exist in database

I am implementing a web api and i have crated three layers,data access layer,business layer and presentation layer. In data access layer I have a method which create a user.I use a transaction which ...
0
votes
1answer
45 views

Architecture to show the number of results within the filter

I am building a web application connected to MySQL database that is supposed to work like an e-shop. User should be able to filter products, but I want to display the number of results that would be ...
2
votes
3answers
200 views

When would I use a database over an object-store for large blobs?

I am designing an application where there will be one column that stores a large amount of text. I'm debating whether the text should be stored in the database itself, or whether the database should ...
12
votes
4answers
24k views

Best approach for a database of long strings

I need to store questions and answers in a database. The questions will be one to two sentences, but the answers will be long, at least a paragraph, likely more. The only way I know about to do this ...
-4
votes
2answers
73 views

What are distributed transactions supposed to satisfy?

I am lost when reading Design Data Intensive Applications. In a nondistributed database system, nondistributed transactions are to satisfy ACID. In a distributed data store, what are distributed ...
-3
votes
1answer
48 views

How to store a limited set of custom properties without using an EAV Schema

I'm designing a solution in .Net which will be hosted on Azure. I have a system which will allow users to specify custom characteristics for their objects which they will track over time. I am certain ...
0
votes
0answers
12 views

Dynamically retrieve associated data in View

I'm afraid on StackOverflow this question might be rejected as "opinion based". I'm using CakePHP 3.7.9 and in one View I have a form with a select and a number input: echo $this->Form->...
3
votes
1answer
203 views

Designing database job monitoring

I have several Oracle database where my in-house applications are running. Those applications use both dba_jobs and dba_scheduler_jobs. I want to write monitoring function: check_my_jobs which will ...
2
votes
3answers
531 views

Database migrations and Code deployment

Should Database migrations be in included in the same deployment project as the code? If they are not included in the same deployment, the Database deployment package and the Code deployment package ...
0
votes
1answer
105 views

Custom service layer logic vs Custom DB query

I am trying to confirm best practice around architecture I am working with. Frontend <-> API gateway <-> Service layer (Rest API) <-> DB I have an incoming request which is asking for huge ...
0
votes
1answer
218 views

Appropriate design / technologies to handle dynamic string formatting?

recently I was tasked with implementing a way of adding support for versioning of hardware packet specifications to one of our libraries. First a bit of information about the project. We have a ...
3
votes
2answers
103 views

For Evolutionary Database Tools like Flyway or LiquidBase, it is better to integrate the tool or left it as a stand alone step in the CI process?

Although I believe my question is language and DB agnostic, I will provide the specifics technologies in this case study because it might help understanding my question (sorry for my English). It's ...
5
votes
1answer
150 views

Transactional requests and failure recovery in database systems

Let's say we deal with a quite naive, simple version of a database, which uses the classic version of redo-undo logs for failure recovery. For a "classic" request transferring an amount from one ...
9
votes
1answer
665 views

Data durability guarantees in Kafka

Is it wise to use kafka as the 'source of truth' for mission critical data? The setup is: kafka is the underlying source-of-truth for the data. -querying is done on caches (I.e. Redis, ktables) ...
0
votes
1answer
127 views

Is my database schema going to work? Tracking git issues

My boss wants an automated weekly report on how many git issues have been moved to and from various "lanes" (pipelines) on a virtual kanban board. I wrote a quick python script to get that data using ...

1 2 3 4 5 36