Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

Filter by
Sorted by
Tagged with
0
votes
1answer
146 views

Saga vs. Unit of work

What's the relationship between the two? Both are for atomic operations. Both results need to be either success, or failure with rollback to original state.
1
vote
2answers
102 views

How should I design an architecture for an endpoint to take in huge amount of data without loading the database?

Every day, there will be a set of new data that I will need to write and update to the database. This set of data can contain 8-10million records. This new set of data comes from Service A and it ...
2
votes
1answer
68 views

How to handle frequent update requests to an API which is connected to the database?

In a document editing web application such as Google Docs or Evernote, sync/save has to happen almost instantaneously to make sure data loss is minimal in case of a connection failure or a client ...
-2
votes
1answer
43 views

In DBMS are Secondary Indices ordered?

I've read that the Heap File Organization is usually used to structure a secondary index. This makes me suspect that usually, a secondary index is unordered, as opposed to the primary index, that is ...
0
votes
0answers
24 views

How to generate object Ids for both local and online storage?

My application has both offline (SQLite) and online (iCloud) storage. See: Offline First: How Do I Fetch Data? Now, as stated in my past question above, the application is offline first, and I'm ...
2
votes
0answers
134 views

Data Types Based Database

I was wandering if there exists some kind of database that organises data based on some declared type rather than tables or collections. I'll try to explain better with an example. Let's say I have ...
1
vote
2answers
138 views

Can a wide table be the correct choice in design?

I am relatively new to database architecture and I have been tasked with designing a database that will store 250 values per entry by my group. This is pushing the upper limit for fields in an Access ...
1
vote
3answers
120 views

How do I query something when I don't know how its structured?

How does one write & run queries against a database/warehouse if they have no knowledge of how the data is structured? By "structure" I mean table names, fields, etc. Unless I have table names, ...
1
vote
1answer
73 views

Offline First: How Do I Fetch Data?

I'm new to this community, I'm glad I found this and so I can ask questions like this one (as per https://meta.stackexchange.com/questions/68384/whats-the-difference-between-stack-overflow-and-...
0
votes
1answer
95 views

Recommended Long Term Data Storage Format

I have an application (written in Go) with a lot of GPS location information. Each record consists of a latitude and longitude plus somewhere around 50 fields of various pieces of meta data, some of ...
0
votes
1answer
75 views

How to manage breaking updates in an API & database

So I have been working on my mobile app for a while and it is live and in production. Because we released the app prematurely and are continuously improving current features and making them more ...
1
vote
1answer
202 views

Triangle relationship in database

I have a problem figuring out the relations between some of my database tables. The scenario for this buisiness case is a b2b buissnes. I have 3 tables: users companies distributors. A Company ...
0
votes
2answers
89 views

Correct place to store semi dynamic data

What I mean by semi dynamic data are data that are expected to be changed only with product versions. Now imagine this scenario, In version 1 of the product, I have item status : SUCCESS , FAIL In ...
1
vote
2answers
48 views

Architecture of “tags” for contacts in a marketing system and hard-coded IDs so functions can refer to them

I have these tables: contacts, tags, contact_tags. (contact_tags has columns contact_id and tag_id.) Administrators can manually create new tags via an internal website. Administrators can also ...
-3
votes
2answers
113 views

How to structure a database where a record can be a child of a parent or a grand parent record

I have these concepts for a database application: Session, Subsession and File A session can have multiple files A session can have multiple subsessions A subsession can have multiple files A session ...
-2
votes
1answer
97 views

Store line numbers of CSV in the database

In our environment, we often have to import CSV files into a database, where each line represents a record. Though there is no actual business use case for storing the original line numbers inside the ...
0
votes
2answers
119 views

What's the best way to save/track change requests on multiple objects in a database?

I have a schema containing contracts, sub-contracts, services offered by each of those and sub-services. I also have a application that allows to extend the duration of the contract and to modify ...
2
votes
1answer
30 views

Do I need to put foreign keys into a metadata repository?

As I was creating a metadata repository, I came across a doubt. As I'm new to databases, I was wondering if I should add the foreign keys data into the tables that have them (for context, imagine a ...
0
votes
0answers
42 views

Handling a large local database in a chrome extension

I'm creating a chrome extension which needs a large local database. After researching my options, I figured my best bet is probably to use the chrome.storage.local API and give my extension the ...
0
votes
0answers
113 views

Optimizing web dashboard with editable elements for performance by introducing ElasticSearch

To improve performance, we are optimizing our complex tabular web-based dashboard that loads a lot of data (many columns and rows). Many cells in the table are editable so the data is mutable. Our ...
2
votes
1answer
45 views

Data modeling user-object permissions, between claims and metadata

I've been thinking about how to map user claims to permissions in a way that scales from a read and write perspective, probably using a stand-alone service. What we have today doesn't scale for ...
0
votes
0answers
55 views

What is the correct way of setting and then differentiating between paid and free user in Firebase / Firestore?

I am developing a mobile application that will have both free and paid versions. The way I see how I could differentiate between what is a paid and free user is to set a property isPaid on their ...
0
votes
1answer
160 views

Should you clean up your DB after integration tests?

Is it good practice to clean up your DB after running integration tests (which tests APIs that put data into that DB)? I want to have a cleanup method after my integration tests run which will access ...
1
vote
1answer
95 views

Saving flags in the database without relationships to other tables

I have an admin panel which can set a lot of options which will affect some other services drastically. For example I got one that if I create a different category I can just insert it in the Category ...
3
votes
3answers
100 views

Delete user data or replace it with empty resource?

Say, we have a public facing REST API which lets user manipulate data using POST, GET and DELETE. The API is exposed via Service A which at the backend calls Service B where most of the business logic ...
0
votes
0answers
26 views

Sequence Diagram Assistance with external database

I have 5 Java/android classes which are in sequential order. The user inputs data into the text box and clicks next. Two things happen, 1 the data the user inputs is passed to the next activity and it ...
0
votes
1answer
80 views

Web application and persistence choices

Right now we have a highly analytical system which collects tons of various metrics. We need to display this data in a web ui for users to consume. Since we are in early stages of development we are ...
0
votes
1answer
50 views

Event planning application database schema

I am planning to develop an Event Planning Application. Currently I'm in the stage of planning the app structure, for example database schemas. Basically there will be two kinds of users: The event ...
2
votes
0answers
83 views

Data pipeline architecture: airflow triggered by message broker

Let us say we have: a web app with a Postgres DB that produces data over time, another DB optimized for analytics that we would like to populate over time. My goal is to build and monitor an ETL ...
1
vote
1answer
44 views

Multitenant setup with master account

I would like to make a SaaS which would be a multitenant system. Development is done in Laravel, which is less important. What I would like to know is how would you handle having a master admin ...
0
votes
1answer
126 views

Is my database schema going to work? Tracking git issues

My boss wants an automated weekly report on how many git issues have been moved to and from various "lanes" (pipelines) on a virtual kanban board. I wrote a quick python script to get that data using ...
-1
votes
1answer
56 views

What is the best design for this use case regarding bidirectional relationship? [closed]

For this question let's consider a system in which the users can match with each other (let's say for simplicity exactly like the dating apps). User1 matches User2 (and the other way around) only in ...
1
vote
2answers
133 views

Architecture/algorithm for unusual recommendation system

There are thousands (or tens of thousands) of possible movie titles. New user enters my website and selects hundreds of titles that he likes. The only single goal of my website is to output the list ...
0
votes
1answer
29 views

Update items based on a read operations on same items is a bad practice?

DynamoDB is the database solution, I'm using to store my items. It's a NoSQL/Document database and I got a real problem on real-time updates of items. First, the saved format is like { expiredTime:...
0
votes
2answers
170 views

Is it a software engineer's responsibility to structure data as opposed to a DBA?

As a backend engineer at a small company, we're running into a question of whether we need a DBA or another backend engineer (SWE2). The reason why this is a question stems from a difficulty in ...
0
votes
1answer
60 views

Strategies for Rotating MYSQL Encryption Keys

I have a web app that connects to a MYSQL database which has some encrypted data in it. The encrypted data is encrypted using the the mysql AES_ENCRYPT function and a key. Now, this key has been ...
0
votes
1answer
107 views

I have a data set of over a million addresses and I want to display the closest N locations to a given address or current location

I am a student working on a personal project which is essentially a location finder that will be on a website. I have a data set of over a million addresses and I want to display the closest N ...
-1
votes
2answers
131 views

When self-referencing in a table, do you prefer `parent_id = null` or `parent_id = id` for a row referencing to itself?

For the discussion, I will provide my example. But I am interested in the broad guidelines. I save in my postgres DB legislation (law texts) composed of a single header (H), paragraphs (P), sub-...
0
votes
1answer
63 views

Data backend for a 'Nearby' places application

Context I am building an application for searching places 'Nearby' akin to Yelp/Google. Objective for this question Recommend a strategy to do paginated searches based on location and ask for ...
0
votes
1answer
126 views

Database Design Structure

I am building a WebApp which has a list of Assets. I have a peculiar requirement and I am unsure of the best way to store the data in SQL. Requirement: My Asset Headers currently have a relation to ...
0
votes
1answer
104 views

Bridge/Join Table for a Many to Many Relationship between One Table and Multiple Others

Background I have a relational database an I want to have a table that contains flags. I want multiple tables in my database to have a many to many relationship with the flag table. Options I ...
0
votes
1answer
192 views

Is it good practice to create a static class for database tables calculations?

I need to implement some system (using python pandas dataframe in my case) that parses raw data, then adds on calculated data, and then validates that calculated data (boolean output on some columns) ...
0
votes
0answers
18 views

Updating an entity and many relationships: database vs application layer

We are working on a transformation project It has records in table entityA and entityB, which has a relation entityAB. We need to delete records in table entityA and recreate them with some other ...
0
votes
1answer
61 views

How to store data which can be a foreign key or free text

I am not sure how to explain my problem, but basically I have an input field which lets you enter member's name or select one from a dropdown list. If the user has selected the person from the list ...
0
votes
1answer
76 views

Deciding which microservice should write to the database

Let's assume the following architecture where EventCreatorApp creates an Event which is logged to DB and sent to WorkerApp for processing WorkerApp processes the event, handles failures, etc. ...
2
votes
1answer
93 views

Can I reduce locking for this scenario using Hibernate and H2

My Java application uses Hibernate and H2. It process music files within folders. Typically for a folder it would read the files in the folder and store each file as a Song class in database, then ...
0
votes
1answer
41 views

Temporary users to restrict db access of query?

I am new to database managment, and I am wondering if it is a good idea to create temporary users to restrict access to only a single table. Like having an user named _logs can only (read?) and ...
1
vote
1answer
100 views

Multiple applications with single database for IoT

I'm using a raspberry pi 2B with raspbian lite. I have made a program that communicates with a LoRa module (RFM95, sx1276 chip) and collects data from a sensor. The data it collects, are stored in a ...
2
votes
1answer
37 views

Setup pipeline to analyze data stored in web app DB

Background: So there is a (Ruby) web app with a production Postgres DB (hosted in the cloud) I would like to run some machine learning algorithms in a Python setting on the production data and (...
3
votes
2answers
241 views

How to test database dependent functionalities?

I have written this function which checks if a user already exists in the application: async function ValidateUserExists(username, email){ if(!username || !email) throw new Error('Invalid number ...

1 2 3 4 5 36