Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

Filter by
Sorted by
Tagged with
0
votes
1answer
81 views

Can a wide table be the correct choice in design?

I am relatively new to database architecture and I have been tasked with designing a database that will store 250 values per entry by my group. This is pushing the upper limit for fields in an Access ...
1
vote
3answers
103 views

How do I query something when I don't know how its structured?

How does one write & run queries against a database/warehouse if they have no knowledge of how the data is structured? By "structure" I mean table names, fields, etc. Unless I have table names, ...
1
vote
1answer
44 views

Offline First: How Do I Fetch Data?

I'm new to this community, I'm glad I found this and so I can ask questions like this one (as per https://meta.stackexchange.com/questions/68384/whats-the-difference-between-stack-overflow-and-...
0
votes
1answer
79 views

Recommended Long Term Data Storage Format

I have an application (written in Go) with a lot of GPS location information. Each record consists of a latitude and longitude plus somewhere around 50 fields of various pieces of meta data, some of ...
0
votes
1answer
59 views

How to manage breaking updates in an API & database

So I have been working on my mobile app for a while and it is live and in production. Because we released the app prematurely and are continuously improving current features and making them more ...
1
vote
1answer
139 views

Triangle relationship in database

I have a problem figuring out the relations between some of my database tables. The scenario for this buisiness case is a b2b buissnes. I have 3 tables: users companies distributors. A Company ...
0
votes
1answer
51 views

Correct place to store semi dynamic data

What I mean by semi dynamic data are data that are expected to be changed only with product versions. Now imagine this scenario, In version 1 of the product, I have item status : SUCCESS , FAIL In ...
1
vote
2answers
43 views

Architecture of “tags” for contacts in a marketing system and hard-coded IDs so functions can refer to them

I have these tables: contacts, tags, contact_tags. (contact_tags has columns contact_id and tag_id.) Administrators can manually create new tags via an internal website. Administrators can also ...
-3
votes
2answers
108 views

How to structure a database where a record can be a child of a parent or a grand parent record

I have these concepts for a database application: Session, Subsession and File A session can have multiple files A session can have multiple subsessions A subsession can have multiple files A session ...
-2
votes
1answer
93 views

Store line numbers of CSV in the database

In our environment, we often have to import CSV files into a database, where each line represents a record. Though there is no actual business use case for storing the original line numbers inside the ...
0
votes
2answers
80 views

What's the best way to save/track change requests on multiple objects in a database?

I have a schema containing contracts, sub-contracts, services offered by each of those and sub-services. I also have a application that allows to extend the duration of the contract and to modify ...
2
votes
1answer
29 views

Do I need to put foreign keys into a metadata repository?

As I was creating a metadata repository, I came across a doubt. As I'm new to databases, I was wondering if I should add the foreign keys data into the tables that have them (for context, imagine a ...
0
votes
0answers
39 views

Handling a large local database in a chrome extension

I'm creating a chrome extension which needs a large local database. After researching my options, I figured my best bet is probably to use the chrome.storage.local API and give my extension the ...
0
votes
0answers
106 views

Optimizing web dashboard with editable elements for performance by introducing ElasticSearch

To improve performance, we are optimizing our complex tabular web-based dashboard that loads a lot of data (many columns and rows). Many cells in the table are editable so the data is mutable. Our ...
1
vote
1answer
35 views

Data modeling user-object permissions, between claims and metadata

I've been thinking about how to map user claims to permissions in a way that scales from a read and write perspective, probably using a stand-alone service. What we have today doesn't scale for ...
0
votes
0answers
55 views

What is the correct way of setting and then differentiating between paid and free user in Firebase / Firestore?

I am developing a mobile application that will have both free and paid versions. The way I see how I could differentiate between what is a paid and free user is to set a property isPaid on their ...
0
votes
2answers
85 views

Should you clean up your DB after integration tests?

Is it good practice to clean up your DB after running integration tests (which tests APIs that put data into that DB)? I want to have a cleanup method after my integration tests run which will access ...
1
vote
1answer
84 views

Saving flags in the database without relationships to other tables

I have an admin panel which can set a lot of options which will affect some other services drastically. For example I got one that if I create a different category I can just insert it in the Category ...
3
votes
3answers
93 views

Delete user data or replace it with empty resource?

Say, we have a public facing REST API which lets user manipulate data using POST, GET and DELETE. The API is exposed via Service A which at the backend calls Service B where most of the business logic ...
0
votes
0answers
21 views

Sequence Diagram Assistance with external database

I have 5 Java/android classes which are in sequential order. The user inputs data into the text box and clicks next. Two things happen, 1 the data the user inputs is passed to the next activity and it ...
0
votes
1answer
75 views

Web application and persistence choices

Right now we have a highly analytical system which collects tons of various metrics. We need to display this data in a web ui for users to consume. Since we are in early stages of development we are ...
0
votes
1answer
37 views

Event planning application database schema

I am planning to develop an Event Planning Application. Currently I'm in the stage of planning the app structure, for example database schemas. Basically there will be two kinds of users: The event ...
1
vote
0answers
37 views

Data pipeline architecture: airflow triggered by message broker

Let us say we have: a web app with a Postgres DB that produces data over time, another DB optimized for analytics that we would like to populate over time. My goal is to build and monitor an ETL ...
0
votes
0answers
23 views

Multitenant setup with master account

I would like to make a SaaS which would be a multitenant system. Development is done in Laravel, which is less important. What I would like to know is how would you handle having a master admin ...
0
votes
1answer
96 views

Is my database schema going to work? Tracking git issues

My boss wants an automated weekly report on how many git issues have been moved to and from various "lanes" (pipelines) on a virtual kanban board. I wrote a quick python script to get that data using ...
0
votes
0answers
52 views

Effective database design for time card tracking

I'm building a timecard system for a company of ~50 employees. My problem is designing an efficient way of storing this data. Currently, their timecards are done on a table like the one shown below. ...
-1
votes
1answer
53 views

What is the best design for this use case regarding bidirectional relationship? [closed]

For this question let's consider a system in which the users can match with each other (let's say for simplicity exactly like the dating apps). User1 matches User2 (and the other way around) only in ...
1
vote
2answers
123 views

Architecture/algorithm for unusual recommendation system

There are thousands (or tens of thousands) of possible movie titles. New user enters my website and selects hundreds of titles that he likes. The only single goal of my website is to output the list ...
0
votes
1answer
29 views

Update items based on a read operations on same items is a bad practice?

DynamoDB is the database solution, I'm using to store my items. It's a NoSQL/Document database and I got a real problem on real-time updates of items. First, the saved format is like { expiredTime:...
0
votes
2answers
160 views

Is it a software engineer's responsibility to structure data as opposed to a DBA?

As a backend engineer at a small company, we're running into a question of whether we need a DBA or another backend engineer (SWE2). The reason why this is a question stems from a difficulty in ...
0
votes
1answer
34 views

Strategies for Rotating MYSQL Encryption Keys

I have a web app that connects to a MYSQL database which has some encrypted data in it. The encrypted data is encrypted using the the mysql AES_ENCRYPT function and a key. Now, this key has been ...
-1
votes
1answer
105 views

I have a data set of over a million addresses and I want to display the closest N locations to a given address or current location

I am a student working on a personal project which is essentially a location finder that will be on a website. I have a data set of over a million addresses and I want to display the closest N ...
-1
votes
2answers
116 views

When self-referencing in a table, do you prefer `parent_id = null` or `parent_id = id` for a row referencing to itself?

For the discussion, I will provide my example. But I am interested in the broad guidelines. I save in my postgres DB legislation (law texts) composed of a single header (H), paragraphs (P), sub-...
0
votes
1answer
53 views

Data backend for a 'Nearby' places application

Context I am building an application for searching places 'Nearby' akin to Yelp/Google. Objective for this question Recommend a strategy to do paginated searches based on location and ask for ...
0
votes
1answer
122 views

Database Design Structure

I am building a WebApp which has a list of Assets. I have a peculiar requirement and I am unsure of the best way to store the data in SQL. Requirement: My Asset Headers currently have a relation to ...
0
votes
1answer
47 views

Bridge/Join Table for a Many to Many Relationship between One Table and Multiple Others

Background I have a relational database an I want to have a table that contains flags. I want multiple tables in my database to have a many to many relationship with the flag table. Options I ...
0
votes
1answer
178 views

Is it good practice to create a static class for database tables calculations?

I need to implement some system (using python pandas dataframe in my case) that parses raw data, then adds on calculated data, and then validates that calculated data (boolean output on some columns) ...
0
votes
0answers
18 views

Updating an entity and many relationships: database vs application layer

We are working on a transformation project It has records in table entityA and entityB, which has a relation entityAB. We need to delete records in table entityA and recreate them with some other ...
0
votes
1answer
56 views

How to store data which can be a foreign key or free text

I am not sure how to explain my problem, but basically I have an input field which lets you enter member's name or select one from a dropdown list. If the user has selected the person from the list ...
0
votes
1answer
75 views

Deciding which microservice should write to the database

Let's assume the following architecture where EventCreatorApp creates an Event which is logged to DB and sent to WorkerApp for processing WorkerApp processes the event, handles failures, etc. ...
2
votes
1answer
74 views

Can I reduce locking for this scenario using Hibernate and H2

My Java application uses Hibernate and H2. It process music files within folders. Typically for a folder it would read the files in the folder and store each file as a Song class in database, then ...
0
votes
1answer
40 views

Temporary users to restrict db access of query?

I am new to database managment, and I am wondering if it is a good idea to create temporary users to restrict access to only a single table. Like having an user named _logs can only (read?) and ...
1
vote
1answer
97 views

Multiple applications with single database for IoT

I'm using a raspberry pi 2B with raspbian lite. I have made a program that communicates with a LoRa module (RFM95, sx1276 chip) and collects data from a sensor. The data it collects, are stored in a ...
2
votes
1answer
34 views

Setup pipeline to analyze data stored in web app DB

Background: So there is a (Ruby) web app with a production Postgres DB (hosted in the cloud) I would like to run some machine learning algorithms in a Python setting on the production data and (...
4
votes
2answers
225 views

How to test database dependent functionalities?

I have written this function which checks if a user already exists in the application: async function ValidateUserExists(username, email){ if(!username || !email) throw new Error('Invalid number ...
0
votes
1answer
90 views

ER Diagram in Barker Notation

I engineered an ERD diagram using Barker's notation. The problem was to make a database for system of water pipes where one node is connected to another using a section (pipe), each node can have ...
4
votes
2answers
116 views

Database structure for word co-occurrence frequencies in a large corpus

I would like to store the frequencies with which words co-occur with each other over a variety of contexts in a large (> 1 billion tokens) text corpus. I need to store the word pair, the type of co-...
2
votes
1answer
71 views

Allow client side direct access to the database [duplicate]

I am asking advice from the SQL graybeards. Because my following proposition goes against everything I have practiced. In this instance Let's assume we are developing a phone app. A social network ...
0
votes
0answers
35 views

ACL entity and microservices database relations

I would like to build a microservices architecture to handle the following scenario: I have a building. The building has some kinds of devices. And the devices can be used by the users. In the system, ...
1
vote
0answers
116 views

How to store relational data belonging to different client customers?

My users will be businesses with a small number of accounts each: e.g. Business #1 with 3 users, Business #2 with 5 users, etc. I am trying to determine the best way to organise the relation (on ...