Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

-3
votes
0answers
32 views

Transfering data from staging db to production db [on hold]

I'm working on this application where I need to update my database with data of considerable size. But, the data has to go through some quality checks before it can be pushed into the production ...
2
votes
1answer
56 views

Can I reduce locking for this scenario using Hibernate and H2

My Java application uses Hibernate and H2. It process music files within folders. Typically for a folder it would read the files in the folder and store each file as a Song class in database, then ...
5
votes
5answers
3k views

When keeping product price change history, what are the pros&cons of keeping track of price change events only?

When tracking something like pricing history, in my relational database, what is a recommended way to store pricing changes? For example I can: keep price change events only (aka a single date ...
0
votes
1answer
35 views

Temporary users to restrict db access of query?

I am new to database managment, and I am wondering if it is a good idea to create temporary users to restrict access to only a single table. Like having an user named _logs can only (read?) and ...
33
votes
3answers
31k views

Is there any reason to use varchar over text columns in a database?

Is varchar just a remnant from before text came around, or are there use cases where you would want to use a varchar? (Or char for that matter..) (I use Postgres and MySQL (MyISAM) daily, so those ...
129
votes
9answers
96k views

When would someone use MongoDB (or similar) over a Relational DBMS?

I'm a bit confused about the whole NoSQL thing and such. When would you choose to use something like MongoDB over something like Oracle or MySQL? I don't really understand the "difference" as far as ...
3
votes
3answers
373 views

Aggregation of data from two Microservices

I have two Microservices A and B. B Microservice has a large set of an entity called User. A Microservice stores the User entity in its own DB if User is configured by an agent. There is no flag ...
11
votes
5answers
7k views

When should we use MongoDB?

MongoDB is a NoSQL database which I've found quite easy to use. Recently I had to develop a simple application which needed to collect some data using HTTP requests and store some results after ...
4
votes
2answers
107 views

Database structure for word co-occurrence frequencies in a large corpus

I would like to store the frequencies with which words co-occur with each other over a variety of contexts in a large (> 1 billion tokens) text corpus. I need to store the word pair, the type of co-...
1
vote
1answer
62 views

Multiple applications with single database for IoT

I'm using a raspberry pi 2B with raspbian lite. I have made a program that communicates with a LoRa module (RFM95, sx1276 chip) and collects data from a sensor. The data it collects, are stored in a ...
1
vote
1answer
27 views

Setup pipeline to analyze data stored in web app DB

Background: So there is a (Ruby) web app with a production Postgres DB (hosted in the cloud) I would like to run some machine learning algorithms in a Python setting on the production data and (...
6
votes
4answers
16k views

ERD: “many” vs “zero or many”/“one or many” crowfoot notation?

Background I saw this figure describing the different crowfoot notations used in ERD: I'm not able to find the difference between the "many" notation and the "zero or many. However I was able to find ...
-3
votes
0answers
41 views

How to Version Control SQL Scripts Needed for Deployment

A small group of developers and I are working on a side project, and I am currently setting up automated update deployment (for a reasonable definition of automated). Currently, we begin deployment by ...
4
votes
2answers
205 views

How to test database dependent functionalities?

I have written this function which checks if a user already exists in the application: async function ValidateUserExists(username, email){ if(!username || !email) throw new Error('Invalid number ...
2
votes
4answers
166 views

Database agnostic DAO (NoSQL + SQL)

Background While writing a new component, I m in middle of making a decision of SQL/NOSQL database (Mongo vs Mysql) for my storage layer. As of today, mysql seems to be a perfect fit for my use-case (...
0
votes
1answer
79 views

ER Diagram in Barker Notation

I engineered an ERD diagram using Barker's notation. The problem was to make a database for system of water pipes where one node is connected to another using a section (pipe), each node can have ...
2
votes
1answer
66 views

Allow client side direct access to the database [duplicate]

I am asking advice from the SQL graybeards. Because my following proposition goes against everything I have practiced. In this instance Let's assume we are developing a phone app. A social network ...
1
vote
0answers
106 views

How to store relational data belonging to different client customers?

My users will be businesses with a small number of accounts each: e.g. Business #1 with 3 users, Business #2 with 5 users, etc. I am trying to determine the best way to organise the relation (on ...
0
votes
0answers
22 views

ACL entity and microservices database relations

I would like to build a microservices architecture to handle the following scenario: I have a building. The building has some kinds of devices. And the devices can be used by the users. In the system, ...
2
votes
2answers
281 views

Should arbitrary numbers be stored as strings in a database?

I'm using Google as a login system for my latest project. So, my users table has a column named googleid, which stores Google's unique user id, which is a number, with a few dozen digits. For example, ...
1
vote
8answers
652 views

Is there a way to prevent developers leaking data while they are developing?

I am working with a startup company for a while now. They have been affected by some reckless developers in the past. One area they want a real and immediate improvement is when they assign a task to ...
8
votes
1answer
2k views

Making API calls with celery

I'm designing a system for a client where the requirements are: they upload a JSON file (one object/line) make a call to an API with the JSON object as the payload record the state (success/failure) ...
2
votes
1answer
146 views

Unit conversion of data

My team is developing a web application, where we have in the requirements to display the data in multiple unit which is set by the end user. Where should we write down the unit conversion logic, on ...
-3
votes
5answers
148 views

One Single Table or Two Smaller Tables

We currently have a Users table in the database, this has quite a few columns in it. Around 50% of the columns are barely used in the system (only on one or two pages). We've been discussing amongst ...
1
vote
1answer
81 views

Approach to handle network breakdowns in application dealing with database?

I have an application that writes data to database. In case of network breakdowns, the connection to database gets lost & my db queries freezes the whole application. So, in order to make it ...
1
vote
1answer
77 views

About databases in docker containers

I'm trying to understand containerized apps and databases and I'm trying to understand the microservice architecture using kubernetes. One thing that I couldn't get my mind convinced is the database ...
126
votes
17answers
12k views

How do you store “fuzzy dates” into a database?

This is a problem I've run into a few times. Imagine you have a record that you want to store into a database table. This table has a DateTime column called "date_created". This one particular record ...
18
votes
3answers
6k views

How to store ordered information in a Relational Database

I am trying to understand how to properly store ordered information in a relational database. An example: Say I have a Playlist, consisting of Songs. Inside my Relational Database, I have a table of ...
0
votes
1answer
40 views

How do database storage engines handle new columns?

I really don't know which StackExchange site this belongs on. It didn't seem to match the "on-topic" list of DBA.SE or SO.SE ... so please let me know if I should move it. I also couldn't find a ...
0
votes
1answer
145 views

When is the point where a JSON text file storage is not enough?

I guess it was Robert C. Martin who campaigned against using SQL in one of his apps, where he was just fine without DB and SQL initially. Later, thanks to the abstraction a DB implementation could be ...
0
votes
0answers
27 views

Object Data must be built on SQL side or in Code Side?

I am thinking of a very flexible application that will build a single entity object from a lot of tables joins. example for an object "Car" with a code something like this SELECT e.id, a1.attr_value ...
-1
votes
2answers
194 views

Anonymize Public Health Records Database

I am working on a project to anonymize PHI in our health records database when databases are copied to our testing environments. Currently, the database is copied and fields in the tables (first name, ...
0
votes
4answers
117 views

Could you please recommend an approach on converting non-unique field to a unique field?

On our prod db we have two tables: companies invoices Relationship: A company has many invoices An invoice belong to a company The invoices table contains duplicate invoice_number for each company. ...
2
votes
1answer
172 views

Designing database job monitoring

I have several Oracle database where my in-house applications are running. Those applications use both dba_jobs and dba_scheduler_jobs. I want to write monitoring function: check_my_jobs which will ...
0
votes
1answer
62 views

Custom service layer logic vs Custom DB query

I am trying to confirm best practice around architecture I am working with. Frontend <-> API gateway <-> Service layer (Rest API) <-> DB I have an incoming request which is asking for huge ...
2
votes
1answer
116 views

Who (and what) should have direct access to a database, and why?

I am developing a chat application as my the first project. I have never had experience of establishing a connection to the remote database server before so I don't really understand who should have ...
0
votes
1answer
204 views

Appropriate design / technologies to handle dynamic string formatting?

recently I was tasked with implementing a way of adding support for versioning of hardware packet specifications to one of our libraries. First a bit of information about the project. We have a ...
1
vote
1answer
72 views

Does a microservice's individual database need to have complete data?

Say you have a Users and Orders service and they have their own database. Since microservices need to stand on their own even if another service is unavailable, does the Orders DB need to have all ...
1
vote
2answers
1k views

Should I use NoSQL (MongoDB) or SQL (postgresql) database for my application?

I am planning to build a live audience interaction webapp similar to Slido as practice/fun project. User groups: admins and anonymous users Admins: login create event (define event period and ...
9
votes
3answers
4k views

Where to put business logic if using Firebase?

I'm about to start developing a single page web application that is very simplified a multi user documentation system. The front end will probably use Angular2. The project has a short deadline, so I'...
1
vote
1answer
46 views

Database structure for an app used to capture data in warehouses

I am in the process of creating an android app for our clients that would replace the paper-trail in their facilities with electronic inspections. Although I have coding experience, I have little ...
1
vote
1answer
147 views

Best way to model pricelist per customer for products in webshop

I am making a b2b-webshop system that will have multiple customers. Each customer will have a separate active pricelist which contains the products that they can buy and the prices that applies for ...
0
votes
1answer
68 views

Database schema for products and their packages

The product is water purifier, and we are selling packages included support for changing filters. Serial is a unique number identifying a single produced item and it's only for water purifiers not ...
0
votes
2answers
122 views

Architecture of data logging and storage system

I've gotten myself a bit more roped into coming up with the design and architecture for a new system due to some departures and looking for some insights. The system has a bunch of different high ...
2
votes
1answer
212 views

How to handle database connection password?

I have a question about using databases in my application. I have the User table which stores password for my users. I know how to handle user authentication, but how to handle the password for the ...
0
votes
0answers
26 views

How to store message content in a webservice

I am building a service responsible for sending email/sms communications. The service receives a request that contains information necessary to build an email/sms template and the recipient ...
0
votes
3answers
355 views

Should I use ISO country code for primary key, or integer ID?

I have or am going to have a database with countries and their emissions. Countries in the API have a 3-letter ISO Code. Should I use this for identifying the countries in database, or just a plain ...
94
votes
3answers
48k views

How is localStorage different from indexedDB?

localStorage and indexedDB are used for offline storage of data in HTML5. What are their key differences and which one is preferable in what situations?
1
vote
2answers
139 views

What is the best model to catch DB side error?

We have a web application that all update, insert and DB operation procedure handle via SQL Server DB 2017. What is the best way to handle and get error (in Web-app or SQL server error handler?). for ...
3
votes
3answers
1k views

Is it okay to use a non-ID column for foreign key reference?

I have my tables as below Role(Role ID, Name, Other_Columns) Command(Command ID, Name, Other_Columns) I have association table RoleCommand. Is it okay to have RoleCommand(RoleName, CommandName) ...