Questions tagged [database]

This tag is for general database questions. If you question is specific to SQL or NoSQL, use the corresponding tags instead.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
43 views

Designing APIs that don't expose database primary keys

People say "don't expose primary keys from the database in your API" because its a major security leak, so I'm trying to come up with a way for: RESTful HTTP requests to reference server-...
user avatar
-1 votes
1 answer
21 views

Picking data store for Location Tracking Based Application

I have one design problem for one of my projects based on location tracking. The scenario is I have multiple devices(for some business) all over the city and have GPS installed inside, the device is ...
user avatar
0 votes
1 answer
34 views

Mixing static referred and dynamic data in database

I’m making an inventory for a game using nosql (mongo). I made some ItemType records (contains common data like icon, name, weight) and some ItemInstance records (representing specific items, ...
user avatar
0 votes
1 answer
52 views

Database design - how to model database tables for similar objects - booleans / enums / inheritance / archived vs active

I only recently started to work with backends. My database is supposed to store information about maven-like artifacts. There are going to be applications and configuration artifacts. Now those ...
user avatar
  • 107
0 votes
0 answers
41 views

Database versioning when testing layered design

Let's say there are two layers of an application: Business logic layer Data access layer Assume integration tests need to be written also for the Business logic layer. Alas tests that run against ...
user avatar
0 votes
1 answer
52 views

Form builder app database design adding versioning

i'm making a custom form builder app similar to google forms or wufoo and i've completed the basic database design. However there is one thing i can't figure out. I'd like to keep track of version of ...
user avatar
  • 109
0 votes
1 answer
134 views

How to create financial reports with detailed information

I'm working on an application where it has financial control for the incomes and expenses of the company. These informations can be controled by accounts (bank accounts - can be 1 or more), categories ...
user avatar
0 votes
3 answers
164 views

Writing a growing simulation that scales

Or ways to get better run time than a nested loop? How would you design this? I hope this is the place to ask semi-vague software engineering questions. My simple simulation is like this: There is a ...
user avatar
  • 119
0 votes
1 answer
121 views

Database design for a reccomendation system

I am working on a project which will have a recommendation system and I am in the database design phase, I have a good foundation in programming but databases is a new topic for me, I have designed a ...
user avatar
  • 9
-1 votes
2 answers
141 views

Should I store non-editable items lists in a database?

Relational database is mostly used for storing editable user data. But how about storing non-editable lists of information? Possible examples are: GUI-related: main menu items, selectbox options, ...
user avatar
1 vote
2 answers
135 views

Making data available for multiple microservices

We have a design challenge here for a project we are working on and I wonder if folks from the community can provide some guidance: Our product is built in a microservices structure. So we have ...
user avatar
  • 111
0 votes
2 answers
77 views

Is it a good idea to use database sequential id as a key in external key-value storage?

I use Postgres database to store user information. Some of that information is sensitive so it was decided to store that information in Vault. The user table in Postgres has id's generated by a ...
user avatar
  • 145
1 vote
0 answers
33 views

Backend Design: System requires both ElasticSearch and MySQL. In which scenario we fetch data from MySQL or ElasticSearch?

So, one of the backend system, having 5 resource types (tables) and some foreign keys as well, requires full text support for two of the tables. Also, for some use-cases, we require inner-joins for ...
user avatar
0 votes
3 answers
121 views

Can you suggest ways to assure client data confidentiality in a SaaS product managed by yourself?

I am designing a multitenant SaaS platform where the database will hold information for different clients. The application consists of an Angular frontend and a REST API Backend. Some of the clients ...
user avatar
1 vote
1 answer
83 views

Contextual Implementation Overhead vs Redundancy

I have an application that has use cases relative to where it is used. A local client and server, where all the server data is in a single location and the client queries that server knowing it is ...
user avatar
16 votes
5 answers
4k views

Why would an "articles" table (with no relationship to any other table) have a primary key?

Let's say that I have an articles table that have two columns: title and content, and let's say that this articles table doesn't have any relationship with any other table. Why would this articles ...
user avatar
0 votes
1 answer
59 views

Synchronous replication of databases

Say we have a software system with a web server serving writes to a Master DB. This Master DB is then replicated to several read-only DB replicas. From my understanding, the process of asynchronous ...
user avatar
-1 votes
2 answers
85 views

Difficulty understanding how denormalization results in more storage necessary

I am having difficulty understanding how denormalization results in more storage necessary. For example, let's say that in a normalized relation, there's Table 1 and Table 2. In order to join Table 1 ...
user avatar
  • 153
0 votes
1 answer
71 views

What considerations need to be made for choosing DBs in the OLAP portion of a scaling software system?

I'm trying to learn about the different types of database architectures and how their differences influence which one or combination of should be used in software system. I understand that OLTPs ...
user avatar
0 votes
1 answer
83 views

Benefits of Microservices processes split per kind? of process?

At our job we are having a heated discussion with the DevOps team on how to architect and modularize our microservices. We come from a 2-coupled-monoliths approach, so we are trying to avoid common ...
user avatar
  • 11
-1 votes
1 answer
51 views

Navigating a many-to-many relationship and determining price from a bulk API Call

I think it will help to give a little context, not only about the DB architecture but the actual product. I am building a business where we will be selling pallets of liquidated merchandise. Think a ...
user avatar
0 votes
0 answers
67 views

Prevent users from creating "global" database connections for a forking general purpose server?

I've created a general purpose scalable server in PHP. It's scalable in that the main process merely listens for incoming connections and immediately distributes them to forked child processes which ...
user avatar
-3 votes
2 answers
124 views

How Can I Create An Immutable Database That Can Be Truncated? [closed]

I have a requirement from a bank where they need to store data in a tamper proof way but at the same time they need to purge it after 7 years. The current process is they receive a letter and file it. ...
user avatar
0 votes
1 answer
123 views

How are Object Oriented Databases represented?

I have read many articles online but each one of them fail to give even a basic example on how an Object Oriented Database is stored and represented. For example each of the articles say that they are ...
user avatar
-3 votes
1 answer
110 views

How much access should a developer have to a Staging database [closed]

We want our developers to be able to diagnose production issues in staging. For the sake of this, we are planning to make an exact replica of production into staging. We are thinking however, if we ...
user avatar
  • 9
2 votes
1 answer
126 views

Should integration tests of a repository pattern use low-level ORM calls

Problem summary: In an application with wrapper methods over SQLAlchemy add() and query() methods, can integration tests that use the add() method wrapper use the query() method wrapper to validate ...
user avatar
  • 131
-2 votes
1 answer
122 views

Database index - Basic understanding

Trying to clarify my knowledge on databases and indexes. I'd like to know how exactly it works. So I have a few questions: When indexing a table over column or set of columns, a new table is created ...
user avatar
  • 113
9 votes
5 answers
755 views

Today I wrote "tests" for the testing code. Was it the right thing? Is it a smell?

My application has big integration with the database. Classes that use the database are very crucial for the system, so I write small unit tests for classes that I call Repository. The reason behind ...
user avatar
  • 695
1 vote
1 answer
180 views

How to store a huge volume of time-series datapoints in an efficient way?

We have an application producing 5k-10k datapoints per second. Each datapoint has more than one metric, alongside its time of creation. We are looking for an efficient, scalable way to store this huge ...
user avatar
  • 127
1 vote
2 answers
2k views

To which Clean Architecture layer should repositories implementations belong?

It's very common to see this use of repository in projects using clean architecture: interface Hero { } interface HeroRepository { findById(id: number): Hero; } class FetchHeroUseCase { ...
user avatar
1 vote
1 answer
89 views

Database Design, Associative Entities

I'm learning databases and I'm designing an ERD based on simple banking processes. I'm not sure if my design of the entity relationship diagram is correct. Thought Process A customer can open 1 : N ...
user avatar
  • 27
0 votes
1 answer
112 views

How sharding works when you outgrow the current shard architecture?

Looking at sharding techniques, you basically have hash-based or range-based versions. Hash based is more random, range-based is more heuristic based. Say you initially have 2 shards (separate ...
user avatar
  • 2,408
2 votes
3 answers
290 views

Should you use interfaces or classes in database entities?

Let's say I'm making a library database with books and authors, and I want to model the domain with interfaces, should I also specify the relations in the interfaces? For example, if I have the ...
user avatar
-2 votes
1 answer
56 views

Should cached data be kept separate from primary tables?

I have a table in an SQL database for storing information about some business object that looks something like this (details changed so as not to give away what company I work for): CREATE TABLE ...
user avatar
0 votes
2 answers
161 views

Share data between users without a database, php [closed]

How would I create for example a live chat, where you "post" a piece of text and it is displayed to other users in a "chat room", without the need of permanently storing it. My ...
user avatar
  • 23
-3 votes
1 answer
47 views

Ways & Methods switch between MongoDB Database after selecting a country. (MERNG)

I have a concept in which a user selects a country, which will either filter or go to that database; I was thinking of alternatives or options that would help in the process. These are my first plans ...
user avatar
0 votes
2 answers
210 views

When updating a model on a RESTful API, should there be an update endpoint per field? or one endpoint for the model?

For a RESTful API, consider a model schema as follows: MyCoolObject { field_a field_b field_c } Is it better to create one update endpoint to update one or many fields on the model? Or create ...
user avatar
-1 votes
2 answers
93 views

Database responsibility segregation

I have the following architecture: Database (Postgres), Batch(java app - long running operations that delete, update, insert lots of data in the database), Rest API(java app - provides rest services ...
user avatar
2 votes
4 answers
162 views

How to design around webapp Out of Memory crashing on export from SQL DB to Excel?

I have a monolith CRUD application with a feature - possibility to export some DB entires to an Excel file. The export is done within a background task. Often, however, when user presses the button ...
user avatar
  • 75
0 votes
4 answers
368 views

HOWTO: Full-text search over an encrypted database?

I want to adhere to the best practices and obviously encrypt the data sent to me by user. And I also want to provide the full-text search feature to the user (user can only search their own data, if ...
user avatar
  • 75
-3 votes
1 answer
122 views

Most efficient way to get average data for thousands of records from the database [closed]

I have a database that keeps a temperature readouts, which are being added - let's say - every minute: +----+---------------------+-------------+-----------+ | id | timestamp | temperature | ...
user avatar
  • 149
0 votes
0 answers
21 views

DRF API - using Multitable Inheritance vs "profiles"

I'm looking forward to your opinion on how to do this. This is a backend that will be served as a Django Rest Framework API. There will be multiple types of properties. Properties have approximately ...
user avatar
  • 119
1 vote
1 answer
38 views

Alternatives to ActiveX Data Objects for Serverless Implementations?

I have a few different web front ends that I use ActiveX Data Objects and ADODB connections to query an Access Database on a network drive. Usually there's some JavaScript logic that helps build the ...
user avatar
0 votes
1 answer
42 views

Design approach where to persist autocompletion suggestions

I am currently planning a movie recommendation application. Within this context, the user can fill out a form including different information (e.g. title, description, etc.). The new entry gets then ...
user avatar
0 votes
1 answer
153 views

Low Latency for multi-million record database in a web app [closed]

As a forewarning, this is for a school project with other school teammates, so my question is just for the sake of learning best practices. We are creating a recommendation system web app which ...
user avatar
0 votes
1 answer
54 views

Data Class: spread boilerplate for each different database over multiple class copies, or add complexity to original class?

I'm creating a prototype mobile app that will store data locally in an sqLite database and also on the cloud (CloudKit for now). The cloud storage uses a certain way of storing the data, and sqlite ...
user avatar
  • 1,476
1 vote
1 answer
76 views

An effective database migration pattern for many instances of the same app over a long period of time

How do you handle migrations to your database after a long period of time? I'm using fluent migrator, and after some months or years, the number of migrations could be (and has been in previous ...
user avatar
1 vote
2 answers
59 views

Compare diff between similar objects stored in two different databases and delete the missing records from the later

We have a legacy system (app 1) that has millions of records of an object Project. It has ProjectId and OrganizationId columns along with other properties. A recently added service (app 2) gets the ...
user avatar
  • 19
0 votes
0 answers
49 views

How to properly ingest mutable data into Snowflake?

Overview My app produces 100-1000 files of about 1 MB, per minute. And each file represents one single record. The records are json elements with dynamic structure. They are savegames from a videogame....
user avatar
  • 101
-1 votes
2 answers
113 views

Should I make all my business logic depend on the data that comes from the cache?

One of my web services hits a few hundred thousand visits per day where most of the operations are CRUD and we are thinking to get a cache sitting in front of our SQL database. I have had implemented ...
user avatar
  • 107

1
2 3 4 5
41