Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

6
votes
0answers
81 views

When connecting to a server using the DRDA protocol, is it true that the first Client-To-Server command MUST be EXCSAT chained with ACCSEC?

When connecting to a server using the DRDA protocol, is it true that the first Client-To-Server command MUST be EXCSAT chained with ACCSEC? I found 2 different answers when I googled it. If you look ...
5
votes
0answers
240 views

Concurrent fault-safe data structure

I am building an application that aims to process ~10M data items per second. Each item is exactly 42 bytes small (including a sorting key) which means the total data rate will not be big 420 MB/s. ...
3
votes
0answers
216 views

Architecture: mobile apps for different clients with their own dbs different servers?

My general feeling is that this is a question which must have been answered but I am struggling to find resources for it. I know little about mobile app dev but my knowledge of web services is pretty ...
3
votes
0answers
495 views

Is Event sourcing a good fit for database sync (master-multiple-slaves)?

I'm looking for a good way to maintain in sync local sqlite databases that could be offline to a master one, for a invoice app. Then I read about EventSourcing and Log structured storage and wonder ...
3
votes
0answers
641 views

Database version management on mobile apps

I am about to start building a mobile (iOS/Android) app that uses a local database (probably SQLite). Since this is my first "real" mobile app, I am wondering what the standard approaches are to ...
2
votes
0answers
23 views

Designing provisioning strategies

I'm trying to redesign a provisioning application that administrates other application through their REST API. The application is quite simple, we have groups, we have users and roles and we have a ...
2
votes
0answers
48 views

Data durability guarantees in Kafka

Is it wise to use kafka as the 'source of truth' for mission critical data? The setup is: kafka is the underlying source-of-truth for the data. -querying is done on caches (I.e. Redis, ktables) ...
2
votes
0answers
70 views

Chronicle Queue: how to make a stack and how to limit disk usage

My server and several clients are different JVMs running on the same machine, they need to share persisted time series data: server reads a time series from somewhere, writes to a persisted embedded ...
2
votes
0answers
275 views

File upload sync for offline desktop application. (Electron)

We are making an offline desktop application in Electron that needs to sync when the user goes online. Sqlite is used as the database when offline and it gets synced to postgres on the server when ...
2
votes
0answers
82 views

Idea on replacing DAO/Domain obj pattern

I know someone has probably already thought of this, but here is a way to replace DAOs and Domain objects: For example public class Bike { private int id; public Bike(String model) { //...
2
votes
0answers
351 views

Class table inheritance using Slick: always an anti-pattern?

My application's model includes a number of entities that share many common properties, and are operated on by many of the same classes. If I was using an ORM, I'd define an abstract class with the ...
2
votes
0answers
551 views

How do I implement Socket.io Leaderboard using AWS Dynamo DB cost-effectively?

Every time a user submits a score, Socket.IO causes all the users get their leaderboard updated. This causes every single user connected through Socket.IO to read from the Dynamo DB database (every ...
2
votes
0answers
154 views

Are there downsides to writing PDO/mysqli statements into my code directly vs using my own redirect wrapper methods?

TL;DR: To access DB functionality in my codebase I currently use a custom layer throughout the code. The layer needs an upgrade in order to begin accepting prepared statements, which it currently ...
2
votes
0answers
244 views

Logging Database Changes

What is the proper way of storing a changelog, similar to git, for data in a database. Currently, I have it set up so that every line in a table is given a unique string id that is used to determine ...
2
votes
0answers
209 views

Saving tabular, spreadsheet-like data in a relational database?

I'm hoping this question isn't too broad or too prone to opinionated answers, since I could really use some pointers. I'm trying to come up with a way to persist tabular, spreadsheet-like1, data in a ...
2
votes
0answers
360 views

Maximising data processing speed: in-memory or write to db first?

Currently working on a system which allows users to upload data files, which we then "clean" (i.e. process in various ways). Speed of processing and return to the user is an important consideration, ...
2
votes
0answers
88 views

Suggestions for Storing large collection of related words

I am trying to create a simple sentence generator that uses templates and a database of words. It will be a website where essentially a user could click a button to generate a sentence. For example: ...
2
votes
0answers
622 views

how to import large amounts of data without locking sqlite database

Background Information I have a web app that provides the user with the usual CRUD functionality for "widgets" let's say. But I also have give them the option of creating widgets in bulk via ...
2
votes
0answers
471 views

Syncing structured data and files in Android App with an ORM?

I am developing an Android App which I have developed for iOS before. The app requires that structured data and files are synced across android devices. In iOS, I was using Core Data and synced to ...
2
votes
0answers
168 views

What are some important considerations when developing a DBaaS (DataBase as a Service)

I'm interested in creating a Database as a Service. I want users of the service to be able to do a few seemingly not very simple things. Sign up and be given access to a JSON datastore that they can ...
1
vote
0answers
43 views

Strategies for dealing with Stale data and Fetching

I'm working with firestore and have encountered several situations where a document will contain stale data which is a duplicate of the source of truth located elsewhere in the database. For ...
1
vote
0answers
36 views

Single users database for many apps

I'm designing a database that holds the information for users of three diferentes apps that share the access to the data trough an API. For every user i'm storing credentials, profile and roles. What ...
1
vote
0answers
40 views

separate table for one of the type or java enum?

I want to make budget tracking web-app, tracking all savings and expenses with corresponding dates. Later generating reports, summaries etc.. I came up this design Frequency Table - does it have to ...
1
vote
0answers
56 views

How can I offload customer's data needs onto the customer?

I write code for an Engineering application where engineers keep giving me various specs in a form of tabular data - for example, a table with product model names and many various parameters for each ...
1
vote
0answers
41 views

Java dashboard for machine data

I am currently working on a project for a production facility. The machines in the production are saving so called 'Telemetry Data', measurements like water pressure, plastic volume or temperature ...
1
vote
0answers
31 views

KNN Implementation in production for a retrieval arquitecture (CBIR)

We are going to start the implementation of a Content-based image retrieval (CBIR) that It will be integrated in our main product. We did tests and proof of concepts with different libraries that ...
1
vote
0answers
115 views

How to maintain table relationship across services in with a micro-service architecture

I have a scraper service that looks for media online. For every source type, there are two tables. A common source table and a source_type_media table that are related by source.id to ...
1
vote
0answers
78 views

Polling Postgres DB architecture

We are using Spring Integration DB polling which polls a table every second and executes some logic based on result. My questions are: What are some good practices we can implement to account for DB ...
1
vote
0answers
50 views

Multiple version of database content

I am working on schedule module of a project management software, every project has two types of schedules. Base Schedule Actual Schedule The actual schedule is a copy of base schedule which is ...
1
vote
0answers
162 views

Are there any advantages of Cloud SQL Proxy over a DBaaS?

I'm currently migrating an app from Heroku to Google Cloud Platform (with Kubernetes Engine and Cloud SQL). In using Heroku, I've become used to the incredibly simple practice of defining a ...
1
vote
0answers
126 views

Location specific caching for read-heavy data (server architecture)

This project will serve many duplicate requests with location-specific answers. I.e. 10,000 people in New York will all get the same server response (a list of businesses in New York), but one person ...
1
vote
0answers
283 views

Using a hardware load balancer for PostgreSQL / MySQL

I need to ensure high availability of my 3-tier architecture system. Currently I have the database mirrored by using MySQL Master-Master. All app boxes talk to the primary DB server In case of ...
1
vote
0answers
69 views

Where does search query logic go? Database or Service code?

I have a finished Chinese-English dictionary website as a personal project and it works pretty great. I'm using node/Express with node-dirty, which is a barebones file-based JSON database that loads ...
1
vote
0answers
650 views

Reading and saving big data to db

I have a method for reading data from file. The problem is how to handle files that are too big for a simple read and save to database? I was thinking about reading a chunk of it and saving it to ...
1
vote
0answers
44 views

Using fixtures to maintain static database assets

I have a pattern that's something like this: Create a database model UsagePlan which stores how many usage tokens a paid customer receives in exchange for its respective monthly fee. Through my ORM (...
1
vote
0answers
508 views

DB design for a scheduling system

I am currently designing a system to support our employee staffing process. I collect weekly preferences from employees, and feed that into a system for staffing. Availabilities can be defined as ...
1
vote
0answers
109 views

WCF Windows Service with plugin pattern, accessing several webservices and a database

I'm redesigning a WCF Windows Service (InstanceContextMode.Single) which was created by an external supplier, while it was super urgent and when there wasn't really much time to design the software. ...
1
vote
0answers
78 views

Creating an autoupdate for my program and its database

I've already an offline updater as a download but I want to make it automatically. It's a client/server software and a ms sql database. 3 parts have to be updated: update of a proprietary software (...
1
vote
0answers
56 views

concurrency in hierarchical filesystem databases

I am writing a small program that is a client of some protocol. I wanted this program to have a filesystem database. the main directory would contain one directory for a server that is configured, and ...
1
vote
0answers
92 views

high-dimensional index structures

I'm looking for information on searching (filtering) high-dimensional data. I'm not interested in nearest-neighbour search or clustering, but rather filtering/sorting by subset of dimensions - like in ...
1
vote
0answers
96 views

How to deal with weekly schema updates on database production server? (ASPNet MVC and Entity Framework)

I'm working with scrum methodology. thus weekly schema updates are performed to the final product. However our customer is already populating the database. How can I update the database without ...
1
vote
0answers
493 views

how to store state, cities and countries in Neo4j

I'm a building a registration form for my website and need to populate the country, state and city field. All these fields are inter-linked i.e depending on country, state field will be set and ...
1
vote
0answers
32 views

Offsetting high usage in a multi-tenant database environment?

I'm writing a research essay over distributed cloud-based systems, and one of the topics that I'm trying to touch on is potential solutions to high database utilization in a single-database multi-...
1
vote
0answers
1k views

Speed: Named Shared Memory vs Memory Mapped File vs Normal Memory

Let's say I have a server, where clients who connect are constantly updating their inventory. I could simply save the changes in to the database after an item has been updated, or I could update the ...
1
vote
0answers
80 views

Creating duplicate instances of objects for timeline functionality

I am working on a Spring-MVC based project, in which I would like to add Timeline functionality. The most important object in our project is the Note, there are objects associated with it like ...
1
vote
0answers
127 views

Converting our B2B application to SaaS need architecture advice

We have developed a small ERP application and the architecture right now is a 1-1 deployment/codebase for each client. We are now trying to convert it to a SaaS solution and we have the following ...
1
vote
0answers
36 views

Splitting the application into two different hosting providers

Due to cost I am currently thinking of using AWS for scalable hosting of the node.js backend of the application that would be nice to have monitoring,load balancing and all the goodies of the AWS ...
1
vote
0answers
45 views

Correlating Search Classifiers in a Database Scan for Sensitive Information

Problem Description I am working on an enterprise data discovery project that is designed to scan databases for sensitive information. The basic search unit is called a classifier and covers things ...
1
vote
0answers
209 views

Compiling data from multiple API sources, multiple languages, into a single database (MySQL or Mongo)

I'm thinking about a database schema, and would really appreciate if some of you could look at where I'm up to and offer some advice.. The mission - We have to write a program that will fetch data ...
1
vote
0answers
56 views

Database data Bucketing Report Design

Here's the scenario: I have item x (item_id, customer_number, cost) which can be submitted to another system multiple times over months which may or may not reject item x before finally accepting it ...