Questions tagged [database]

This tag is for general database questions; if you question is specific to SQL use that tag instead.

121 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6
votes
0answers
87 views

When connecting to a server using the DRDA protocol, is it true that the first Client-To-Server command MUST be EXCSAT chained with ACCSEC?

When connecting to a server using the DRDA protocol, is it true that the first Client-To-Server command MUST be EXCSAT chained with ACCSEC? I found 2 different answers when I googled it. If you look ...
5
votes
0answers
325 views

Concurrent fault-safe data structure

I am building an application that aims to process ~10M data items per second. Each item is exactly 42 bytes small (including a sorting key) which means the total data rate will not be big 420 MB/s. ...
4
votes
0answers
281 views

Data durability guarantees in Kafka

Is it wise to use kafka as the 'source of truth' for mission critical data? The setup is: kafka is the underlying source-of-truth for the data. -querying is done on caches (I.e. Redis, ktables) ...
4
votes
1answer
385 views

Continuous Deployment Database Project VSTS

Looking for continuous deployment strategies regarding SQL server database projects targeting Azure SQL using VSTS. My scenario... Using VSTS for CI/CD Using a SQL Server Database Project to define ...
4
votes
0answers
426 views

Class table inheritance using Slick: always an anti-pattern?

My application's model includes a number of entities that share many common properties, and are operated on by many of the same classes. If I was using an ORM, I'd define an abstract class with the ...
3
votes
3answers
417 views

Aggregation of data from two Microservices

I have two Microservices A and B. B Microservice has a large set of an entity called User. A Microservice stores the User entity in its own DB if User is configured by an agent. There is no flag ...
3
votes
0answers
220 views

Architecture: mobile apps for different clients with their own dbs different servers?

My general feeling is that this is a question which must have been answered but I am struggling to find resources for it. I know little about mobile app dev but my knowledge of web services is pretty ...
3
votes
0answers
540 views

Is Event sourcing a good fit for database sync (master-multiple-slaves)?

I'm looking for a good way to maintain in sync local sqlite databases that could be offline to a master one, for a invoice app. Then I read about EventSourcing and Log structured storage and wonder ...
3
votes
0answers
675 views

Database version management on mobile apps

I am about to start building a mobile (iOS/Android) app that uses a local database (probably SQLite). Since this is my first "real" mobile app, I am wondering what the standard approaches are to ...
3
votes
1answer
684 views

Design for syncing data from remote API

I am trying to find a robust and correct solution for syncing data from different sources to my local relational database. I have different sources of data for eg: CSV, REST API and probably network ...
2
votes
0answers
30 views

Designing provisioning strategies

I'm trying to redesign a provisioning application that administrates other application through their REST API. The application is quite simple, we have groups, we have users and roles and we have a ...
2
votes
0answers
134 views

Chronicle Queue: how to make a stack and how to limit disk usage

My server and several clients are different JVMs running on the same machine, they need to share persisted time series data: server reads a time series from somewhere, writes to a persisted embedded ...
2
votes
0answers
349 views

File upload sync for offline desktop application. (Electron)

We are making an offline desktop application in Electron that needs to sync when the user goes online. Sqlite is used as the database when offline and it gets synced to postgres on the server when ...
2
votes
1answer
173 views

Designing database job monitoring

I have several Oracle database where my in-house applications are running. Those applications use both dba_jobs and dba_scheduler_jobs. I want to write monitoring function: check_my_jobs which will ...
2
votes
0answers
84 views

Idea on replacing DAO/Domain obj pattern

I know someone has probably already thought of this, but here is a way to replace DAOs and Domain objects: For example public class Bike { private int id; public Bike(String model) { //...
2
votes
0answers
730 views

Reading and saving big data to db

I have a method for reading data from file. The problem is how to handle files that are too big for a simple read and save to database? I was thinking about reading a chunk of it and saving it to ...
2
votes
0answers
570 views

How do I implement Socket.io Leaderboard using AWS Dynamo DB cost-effectively?

Every time a user submits a score, Socket.IO causes all the users get their leaderboard updated. This causes every single user connected through Socket.IO to read from the Dynamo DB database (every ...
2
votes
0answers
156 views

Are there downsides to writing PDO/mysqli statements into my code directly vs using my own redirect wrapper methods?

TL;DR: To access DB functionality in my codebase I currently use a custom layer throughout the code. The layer needs an upgrade in order to begin accepting prepared statements, which it currently ...
2
votes
0answers
283 views

Logging Database Changes

What is the proper way of storing a changelog, similar to git, for data in a database. Currently, I have it set up so that every line in a table is given a unique string id that is used to determine ...
2
votes
0answers
213 views

Saving tabular, spreadsheet-like data in a relational database?

I'm hoping this question isn't too broad or too prone to opinionated answers, since I could really use some pointers. I'm trying to come up with a way to persist tabular, spreadsheet-like1, data in a ...
2
votes
0answers
392 views

Maximising data processing speed: in-memory or write to db first?

Currently working on a system which allows users to upload data files, which we then "clean" (i.e. process in various ways). Speed of processing and return to the user is an important consideration, ...
2
votes
0answers
88 views

Suggestions for Storing large collection of related words

I am trying to create a simple sentence generator that uses templates and a database of words. It will be a website where essentially a user could click a button to generate a sentence. For example: ...
2
votes
0answers
655 views

how to import large amounts of data without locking sqlite database

Background Information I have a web app that provides the user with the usual CRUD functionality for "widgets" let's say. But I also have give them the option of creating widgets in bulk via ...
2
votes
0answers
473 views

Syncing structured data and files in Android App with an ORM?

I am developing an Android App which I have developed for iOS before. The app requires that structured data and files are synced across android devices. In iOS, I was using Core Data and synced to ...
2
votes
1answer
339 views

Three phase commit protocol

I'm trying to understand the three phase commit protocol by reading the wiki page (http://en.wikipedia.org/wiki/Three-phase_commit_protocol#Motivation). In the description for the diagram I posted ...
2
votes
0answers
170 views

What are some important considerations when developing a DBaaS (DataBase as a Service)

I'm interested in creating a Database as a Service. I want users of the service to be able to do a few seemingly not very simple things. Sign up and be given access to a JSON datastore that they can ...
2
votes
2answers
257 views

Should audit records have their own id?

I'm creating audit tables to track changes made to some of my crucial tables and I'm wondering whether there's a point to history rows having their own id. Here's how my interfaces look like: public ...
1
vote
0answers
116 views

How to store relational data belonging to different client customers?

My users will be businesses with a small number of accounts each: e.g. Business #1 with 3 users, Business #2 with 5 users, etc. I am trying to determine the best way to organise the relation (on ...
1
vote
0answers
58 views

Strategies for dealing with Stale data and Fetching

I'm working with firestore and have encountered several situations where a document will contain stale data which is a duplicate of the source of truth located elsewhere in the database. For ...
1
vote
1answer
39 views

how simulate sequential consumers in Kafka

I have kafka consumer that get data from kafka and do some process if some circumstances meet and send it for another consumer to do it's job. How can i build something like this? One way I think ...
1
vote
0answers
65 views

Single users database for many apps

I'm designing a database that holds the information for users of three diferentes apps that share the access to the data trough an API. For every user i'm storing credentials, profile and roles. What ...
1
vote
0answers
59 views

separate table for one of the type or java enum?

I want to make budget tracking web-app, tracking all savings and expenses with corresponding dates. Later generating reports, summaries etc.. I came up this design Frequency Table - does it have to ...
1
vote
0answers
58 views

How can I offload customer's data needs onto the customer?

I write code for an Engineering application where engineers keep giving me various specs in a form of tabular data - for example, a table with product model names and many various parameters for each ...
1
vote
0answers
61 views

Java dashboard for machine data

I am currently working on a project for a production facility. The machines in the production are saving so called 'Telemetry Data', measurements like water pressure, plastic volume or temperature ...
1
vote
0answers
34 views

KNN Implementation in production for a retrieval arquitecture (CBIR)

We are going to start the implementation of a Content-based image retrieval (CBIR) that It will be integrated in our main product. We did tests and proof of concepts with different libraries that ...
1
vote
0answers
159 views

How to maintain table relationship across services in with a micro-service architecture

I have a scraper service that looks for media online. For every source type, there are two tables. A common source table and a source_type_media table that are related by source.id to ...
1
vote
0answers
118 views

Polling Postgres DB architecture

We are using Spring Integration DB polling which polls a table every second and executes some logic based on result. My questions are: What are some good practices we can implement to account for DB ...
1
vote
0answers
191 views

Are there any advantages of Cloud SQL Proxy over a DBaaS?

I'm currently migrating an app from Heroku to Google Cloud Platform (with Kubernetes Engine and Cloud SQL). In using Heroku, I've become used to the incredibly simple practice of defining a ...
1
vote
0answers
44 views

Where do I store the record sources for a form's controls?

I'm working on a Windows application that is used for tracking document review and audit information for my company. The back end is a SQL server database and most of the user tasks are basic CRUD ...
1
vote
0answers
152 views

Location specific caching for read-heavy data (server architecture)

This project will serve many duplicate requests with location-specific answers. I.e. 10,000 people in New York will all get the same server response (a list of businesses in New York), but one person ...
1
vote
0answers
320 views

Using a hardware load balancer for PostgreSQL / MySQL

I need to ensure high availability of my 3-tier architecture system. Currently I have the database mirrored by using MySQL Master-Master. All app boxes talk to the primary DB server In case of ...
1
vote
0answers
70 views

Where does search query logic go? Database or Service code?

I have a finished Chinese-English dictionary website as a personal project and it works pretty great. I'm using node/Express with node-dirty, which is a barebones file-based JSON database that loads ...
1
vote
0answers
47 views

Using fixtures to maintain static database assets

I have a pattern that's something like this: Create a database model UsagePlan which stores how many usage tokens a paid customer receives in exchange for its respective monthly fee. Through my ORM (...
1
vote
0answers
580 views

DB design for a scheduling system

I am currently designing a system to support our employee staffing process. I collect weekly preferences from employees, and feed that into a system for staffing. Availabilities can be defined as ...
1
vote
0answers
121 views

WCF Windows Service with plugin pattern, accessing several webservices and a database

I'm redesigning a WCF Windows Service (InstanceContextMode.Single) which was created by an external supplier, while it was super urgent and when there wasn't really much time to design the software. ...
1
vote
0answers
81 views

Creating an autoupdate for my program and its database

I've already an offline updater as a download but I want to make it automatically. It's a client/server software and a ms sql database. 3 parts have to be updated: update of a proprietary software (...
1
vote
0answers
58 views

concurrency in hierarchical filesystem databases

I am writing a small program that is a client of some protocol. I wanted this program to have a filesystem database. the main directory would contain one directory for a server that is configured, and ...
1
vote
0answers
95 views

high-dimensional index structures

I'm looking for information on searching (filtering) high-dimensional data. I'm not interested in nearest-neighbour search or clustering, but rather filtering/sorting by subset of dimensions - like in ...
1
vote
0answers
101 views

How to deal with weekly schema updates on database production server? (ASPNet MVC and Entity Framework)

I'm working with scrum methodology. thus weekly schema updates are performed to the final product. However our customer is already populating the database. How can I update the database without ...
1
vote
0answers
573 views

how to store state, cities and countries in Neo4j

I'm a building a registration form for my website and need to populate the country, state and city field. All these fields are inter-linked i.e depending on country, state field will be set and ...