Questions tagged [data]
Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of interconnected nodes).
-2
votes
0answers
34 views
Design Pattern for Data Enrichment vs Similarity Matching [duplicate]
I have some things (JSON) coming from various sources (Search engine, Crawled HTML, APIs...). I'm trying to match similar things from the different sources as follows:
[1] Collect -> [2] Prepare -&...
1
vote
1answer
27 views
At what size array in Swift would you consider switching to CoreData?
I've developed a reading tracking app in Swift and for simplicity's sake I used arrays for my model. It stores some data about the books a user reads and a small image of the cover. I save them to ...
0
votes
1answer
97 views
Encoding of timestamps over cosmological time (64-bit linear vs 32-bit logarithmic)
I understand this is the place to ask questions pertaining to software design. If my question is considered to opinion-based, is there a forum or something better suited for the discussion?
...
3
votes
1answer
113 views
Arranging blocks of data for efficient memory access
suppose I have an algorthm that has N blocks of data, and an operation needs to be performed on all blocks such that requires 2 blocks to be in memory at any given time to be done for all possible ...
2
votes
1answer
106 views
Is it possible to create once loadable files?
I know very little about encryption, so I figured I'd ask the internet if it's possible to do the following (and if so, what kind of algorithm should I be looking at):
Write some code that saves data ...
2
votes
1answer
87 views
Generic way to describe data synchronization between systems?
Is there a generic way (useful formalisms) to think about data synchronization between two systems, especially in the case of slightly asymmetric roles (eg, one system contain master data, another one ...
0
votes
3answers
174 views
Getting random yet consistent data for testing
I have a big codebase yet to be covered with tests and rather limited access to the real data.
The code that needs to be covered is very data dependent. And the data is anything but shallow and ...
4
votes
1answer
75 views
Maintaining parallel libraries: Binary files access + Metadata Database ORM design
I'm dealing with a higher-level data abstraction that I would appreciate some input on.
I'm working on an application that uses a large data lake. The data lake is consisted of thousands of large ...
0
votes
1answer
108 views
Am I using Socket.io wrongly? Or use REST instead?
In client:
function test(){
const socket = io.connect(SERVER URL)
socket.emit('read_user_name', '1') //user name id: 1
socket.on('returned_user_name', (data) => {
console.log(data)
...
-1
votes
1answer
35 views
How should I update back-end data (e.g. follows, likes, etc.) which changes at an inconsistent rate?
(This is a conceptual question but as reference, I'm using Android Studio (Java) and Firebase Firestore...)
My app currently has a structure where the user can follow authors and favorite their works....
-1
votes
1answer
64 views
Formula Parsing
I have created an application to import CSV files, each CSV file that the application imports can contain different data, with new formats of the CSV files being added in the future. Rather than hard ...
3
votes
2answers
134 views
Is this data as a result of business logic or is this content as a result of GUI logic?
I'm aware of the general distinction between data and content, namely context, but here I have a case that is clearly causing a lot of confusion and debate within the company I work for.
There is a ...
1
vote
1answer
35 views
Data Matching In VBA - Best way to deal with dynamic data and user entry?
Background
I am currently building this project with VBA, just to keep in the back of your mind when thinking about my question.
Imagine 2 adjacent blocks, in Excel. The first block is made up of ...
3
votes
1answer
98 views
Design of file I/O -> processing -> file I/O system
I'm in the process of designing a new desktop application which is very different from other stuff I did before, and so I'll be happy if I could be pointed towards the right direction regarding the ...
2
votes
5answers
268 views
Which layer should have responsibility for rounding numbers?
I am developing a financial system and want to have a defined policy for rounding monetary values.
Given the following layers:
View
API
Entity Model
Persistence
If I am passing a monetary value ...
4
votes
2answers
251 views
Game Playing AI - Strategy to overcome the transposition table taking up too much memory?
I've made an engine that plays Connect Four using the standard search algorithms (minimax, alpha-beta pruning, iterative deepening, etc). I've implemented a transposition table so that the engine can ...
4
votes
1answer
175 views
Self describing data with UI
I have a use case where my team owns a web UI to aggregate data from 10+ service teams and present it to the customer in order to collect more information . This data (which is in json format) is very ...
5
votes
6answers
631 views
Is there a programming term that means property or field?
I think this question can span multiple languages, but in C# specifically there is a clear distinction between a field (ex: private int a;) and a property (ex: private int b { get; set; }). While ...
1
vote
1answer
46 views
Dictionary Coding for OLAP
Let's say we have a generic table like below:
id, name, price, quantity
20 product_x 5,00 100
20 product_y 5,00 100
20 ...
0
votes
2answers
72 views
Should / When to create javascript object to serialize data?
I'm working on a React/Redux application. I receive a data object from the api resembling this:
const dataObj = [
{
id: 1,
name: "Filo",
activity: 9,
info: {
statusCode: 4
...
1
vote
1answer
93 views
Improvements in a “data - manager - wrapper” architecture
I have programmed using the architecture below, where "client, order, piece" are just data classes, have no methods, and "managers" are the manipulators of that data, the "ClothingStore" is a wrapper ...
3
votes
1answer
114 views
Regulations about collecting data from software running at clients hardware (apps 'phoning home') [closed]
We develop an application which has now been installed on a few thousand customers sites.
Unfortunately, we have no customer data collection built into the software. This is something we want to ...
-2
votes
2answers
46 views
database/orm model design for contacts
I am building a internal crm which has contacts which are just people with first name, last name, email, phone etc. Contacts can be one or more types e.t.c. job candidates, hiring managers, customers, ...
2
votes
2answers
117 views
Cumulative sum of overlapping increase/decrease over time
I am looking for an efficient way to handle a calculation. I have lots of timestamped events, representing loading and unloading of trucks. I need to construct the filling rate of these trucks over ...
1
vote
2answers
182 views
How to find matching profiles?
I'm developing backend for a dating app, in which each user has
a profile of his/her characteristics
a profile of ideal match's characteristics
There are dozens of characteristics like gender, ...
1
vote
0answers
56 views
separate table for one of the type or java enum?
I want to make budget tracking web-app, tracking all savings and expenses with corresponding dates. Later generating reports, summaries etc..
I came up this design
Frequency Table - does it have to ...
1
vote
2answers
154 views
Distributed training of many small ML models
I have a data science application that involves training tens of thousands of small individual Gaussian models. By "small", I mean that any individual model can easily be trained on one of our worker ...
1
vote
0answers
58 views
How can I offload customer's data needs onto the customer?
I write code for an Engineering application where engineers keep giving me various specs in a form of tabular data - for example, a table with product model names and many various parameters for each ...
3
votes
2answers
2k views
File storage microservice design
Problem overview:
I'm creating a spring application just for learning purposes. I would like to create a microservice just for files, which:
at the beginning would have only two basic endpoints ...
3
votes
3answers
188 views
How to store external numeric IDs - as strings or as integers?
Every now and then we programmers need to communicate with external systems (not under our own control). It's frequent that in such cases we need to keep track of some sort of identifiers that these ...
0
votes
1answer
387 views
Battery level prediction model
I have the idea of develop a model that predicts the battery charge level of my system for now until the following 5 days. The battery is charged using a solar panel. I am writing my code in Python 2....
1
vote
1answer
93 views
Which datastructure to use for temporary storage of intermittent and asynchronous sensor data?
Here an outline of the setup:
Input data:
I have several sources for sensor input data (GPS, and a bunch of other sensor data from different sources).
All of those individual inputs consist of a ...
-2
votes
2answers
400 views
data vs business logic
Today my manager said: 'Amir, don't mix the business logic with data in database'.
I surprisingly shocked! I said 'Where?'.
According to wikipedia business logic is:
In computer software, business ...
-1
votes
2answers
73 views
How to structure an application (and its user interface) such that extraneous and/or malformed data can be handled?
I recently applied for a position as a software developer. As a part of the recruitment process, I was given an assignment that involves reading data from a file.
The thing is that the example file ...
3
votes
2answers
153 views
Does Wikipedia Store Every Edit Page? [closed]
I am not sure if this is the best place to ask this question, but I was looking at a Wikipedia article and noticed there were a lot of edits for that article. Since you can view each edited iteration ...
1
vote
1answer
106 views
Are data attributes good identifiers of certain blocks of HTML code?
I have a complex environment which is mostly oriented around / ends up spitting out blocks of HTML, which make up the entire page.
Now, I need a way to "point and click" identify any block on my page ...
3
votes
1answer
1k views
Microservices - sharing data through distributed in-memory cache?
I'm fairly new to both microservices and shared in-memory databases like Redis.
I'm wondering though if it would be a good idea to combine the two to address the challenges of data duplication, ...
0
votes
1answer
69 views
Abstract data communication in modular design
I'm doing the analysis for a software which I've had in mind for a long time. Its purpose is to turn sound into an image, apply graphical transformations on it, and turn it back into sound to hear the ...
1
vote
1answer
91 views
Software layers and data mapping
Before I would ask my question, here is the context: I am working on one part of a relatively big software. In its simplest form there are 3 layers of the software lets call them lower layer, middle ...
1
vote
0answers
184 views
Putting the rowmapper inside the domain class
Normally in java the RowMapper is placed in its own class.
Instead, what about this design, putting the RowMapper inside each domain object, and giving all of them a consistent name:
public class ...
0
votes
1answer
124 views
What is the best way to represent data with inheritance?
In my current work, we need to serialize some data and use it later.
Our data is in this format:
<resources>
<style name="Person">
<item name="name">name</item>
<...
1
vote
2answers
415 views
Where to place POCOs validation - N-layers architecture
I am developing an n-layer app.
One of the layer is the BusinessLayer and consumes a set of POCOs defined in the CoreLayer. Also I have a PresentationLayer (WinForms)
CoreLayer
POCOs (classes with ...
1
vote
2answers
93 views
Design patterns / Names for data consolidation [on hold]
I'm new to coding and I don't know how what I'm trying to do is called, and I want to find out so I can do more research and not re-invent the wheel. I looked for things such as data consolidation/...
-1
votes
1answer
501 views
Does it make sense to iterate a ranged for loop using constant reference here?
I have the following code, and I was wondering if I'm optimally iterating through my ranged for loop:
struct data_type
{
int a;
int b;
};
int main()
{
// Assume I have initialized a ...
-4
votes
1answer
74 views
What options are there to administrate a database for an online business? [closed]
Say you create a small online business from scratch, a nice Rails or Django app or something to that effect.
With luck, you end up with a database full of customer orders and things.
Now, aside ...
0
votes
1answer
523 views
How to store large test data sets. Git? [duplicate]
We have a camera application that does measurements on pictures. We have a feature for saving a "demo" containing the pictures/data used, and to replay this for test purposes. We use such data sets ...
-2
votes
2answers
138 views
Are there benefits to having a SQL-like read only database layer over a Blockchain distributed ledger? [closed]
Distributed Blockchain datastores provide inmutability to recorded transactions and only allow to add records accepted by a consensus algorithm in sequence. You could store data on the blockchain or ...
1
vote
1answer
57 views
How to solve double withdrawal attacks on decentralized databases
I need a way to solve double withdrawal attacks in decentralized databases.
For example, a banking infrastructure has multiple servers with their own databases, the advantage is that it lets citizens ...
3
votes
2answers
133 views
Is it acceptable to test based on test output data rather than input data in unit tests?
I'm used to write unit tests with assertions which are based on the input, e.g. (hopefully self-explanatory and let's assume that using random test data is fine)
int a = random();
int b = random();
...
3
votes
1answer
101 views
Should files/data follow rules like 'open-closed principle'?
Does 'open-closed principle' also includes files and data of the app? According to https://softwareengineering.stackexchange.com/a/296955/248528, it says files and data are't included, but it doesn't ...
