Questions tagged [data]

Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of interconnected nodes).

Filter by
Sorted by
Tagged with
80
votes
7answers
23k views

Should temporary files be saved to /tmp or the current working directory?

I have a program that needs to generate temporary files. It is written for cluster machines. If I saved those files to a system-wide temporary directory (eg: /tmp), some users complained the program ...
77
votes
14answers
25k views

Are data type declarators like “int” and “char” stored in RAM when a C program executes?

When a C program is running, the data is stored on the heap or the stack. The values are stored in RAM addresses. But what about the type indicators (e.g., int or char)? Are they also stored? ...
43
votes
2answers
31k views

Data Synchronization in mobile apps - multiple devices, multiple users

I'm looking into building my first mobile app. One of the core features of the application is that multiple devices/users will have access to the same data -- and all of them will have CRUD rights. I ...
42
votes
12answers
5k views

Is there an alternative to bits?

Is there an alternative to bits as the smallest unit of data? Something that won't be only 0 or 1, but actually hold many possible states in between? Wouldn't it be more natural to store floats like ...
42
votes
4answers
7k views

Should test data be checked into version control?

I'm writing some test code for a feature which processes PDF files. The basic idea behind the tests is that I point them towards some PDFs I've selected specially, they process them and I check that ...
41
votes
7answers
53k views

Why the static data members have to be defined outside the class separately in C++ (unlike Java)?

class A { static int foo () {} // ok static int x; // <--- needed to be defined separately in .cpp file }; I don't see a need of having A::x defined separately in a .cpp file (or same file for ...
35
votes
8answers
12k views

Would having 4 states per “bit” rather than 2 mean twice the storage space?

Pretty straightforward fundamental, albeit naive, question: Would having 4 states per "bit" rather than 2 mean twice the storage space? In case that isn't clear, I mean as in if every "storage ...
33
votes
7answers
10k views

Modeling first and lastname separately

Which arguments should someone consider when designing a new system and has to either store the name of a person as one field or separately as first/last name? Pros for single field: Simpler UI No ...
32
votes
3answers
14k views

Does storing plain text data take up less space than storing the equivalent message in binary?

As a web developer I have very little understanding of binary data. If I take the sentence "Hello world.", convert it to binary, and store it as binary in an SQL database, it seems like the 1s and ...
29
votes
5answers
8k views

Data input validation - Where? How much? [closed]

Data input validation always was quite an internal struggle to me. On the verge of adding a real security framework and code to our legacy application rewrite project (which so far pretty much keeps ...
28
votes
8answers
15k views

Why does Garbage Collection only sweep the heap?

Basically, I've learned so far that garbage collection erases forever any data structure that is not currently being pointed to. But this only checks the heap for such conditions. Why doesn't it also ...
28
votes
3answers
16k views

Is there a list of common usernames to reserve in a new system?

I need to reserve usernames in my new website. These generally fall into three categories 1) usernames no-one should have (eg: admin, user, service, help, root, etc) 2) names of super famous people ...
20
votes
3answers
5k views

Professional way to produce a large problem without filling up huge arrays: C++, free memory from part of an array

I'm developing a physics simulation, and as I'm rather new to programming, I keep running into problems when producing large programs (memory issues mainly). I know about dynamic memory allocation and ...
19
votes
1answer
401 views

How should HTML data formats be applied in everyday situations?

Given Google's shift towards a stronger focus on page markup data, how do the data formats used in Schema.org work alongside those for Microformats? How do these (and other specs) compliment each ...
18
votes
2answers
16k views

Data validation: separated class or not?

When I have a lot of data that needs to be validated, should I create a new class for the sole purpose of validation or should I stick with in-method validation? My particular example contemplates a ...
17
votes
4answers
4k views

How should I store readonly data to deploy with my application?

I'm developing a desktop application, and this application requires some information to run, but it doesn't change any of this information (the data must be loaded on every execution of the app, but ...
14
votes
8answers
2k views

Are there examples of non CRUD approaches?

I'm a programmer but also have worked as an archivist. As archivist it's a lot about keeping data. I often get into arguments with colleagues when it comes to operations on data. I don't like the U ...
13
votes
5answers
2k views

What's the proper term for a function inverse to a constructor - to unwrap a value from a data type?

Edit: I'm rephrasing the question a bit. Apparently I caused some confusion because I didn't realize that the term destructor is used in OOP for something quite different - it's a function invoked ...
13
votes
6answers
14k views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
11
votes
5answers
44k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
11
votes
1answer
4k views

How does Microdata rdfa compare to JSON-LD?

The bottom of the Wikipedia "talk" page says that Microdata is dead, and superseded by rdfa and json-ld. What is the difference between the formats (or value), and which one should be used? I'm ...
11
votes
2answers
258 views

Should there be a “data hygiene” index for software - to indicate how clean the program is? not leaving temp files, etc

Should there be a "data hygiene" index for software - to indicate how clean the program is? not creating unused temp files, registry entries, environment variables, etc. For example, look in your ...
11
votes
2answers
1k views

Synchronization between two systems using MongoDB as changelog

We are developing two related systems. One of them (A) will be installed on our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
10
votes
3answers
5k views

Where should I store test data?

I have smaller unit tests that use small snippets from real data sets. I would also like to test my program against full data sets for a multitude of reasons. The only problem is that a single real ...
10
votes
5answers
7k views

How to decide between storage formats and what are example use cases for some of them?

We have different ways to store program data (save files in games, employee databases, program configuration etc.): Plain text (think .ini and .conf) XML Databases (MySQL, SQLite...) .zip and similar ...
10
votes
4answers
2k views

Terminology: Really Delete vs set IsDeleted (“soft delete”) flag

I have a site where a regular user can delete records, where what happens is that an IsDeleted flag gets set. On administrative pages, a privileged user can delete records where an actual SQL DELETE ...
10
votes
3answers
2k views

Microservices and isolated data storage models [duplicate]

I have been looking at Microservices for a while now. The concept is not new but it's communicated in a lightweight manner. So, I am very excited about this. However, there is a question that I am ...
9
votes
8answers
737 views

When does 'optimizing code' == 'structuring data'?

A recent article by ycombinator lists a comment with principles of a great programmer. #7. Good programmer: I optimize code. Better programmer: I structure data. Best programmer: What's the ...
9
votes
2answers
8k views

File storage microservice design

Problem overview: I'm creating a spring application just for learning purposes. I would like to create a microservice just for files, which: at the beginning would have only two basic endpoints ...
9
votes
1answer
668 views

Does current evidence support the adoption of Contextual over Canonical Data Models?

The "canonical" idea is pervasive in software; patterns like Canonical Model, Canonical Schema, Canonical Data Model and so on, seem to come up again and again in development. Like many developers, I'...
9
votes
1answer
4k views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: Controller ...
9
votes
7answers
884 views

Algorithm for determining transactions among weekly data series?

I'm trying to develop a small reporting tool (with sqlite backend). I can best describe this tool as a "transaction" ledger. What I'm trying to do is keep track of "transactions" from weekly data ...
8
votes
3answers
3k views

How to deal with constructors in large data classes [duplicate]

Several times now I have come across the situations where you have some kind of settings class that simply contains a mass of data. Often these classes are simply not valid without at least most of ...
8
votes
3answers
296 views

Is a single object to be preferred over multiple variables?

It was quite hard to put what I meant into a title, but it's easy to put into code. C++ Is this int offset_x = 10; int offset_y = 40; ... element.move(offset_x, offset_y); To be preferred over ...
8
votes
5answers
2k views

Sensitive Data Storage - Best Practices

I recently started working on a personal project where I was connecting to a database using Java. This got me thinking. I have to provide the login information for a database account on the DB server ...
8
votes
3answers
521 views

Data “unsort” / homogeneity algorithm

In an attempt to not reinvent a wheel, I'm asking if anyone has ideas on a data homogeneity algorithm. A brief example: My data has several elements maybe like Number Color Fruit Letter There are ...
7
votes
4answers
647 views

Database design for fields that should be NOT NULL eventually, but may not yet have a value

I have encountered this problem a number of times. Let's say you have a large object that you want to store in the database. Perhaps it's a dating profile for a user, and it has 100 fields. All of ...
7
votes
1answer
3k views

“Reactive programming is programming with asynchronous data streams.” is this true?

I was reading about Reactive programming, and came across this article The introduction to Reactive Programming you've been missing with the quote I put in the question title: Reactive programming ...
7
votes
2answers
1k views

Why build data models in a dynamic language?

Background This question stems out of an architectural discussion we had at work. For the current project we are using Python and an object oriented database. There are input services that we ...
7
votes
1answer
9k views

Benefits of Java ByteArrayInputStream vs byte array?

I have a scenario where my Java process will be receiving a stream of binary data: public class DataHandler { public void handleData(DATA data) { // TODO: Do something with data } } ...
7
votes
1answer
2k views

What the fastest way to pass large data between JVMs?

I have 2 JVMs on the same machine that I want to pass about 1Mb of (serializable) data between ideally in under 5 ms. Under load, using HTTP to localhost takes about 70ms average. I tried hazelcast, ...
7
votes
3answers
353 views

Are there best practices for checking for errors in business logic for DB refreshes? [closed]

I'm working for a company with very large databases which all refresh overnight, ready for users to interrogate in the morning. These over night loads essentially truncate all the tables, and reload ...
7
votes
3answers
4k views

How to do data synchronisation between two applications?

Our client has a CRM application (let's call it A) and a custom built Windows application (VB.NET) (let's call it B) that directly writes to and reads from the CRM database. Application A is used (...
6
votes
5answers
353 views

Why is it considered a good practice to isolate data management in a microservice?

I have seen proponents of isolating data management and storage into microservices in a way that each microservice has its own data store (at least in a logical sense). Hence, joins and aggregations ...
6
votes
1answer
4k views

Is there any use case for using 'L' letter after a number literal in C?

I know that in C code using L after a number tells the compiler that the respective number is L. However I do not see any practical use for this. Do modern compilers still have a use for this ...
6
votes
3answers
4k views

Saving to Registry vs. CustomFile [closed]

I've been using the usual My.Settings... method when it came to saving settings for my program but I was just wondering, If I wanted to save program settings into the Registry and load those settings, ...
6
votes
2answers
203 views

Seeking benefits of both SQL and binary files in a data storage scheme

I'm looking for some input on how to keep certain benefits of both a binary file and a SQL database in an interesting data storage problem. I have a legacy custom binary file format that is ...
6
votes
1answer
862 views

What method should I use to populate a tree as I'm consuming data that is contained within the tree?

I have a tree that consists of 5 types of nodes - V, W, X, Y, and Z. V represents the root node, and a given tree has exactly one element of type V. A V contains one or more Ws, a W contains one or ...
6
votes
1answer
651 views

How much of the data preparation, transformation, and processing belongs in Repository layer?

I have a use case where I display variables in a PDF file. But to get there is not simple, but I could do something like this: /* * takes product model number, retrieves, processes, formats values ...
6
votes
2answers
637 views

Bottleneck performance in ECS

I've been looking into building an entity-component-system. Basically, an entity is just an id wrapped around a struct, components are data belonging to that entity(and reference said id), and systems ...

1
2 3 4 5
8