Questions tagged [data]

Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of interconnected nodes).

Filter by
Sorted by
Tagged with
82 votes
7 answers
30k views

Should temporary files be saved to /tmp or the current working directory?

I have a program that needs to generate temporary files. It is written for cluster machines. If I saved those files to a system-wide temporary directory (eg: /tmp), some users complained the program ...
SmallChess's user avatar
  • 1,176
80 votes
14 answers
31k views

Are data type declarators like "int" and "char" stored in RAM when a C program executes?

When a C program is running, the data is stored on the heap or the stack. The values are stored in RAM addresses. But what about the type indicators (e.g., int or char)? Are they also stored? ...
user16307's user avatar
  • 907
51 votes
4 answers
9k views

Should test data be checked into version control?

I'm writing some test code for a feature which processes PDF files. The basic idea behind the tests is that I point them towards some PDFs I've selected specially, they process them and I check that ...
Swiftheart's user avatar
47 votes
2 answers
33k views

Data Synchronization in mobile apps - multiple devices, multiple users

I'm looking into building my first mobile app. One of the core features of the application is that multiple devices/users will have access to the same data -- and all of them will have CRUD rights. I ...
ProgrammerNewbie's user avatar
43 votes
7 answers
59k views

Why the static data members have to be defined outside the class separately in C++ (unlike Java)?

class A { static int foo () {} // ok static int x; // <--- needed to be defined separately in .cpp file }; I don't see a need of having A::x defined separately in a .cpp file (or same file for ...
iammilind's user avatar
  • 2,232
42 votes
12 answers
6k views

Is there an alternative to bits?

Is there an alternative to bits as the smallest unit of data? Something that won't be only 0 or 1, but actually hold many possible states in between? Wouldn't it be more natural to store floats like ...
Dokkat's user avatar
  • 567
38 votes
7 answers
14k views

Modeling first and lastname separately

Which arguments should someone consider when designing a new system and has to either store the name of a person as one field or separately as first/last name? Pros for single field: Simpler UI No ...
Istvan Devai's user avatar
36 votes
5 answers
23k views

Does storing plain text data take up less space than storing the equivalent message in binary?

As a web developer I have very little understanding of binary data. If I take the sentence "Hello world.", convert it to binary, and store it as binary in an SQL database, it seems like the 1s and ...
john doe's user avatar
  • 1,007
35 votes
8 answers
13k views

Would having 4 states per "bit" rather than 2 mean twice the storage space?

Pretty straightforward fundamental, albeit naive, question: Would having 4 states per "bit" rather than 2 mean twice the storage space? In case that isn't clear, I mean as in if every "storage ...
J.Todd's user avatar
  • 3,823
33 votes
5 answers
11k views

Data input validation - Where? How much? [closed]

Data input validation always was quite an internal struggle to me. On the verge of adding a real security framework and code to our legacy application rewrite project (which so far pretty much keeps ...
mdrg's user avatar
  • 499
32 votes
3 answers
20k views

Is there a list of common usernames to reserve in a new system?

I need to reserve usernames in my new website. These generally fall into three categories 1) usernames no-one should have (eg: admin, user, service, help, root, etc) 2) names of super famous people ...
user avatar
29 votes
8 answers
17k views

Why does Garbage Collection only sweep the heap?

Basically, I've learned so far that garbage collection erases forever any data structure that is not currently being pointed to. But this only checks the heap for such conditions. Why doesn't it also ...
Dark Templar's user avatar
  • 6,303
20 votes
3 answers
6k views

Professional way to produce a large problem without filling up huge arrays: C++, free memory from part of an array

I'm developing a physics simulation, and as I'm rather new to programming, I keep running into problems when producing large programs (memory issues mainly). I know about dynamic memory allocation and ...
Drummermean's user avatar
19 votes
2 answers
22k views

Data validation: separated class or not?

When I have a lot of data that needs to be validated, should I create a new class for the sole purpose of validation or should I stick with in-method validation? My particular example contemplates a ...
dabadaba's user avatar
  • 2,236
19 votes
6 answers
23k views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
ThorSummoner's user avatar
19 votes
1 answer
403 views

How should HTML data formats be applied in everyday situations?

Given Google's shift towards a stronger focus on page markup data, how do the data formats used in Schema.org work alongside those for Microformats? How do these (and other specs) compliment each ...
Phil.Wheeler's user avatar
  • 1,308
18 votes
1 answer
7k views

How does Microdata rdfa compare to JSON-LD?

The bottom of the Wikipedia "talk" page says that Microdata is dead, and superseded by rdfa and json-ld. What is the difference between the formats (or value), and which one should be used? I'm ...
makerofthings7's user avatar
17 votes
4 answers
4k views

How should I store readonly data to deploy with my application?

I'm developing a desktop application, and this application requires some information to run, but it doesn't change any of this information (the data must be loaded on every execution of the app, but ...
appa yip yip's user avatar
16 votes
2 answers
20k views

File storage microservice design

Problem overview: I'm creating a spring application just for learning purposes. I would like to create a microservice just for files, which: at the beginning would have only two basic endpoints ...
user3529850's user avatar
15 votes
5 answers
2k views

What's the proper term for a function inverse to a constructor - to unwrap a value from a data type?

Edit: I'm rephrasing the question a bit. Apparently I caused some confusion because I didn't realize that the term destructor is used in OOP for something quite different - it's a function invoked ...
Petr's user avatar
  • 5,517
14 votes
8 answers
3k views

Are there examples of non CRUD approaches?

I'm a programmer but also have worked as an archivist. As archivist it's a lot about keeping data. I often get into arguments with colleagues when it comes to operations on data. I don't like the U ...
Pieter B's user avatar
  • 13.2k
13 votes
3 answers
8k views

Where should I store test data?

I have smaller unit tests that use small snippets from real data sets. I would also like to test my program against full data sets for a multitude of reasons. The only problem is that a single real ...
AlexLordThorsen's user avatar
12 votes
2 answers
282 views

Should there be a "data hygiene" index for software - to indicate how clean the program is? not leaving temp files, etc

Should there be a "data hygiene" index for software - to indicate how clean the program is? not creating unused temp files, registry entries, environment variables, etc. For example, look in your ...
therobyouknow's user avatar
11 votes
8 answers
780 views

When does 'optimizing code' == 'structuring data'?

A recent article by ycombinator lists a comment with principles of a great programmer. #7. Good programmer: I optimize code. Better programmer: I structure data. Best programmer: What's the ...
New Alexandria's user avatar
11 votes
3 answers
15k views

Is it anti-pattern to have inheritence in a dto?

Are data transfer objects or POJOs meant to be final or can they be extended and create hierarchies for them? It is not clear to me if such a value class is properly designed only as a final class and ...
Jim's user avatar
  • 327
11 votes
4 answers
3k views

Terminology: Really Delete vs set IsDeleted ("soft delete") flag

I have a site where a regular user can delete records, where what happens is that an IsDeleted flag gets set. On administrative pages, a privileged user can delete records where an actual SQL DELETE ...
Bob Kaufman's user avatar
11 votes
2 answers
1k views

Synchronization between two systems using MongoDB as changelog

We are developing two related systems. One of them (A) will be installed on our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
k91's user avatar
  • 119
10 votes
5 answers
7k views

How to decide between storage formats and what are example use cases for some of them?

We have different ways to store program data (save files in games, employee databases, program configuration etc.): Plain text (think .ini and .conf) XML Databases (MySQL, SQLite...) .zip and similar ...
Anto's user avatar
  • 11.2k
10 votes
5 answers
45k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
Mike's user avatar
  • 203
10 votes
1 answer
4k views

"Reactive programming is programming with asynchronous data streams." is this true?

I was reading about Reactive programming, and came across this article The introduction to Reactive Programming you've been missing with the quote I put in the question title: Reactive programming ...
m0meni's user avatar
  • 793
10 votes
3 answers
3k views

Microservices and isolated data storage models [duplicate]

I have been looking at Microservices for a while now. The concept is not new but it's communicated in a lightweight manner. So, I am very excited about this. However, there is a question that I am ...
tugberk's user avatar
  • 994
10 votes
1 answer
766 views

Does current evidence support the adoption of Contextual over Canonical Data Models?

The "canonical" idea is pervasive in software; patterns like Canonical Model, Canonical Schema, Canonical Data Model and so on, seem to come up again and again in development. Like many developers, I'...
Aaronaught's user avatar
  • 44.3k
9 votes
5 answers
2k views

Sensitive Data Storage - Best Practices

I recently started working on a personal project where I was connecting to a database using Java. This got me thinking. I have to provide the login information for a database account on the DB server ...
Kenneth's user avatar
  • 2,703
9 votes
1 answer
5k views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: Controller ...
Scott 's user avatar
  • 221
9 votes
7 answers
987 views

Algorithm for determining transactions among weekly data series?

I'm trying to develop a small reporting tool (with sqlite backend). I can best describe this tool as a "transaction" ledger. What I'm trying to do is keep track of "transactions" from weekly data ...
Swartz's user avatar
  • 141
8 votes
4 answers
1k views

Database design for fields that should be NOT NULL eventually, but may not yet have a value

I have encountered this problem a number of times. Let's say you have a large object that you want to store in the database. Perhaps it's a dating profile for a user, and it has 100 fields. All of ...
Will's user avatar
  • 189
8 votes
3 answers
5k views

How to deal with constructors in large data classes [duplicate]

Several times now I have come across the situations where you have some kind of settings class that simply contains a mass of data. Often these classes are simply not valid without at least most of ...
Thijser's user avatar
  • 191
8 votes
3 answers
311 views

Is a single object to be preferred over multiple variables?

It was quite hard to put what I meant into a title, but it's easy to put into code. C++ Is this int offset_x = 10; int offset_y = 40; ... element.move(offset_x, offset_y); To be preferred over ...
futlib's user avatar
  • 2,157
8 votes
3 answers
614 views

Data "unsort" / homogeneity algorithm

In an attempt to not reinvent a wheel, I'm asking if anyone has ideas on a data homogeneity algorithm. A brief example: My data has several elements maybe like Number Color Fruit Letter There are ...
ExoByte's user avatar
  • 93
8 votes
2 answers
2k views

Why build data models in a dynamic language?

Background This question stems out of an architectural discussion we had at work. For the current project we are using Python and an object oriented database. There are input services that we ...
MrFox's user avatar
  • 3,398
8 votes
4 answers
3k views

What separates text files from binary files, considering they are stored in binary?

There are text files and there are image, video, and music files. Why are image, video, and music files considered binary files, and why are text files not?
Luis Javier's user avatar
8 votes
3 answers
6k views

How to do data synchronisation between two applications?

Our client has a CRM application (let's call it A) and a custom built Windows application (VB.NET) (let's call it B) that directly writes to and reads from the CRM database. Application A is used (...
Kristof Claes's user avatar
7 votes
3 answers
1k views

Prevent analytics from crashing the production database

At the core of my business (say, an online store) is a Postgres database that stores products and transactions. During day-to-day operations, the load on the database is not heavy. However, there is ...
firstname gklsodascb's user avatar
7 votes
2 answers
2k views

Isolating test data in acceptance tests

I'm looking for guidance on how to keep my acceptance tests isolated. Right now the issue I'm having with being able to run the tests in parallel is the database records that are manipulated in the ...
Matt Phillips's user avatar
7 votes
4 answers
2k views

Can data classes contain methods for validation?

OK, say I have a data class for a user: public class User { private String firstName; private String lastName; private long personCode; private Date birthDate; private Gender gender; private String ...
Arturas M's user avatar
  • 571
7 votes
1 answer
12k views

Benefits of Java ByteArrayInputStream vs byte array?

I have a scenario where my Java process will be receiving a stream of binary data: public class DataHandler { public void handleData(DATA data) { // TODO: Do something with data } } ...
herpylderp's user avatar
  • 2,027
7 votes
2 answers
307 views

Seeking benefits of both SQL and binary files in a data storage scheme

I'm looking for some input on how to keep certain benefits of both a binary file and a SQL database in an interesting data storage problem. I have a legacy custom binary file format that is ...
Ian W's user avatar
  • 171
7 votes
1 answer
3k views

What the fastest way to pass large data between JVMs?

I have 2 JVMs on the same machine that I want to pass about 1Mb of (serializable) data between ideally in under 5 ms. Under load, using HTTP to localhost takes about 70ms average. I tried hazelcast, ...
Bohemian's user avatar
  • 1,996
7 votes
3 answers
432 views

Are there best practices for checking for errors in business logic for DB refreshes? [closed]

I'm working for a company with very large databases which all refresh overnight, ready for users to interrogate in the morning. These over night loads essentially truncate all the tables, and reload ...
CatchingMonkey's user avatar
7 votes
1 answer
885 views

How much of the data preparation, transformation, and processing belongs in Repository layer?

I have a use case where I display variables in a PDF file. But to get there is not simple, but I could do something like this: /* * takes product model number, retrieves, processes, formats values ...
Dennis's user avatar
  • 8,227

1
2 3 4 5
10