Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [storage]

Storage - refers to questions where data persistence and storage questions and techniques are asked.

-4
votes
2answers
113 views

How the value of the integer variable is stored in a memory by bytes/bits?

In C program I'm doing below stuff int x = 4; Let us assume integer has 2 bytes in this case. So my question here is the variable x will hold two bytes that mean 16 bits. So here how the value 4 ...
-3
votes
3answers
140 views

Is storing configurations in memory more performance and maintenance efficient? [closed]

My teammate has this class which contains a lot of Strings: public class Config { /** * List of status */ public static final String IN = "bla*"; public static final String ...
5
votes
0answers
210 views

Concurrent fault-safe data structure

I am building an application that aims to process ~10M data items per second. Each item is exactly 42 bytes small (including a sorting key) which means the total data rate will not be big 420 MB/s. ...
0
votes
2answers
117 views

Should a SVG be converted to JSON to be stored in the database?

Consider an application where you can create and edit a SVG file (e.g. draw.io). How should the SVG be stored in the database? With SVG (image/svg+xml) Thus no further processing is required before ...
1
vote
0answers
54 views

How can I offload customer's data needs onto the customer?

I write code for an Engineering application where engineers keep giving me various specs in a form of tabular data - for example, a table with product model names and many various parameters for each ...
3
votes
3answers
125 views

Data storage for small application using XML

I am developing a small application as a hobbyist. This application will track costs (eg. purchases and sales, ...) for a single person company. Only string, date, double, integer, and decimal types ...
3
votes
3answers
174 views

How to store external numeric IDs - as strings or as integers?

Every now and then we programmers need to communicate with external systems (not under our own control). It's frequent that in such cases we need to keep track of some sort of identifiers that these ...
-1
votes
1answer
15 views

Data store for sequential key-value, time limited event “journal”

I'm designing a system in which a key component acts like a "journal", in the sense that it stores a list of sequential events that can change the state of the entire system. Upon request, any other ...
2
votes
2answers
150 views

Does Wikipedia Store Every Edit Page? [closed]

I am not sure if this is the best place to ask this question, but I was looking at a Wikipedia article and noticed there were a lot of edits for that article. Since you can view each edited iteration ...
-4
votes
1answer
140 views

Is this a legit question? [closed]

I see an interview question thats goes like this: Assuming you have an array with numbers 1 to N. Where N is at most 32,000. This array may have duplicates and you do not know what N is. With, 4KB ...
-2
votes
1answer
113 views

What is the preferable date-format used for designing APIs and storage

The date format should support different timezone and Daylight saving issues.
2
votes
2answers
98 views

What is the best practice for pairing objects of a sorted vector?

I have a vector that stores multiple instances of a type T: std::vector<T> vec; From time to time, vec is being sorted for different purposes. Now, I need a list of pairs of objects stored in ...
0
votes
0answers
103 views

What is a good data-structure to store intermediary results of map-reduce pipeline?

I also think about this question as about storing a symmetry group. A quick loose reminder of what symmetry group is: a set of "things" (say, rotations of triangle), and a binary operation (say, ...
-7
votes
1answer
314 views

Importance of storage in c++ [closed]

I want to know that whether storage is an issue with respect to current hardware and software market when we talk about the large size of C++ programs?? Because in certain conditions we have to choose ...
4
votes
4answers
1k views

Binary data formats, how to make ensure you can read different format versions?

On our project we have this data format that we use to process and record data on. As of late our application changed so that many of the data formats parameters have become obsolete. Currently we ...
-1
votes
1answer
492 views

What is the recommended way to storage documents as Microservice / API?

I'am new in Microservices, and this is the scenario. Background: I have an Web Application that requires store and retrieve files and let's say that i need to storage documents (PDFs, Word Documents,...
-3
votes
1answer
56 views

De-duplication procedure [closed]

I knew that by using De-duplication procedure we can save storage costs. 1) I know what is de-duplication but I don't know how it works. Can someone clearly explain how de-duplication procedure works?...
36
votes
8answers
12k views

Would having 4 states per “bit” rather than 2 mean twice the storage space?

Pretty straightforward fundamental, albeit naive, question: Would having 4 states per "bit" rather than 2 mean twice the storage space? In case that isn't clear, I mean as in if every "storage ...
9
votes
3answers
2k views

How are variables stored in a language compiler or interpreter?

Say we set a variable in Python. five = 5 Boom. What I'm wondering is, how is this stored? Does the compiler or interpreter just put it in a variable like so? varname = ["five"] varval = [5] If ...
4
votes
5answers
5k views

Is it ineffecient to store data as json in plain text? [duplicate]

We needed a persistent storage for API keys and I came up with reading and writing json in plain text and the user thinks it works but is it very ineffecient compared to a relational dbms? I just ...
31
votes
3answers
11k views

Does storing plain text data take up less space than storing the equivalent message in binary?

As a web developer I have very little understanding of binary data. If I take the sentence "Hello world.", convert it to binary, and store it as binary in an SQL database, it seems like the 1s and ...
1
vote
2answers
237 views

How do I best store a value with multiple keys?

I need to store two 32-bit numbers in a way that is quickly accessible and as storage space efficient as possible. My keys are a combination of the following two values: port: 0 - 29 vlan: 1 - 4095 ...
2
votes
0answers
113 views

Optimal format for storing key/value data optimized for quick key lookup

For a completely fun project, I want to write a Markov chain chat bot. The algorithm used is quite simple - break down incoming sentences into tokens, storing what words tend to come after each token ...
-1
votes
1answer
98 views

Complete statelessness, how to save password on client in secure way

Thinking about statelessness I wonder how I can overcome the issue of saving the user's password on the client side in a secure but also comfy way. I assume the client always sends credentials to the ...
1
vote
1answer
567 views

From web database (PostgreSQL) to mobile (SQLite)

My company employs people who add and edit records in a PostgreSQL database using a web interface. These updates then need to be compiled into a mobile app (Android, iOS) via SQLite and released as a ...
5
votes
5answers
481 views

Is saving disk space a valid reason to forgo migrating to a standard text format (e.g. JSON)?

A while ago I asked a question about custom text data formats, instead of using existing tools such as XML, JSON, YAML, etc. Now, in favor of converting our custom format to a relational database and ...
1
vote
0answers
73 views

Software Defined Storage (SDS): How to choose right moment to lock it for backing up?

I'm developing a server which provides a virtual block device via SCSI interface. The device is constantly bombarded by requests to read/write data blocks. Sometimes I need to lock this device (or ...
1
vote
1answer
6k views

Is cloud storage such as Dropbox SaaS, PaaS, IaaS or something else entirely? [closed]

I am stuck at how to classify Dropbox and other cloud storage services within the ISP (IaaS,PaaS,SaaS) model. At their core, such services provide a cloud-backed file system which can be read and ...
0
votes
2answers
169 views

Should MySQL database tables only have a single unit of information per row?

I run an automated data collection service at work, which records measurements (temperatures, voltages, etc) from about a dozen sensors at a rate of about once a second. The MySQL database table looks ...
81
votes
14answers
18k views

Why do most log files use plain text rather than a binary format?

Logging is something that is necessary but is (relatively) rarely used. As such it can be made much more compact in terms of storage. For example the data most commonly logged like ip, date, time and ...
2
votes
1answer
271 views

Storing/hosting sensitive business data

In order for users of my software to receive auto-updates, their copy of the software will need remote access to ~100mb of private files (manifest, jars) which means I need a server. Currently I don't ...
-2
votes
3answers
454 views

Upload file to S3 or save it locally?

I have a web application that allows users to upload files. My application does processing on these files (parsing, modifying metadata, etc). What approach should I follow? Should I save the files ...
0
votes
1answer
84 views

Design question to best option for storing data in file or DB

I am working on a feature where I need to store some data and I am not sure what should be my best option to store this data. I am sending data to some third party and want to show some part of that ...
1
vote
1answer
113 views

What is a simple way to store data for word count purposes?

I am writing a program to crawl the web and find frequently used words. What would be a good way to store the data? I was thinning of using an RDBMS with a single table (with columns 'word' and 'count'...
2
votes
2answers
180 views

Storing large binary data for integration tests

I have the following requirement for an application: Store large files (order of megabytes) which I will use as input for non regression test and some expected properties of the processing result ...
2
votes
2answers
103 views

When do I prefer a local DB over serialization for local data storage?

I recently started developing a small calendar application, in which I can add and view several calendar entries, which contain several data like name, date, alarm settings and so on. Those values ...
2
votes
1answer
120 views

Is it better to pull all data immediately or as needed?

I have a program that needs information from a database but it doesn't need all of the information at once, although it WILL need it at some point. Would it be better to pull all of the information ...
1
vote
1answer
861 views

How to store data from firebase to analyze at some later date [closed]

I am currently using Firebase with my iOS app. I use it to authenticate users and then allow user interactivity during the life of the app session. I would like to be able to save a snapshot of my ...
1
vote
1answer
50 views

File structure for storing pixels that changed from one image to another

Have two images with the same dimensions. I want to create a file that contains information on what pixels are different between the two images. The file should also tell me the value of the new ...
0
votes
0answers
51 views

Storing mass user-files

I am thinking about the concepts of a web-application in which users can upload files to the server. I have multiple questions about the storage of these files. Imagine a service with 10,000+ users ...
0
votes
1answer
29 views

How to store the current version of an instance? Store reference to specific instance?

I'm working on a project with has different checklists (questions and answers) associated with an entity (Protocol). There is a business requirement to have these questions be altered in the future ...
3
votes
3answers
196 views

Storing great amounts of data

I was brainstorming and I found quite an interesting topic I couldn't come up with a solution for myself. How does storing excessively large amounts of data work? I mean physically, no matter how big ...
23
votes
4answers
4k views

Is a memory of all possible permutations of a kilobyte block and pointers possible?

This is a hard enough idea to wrap my head around and I would greatly appreciate any edits/help to get it more readable for those in-the-know. Is it theoretically possible to have a hard drive that ...
3
votes
1answer
3k views

Where is the best place to store users' profile pictures?

I run a website that is hosted on Microsoft Azure and I'd like to store users' profile pictures. Which storage solution would be better ? I'm aware of a few ways to do so : Store images as blobs in ...
28
votes
2answers
26k views

PHP: when to use arrays, and when to use objects for mostly-data-storing code constructs?

PHP is a mixed paradigm language, allowing to use and return non-object data types, such as arrays. I pose a question to try to clarify some guidelines for selection of arrays vs objects when deciding ...
5
votes
2answers
209 views

Storing persistent data that only has a single instance but can change

I'm creating a website CMS in Rails (for fun and learning) and I'm trying to figure out how I'm going to store the data for an "about" page. Due to the nature of this page, the data will only have one ...
0
votes
1answer
393 views

Can I use GitHub as my mobile app's data store? [closed]

Given a relatively simple mobile app that consumes semi-static (infrequently updated by hand) data, is it a good idea to simply store said data in a public GitHub repo, in form of a structured ...
0
votes
0answers
55 views

Design for local text storage (70 000 words) in iOS

I would like to code an interactive story app for iOS devices. There will be a large amount of words locally stored.(70 000, 35 000 for each language as the text will be available in two languages). ...
0
votes
2answers
172 views

Is using up 'too much' storage space a practical concern when storing only text in a database in 2015? [closed]

I originally started writing a question on StackOverflow about a clever way to optimise keeping a version history of large text fields in a relational database table, possibly by using deltas instead ...
7
votes
2answers
215 views

How to distinguish whether a file is being saved or deleted? (“Save” in MS Word deletes then re-creates)

I maintain a tool that syncs files between a client computer and a document management server (using the CMIS protocol). Versions history (on the CMIS server) is important. When saving a document ...