2022 Developer Survey is open! Take survey.

Questions tagged [data]

Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of interconnected nodes).

Filter by
Sorted by
Tagged with
1 vote
0 answers
90 views

Best way to store key-value pairs of different types in c++?

I'm reading values from a (PE) binary file. The values have a known length and position in the file and are stored with no padding (right next to each other). They store various types (int, long, ...
user avatar
  • 111
0 votes
0 answers
28 views

What is a common practice to refer to other datas within a data oriented design?

This question has been in my mind for a while now, especially in the context of high performance, interactive 3d applications. Just want to find out what is the general practices in DoD for ...
user avatar
  • 169
1 vote
2 answers
245 views

Repository w/ MVVM Pattern: Data Format Check, Where/When, Best Practice?

TL; DR Where/When to check data format in MVVM + Repository Pattern is the best practice? Any suggestion or theory? The Scene Our team is working on an Android Project. We have a MVVM+repo structure: ...
user avatar
0 votes
3 answers
119 views

Can you suggest ways to assure client data confidentiality in a SaaS product managed by yourself?

I am designing a multitenant SaaS platform where the database will hold information for different clients. The application consists of an Angular frontend and a REST API Backend. Some of the clients ...
user avatar
-3 votes
1 answer
20 views

Adding rows and columns simultaneously to pandas dataframe

I am new to Python (~ 1 month) and am having a hard time figuring out a clean, fast way to simultaneously append rows and columns to a dataframe automatically. As a representative example, I will ...
user avatar
0 votes
2 answers
161 views

Share data between users without a database, php [closed]

How would I create for example a live chat, where you "post" a piece of text and it is displayed to other users in a "chat room", without the need of permanently storing it. My ...
user avatar
  • 23
0 votes
1 answer
26 views

Scalable Web Data Service Approach

I'm looking to build a service that interacts with an existing web application's database. The service would retrieve data from an api for which the web application users own an account and warehouse/...
user avatar
  • 1
0 votes
1 answer
53 views

Data Class: spread boilerplate for each different database over multiple class copies, or add complexity to original class?

I'm creating a prototype mobile app that will store data locally in an sqLite database and also on the cloud (CloudKit for now). The cloud storage uses a certain way of storing the data, and sqlite ...
user avatar
  • 1,477
0 votes
1 answer
47 views

Integration with stripe, have a single source of truth or have info of the user status in our DB

So We're integrating our application with stripe and we have different products There are 2 options : Have stripe as a single source of truth, anytime we need to check the permissions of a user we ...
user avatar
  • 123
0 votes
1 answer
94 views

SQL Foreign Key Relationships and External Data

I work for a company that wants an external system built outside of their core system. I can't share an SQL database, and can't store data in their database. So I am forced to maintain my own SQL ...
user avatar
1 vote
1 answer
35 views

Correct/specific term for renaming/restructuring objects

I am developing an app whose main purpose is to display data in tables, lists and graphs. As such, sometimes I need to rename or restructure some of the data recieved from the backend into a "...
user avatar
0 votes
0 answers
9 views

Looker / Visualisation Bi Tool Model Management

I am faced with a situation where we are now creating a new data team in my company, and this team is providing a data Platform for other teams in the company to use. Basically a data warehouse and ...
user avatar
0 votes
1 answer
59 views

Should you expose API endpoints on an application that is under heavy load or delegate it to another application?

Say I have an application/service A. This application will continuously take in some input X, process it and store it in some database T. This process is quite resource intensive. Now you also wish, ...
user avatar
0 votes
2 answers
298 views

Duplication of data vs loose coupling

Currently, I'm working with a health management application, let's call it application A. It's a partly prebuilt generic application that we are extending on. The main entities are journals, events, ...
user avatar
  • 111
-1 votes
1 answer
82 views

How to optimize the request time for large data response?

I have created a dashboard for rendering a list of clients into a DataTable. Below shown is the data model structure: When I had a few records in the clients schema, let's say a thousand rows, the ...
user avatar
  • 109
0 votes
2 answers
117 views

Data structure for graphs and vectors in C

I'd like to design an implementation for graphs in C. I'm wondering what the most efficient approach in terms of both computational power and memory consumption would be. I've looked at various open-...
user avatar
-1 votes
1 answer
34 views

Time limit or built-in flush of HashTable entries

Is there a standard data type which is a struct or a table or a dictionary which will always delete objects that are older than t minutes? for example if I want to make sessions expire after 60 ...
user avatar
1 vote
4 answers
151 views

Thinking about data as object vs table row [closed]

It often happens that we have a set of structured data. Let's say our data is about charitable organizations. I could turn it into a list/set of objects/structs: struct Charity { let name: String ...
user avatar
0 votes
3 answers
519 views

Can more than one memory address be accessed at one time?

Apologies if this isn't the correct place to ask the question. But, I'm stumped. Can multiple memory addresses be read/written at one time, or must one address be read then another, is it possible for ...
user avatar
  • 184
5 votes
2 answers
455 views

What happens to the rest of memory address space if the data is less than 1 byte?

Sorry if this sounds like a stupid question I’m very new to this. So my question is; within modern computers memory address space is a byte long, so what would happen to the rest of the address space ...
user avatar
  • 184
1 vote
2 answers
86 views

Using objects to construct nearly the same objects

I am getting worried I have some terrible design pattern here in JS/Node.js/MongoDB. It seems to that to create an object, I need an object that looks like that object. This includes data that comes ...
user avatar
0 votes
0 answers
45 views

How should data be formatted between embedded device and app/cloud?

We have an embedded device collecting data from various sensors attached to it. The data is then passed to the application. What is the best way to design the data packets? The data needs to be ...
user avatar
1 vote
3 answers
173 views

Is it a good practice to isolate the database from public APIs?

We develop web-applications using a javascript frontend framework (ReactJS), to retrieve the data to show, the user's browser calls an authenticated REST API which is our backend (Kotlin+Spring), the ...
user avatar
  • 119
0 votes
1 answer
162 views

Python: Function pipeline with multiple return/input values, or use OOP? Best Practices?

I have a 'processing' function and a 'serializing' function. Currently the processor returns 4 different types of data structures to be serialized in different ways. Looking for the best practise on ...
user avatar
1 vote
0 answers
75 views

How to (simply) architecture a way to ingest multiple types of large files, process them, and send data in chunks to web services?

Note: All of this would be in AWS Hi everyone, What would you guys suggest for building something that: Takes in several different input file types (ex: csv, json, jsonl, xml, .gz, ...) That can be ...
user avatar
5 votes
4 answers
173 views

Static data - database or code

I work on small to medium, database driven, line of business applications. What I usually find, especially in older systems, is most of the data lives in the database. By data I mean stuff like a ...
user avatar
  • 75
0 votes
2 answers
84 views

Appropriate storage to store all keyboard and mouse events on my computer?

I'd like to build detail analytics of my computer usage to try and detect patterns and improvement opportunities. So I'm building a software to record every mouse click, keyboard event and window ...
user avatar
  • 249
-3 votes
1 answer
540 views

Difference between read and write access (data, IoT)

i am a lawyer and currently writing an essay about data access in ioT environments. I often stumble over the distinction between read access and write access. Could someone provide me with an example, ...
user avatar
-3 votes
2 answers
706 views

Sending JSON in body of an email

Basically, in our scenario, data has to be sent from a WordPress form to Dynamics 365. My current approach is putting it into an email body in JSON format, that is then extracted and further processed ...
user avatar
0 votes
2 answers
116 views

How can I associate algorithm-specific data to the objects it works with, efficiently yet cleanly?

I'm writing in C++, but this problem applies to most non-high-level languages, and possibly some high-level ones as well. I have a graph of heterogeneous nodes. The graph can be instantiated by ...
user avatar
  • 233
5 votes
12 answers
1k views

Why do or should programmers save data in text based formats like JSON or XML instead of binary?

I see a lot of reasons to use binary over text-based formats. With binary, I find it a lot easier. I can use fread(data, sizeof(struct DataStruct), 1, fileptr) to read, or fwrite to write data. I ...
user avatar
-2 votes
2 answers
267 views

Where to put data generating functions for unit tests

I am writing a unit-test framework (for Matlab/Octave, but that does not matter) which also supports property based testing. Thus, I need some functions which generate random data which are tailored ...
user avatar
  • 115
-1 votes
1 answer
167 views

Best practices for data manipulation to use in admin dashboard application?

I am trying to create admin dashboard to track sales and inventory. Using React front end and Django rest backend API. Right now I am pulling sales and would like to make some charts. What's the ...
user avatar
-7 votes
1 answer
78 views

How would one 'falsify' data collected of you by companies using software? [closed]

Since we live in a world where basically all your web activity is being tracked by some company or your ISP. I was wondering how to 'falsify' the data they collect. For instance by having software go ...
user avatar
0 votes
1 answer
53 views

IOT data interpretation

This is a more architectural question and I am hoping to get more ideas out of this thread. I have a computer vision model detecting daily life activities and sending data to the server every second. ...
user avatar
0 votes
1 answer
117 views

how to store the data for function f(time,length)

I have a function which has discrete time step ,discrete length from origin . These two give height of the function. For example at time 1.2sec , and length 5.5cm from origin height is 10cm. The step ...
user avatar
1 vote
4 answers
298 views

System Design: Efficient way to import hundreds of different CSV formats

OVERVIEW We have multiple data providers who provide us information from a large set of data sources. These end data sources provide data in CSV format. Since, the CSV formats are not consistent ...
user avatar
  • 345
1 vote
1 answer
104 views

An effective way of finding regressions

There is a rather large data flow engine - more than 2000 different flow definitions of "what to do with inbound data". The engine deals with various data formats (flat-file, CSV, JSON, XML, ...
user avatar
  • 211
3 votes
2 answers
233 views

How to ensure data consistency between 2 microservices while both having write permissions

I'm currently building my first ever microservice using NodeJS, but I stumbled upon a problem. TL;DR ~ How to ensure data consistency between 2 microservices while both having write permissions Setup ...
user avatar
  • 163
0 votes
2 answers
1k views

Terminology: Classes that contain only methods [closed]

I am studying the programming language Kotlin, and I just came across the idea of a Data Class. I have a background in Java programming where classes can have fields and methods. I have heard ...
user avatar
  • 41
1 vote
1 answer
99 views

When modules written in different languages interact, is exchanged data always serialized?

I have module written in Python and I would like to write function in e.g. Go but problem is that data needed for computations is often more than 100MB. My question is: is data always serialized and ...
user avatar
  • 127
-5 votes
1 answer
41 views

How is data stored prior to downloading?

I go to several sites that offer their data in XML, JSON, CSV formats. As shown here, https://www.space.org/basicspacedata/query/class/boxscore/format/csv, I choose the format. Question is how is ...
user avatar
0 votes
2 answers
134 views

How to minimize size of data given the need for a null pointer?

So I don't know how to formulate this clearly, as it's quite confusing. Hence the reason for asking this question, to overcome the confusion. I will do my best to make it clear, but part of the ...
user avatar
  • 2,412
-4 votes
1 answer
40 views

What to call a system that gives users customized access to data?

What would be the proper way to describe a system whose sole purpose it is to give users access to large amounts of data in a structured way according to their own criteria? Like say for example ...
user avatar
1 vote
0 answers
31 views

Rolling Analytics - Dates and Dropoffs

I would like to generate analytics on a per-user basis on how many times they've viewed a particular page for 7 days, and how many times they've viewed a page during their lifetime as a user. This ...
user avatar
  • 195
1 vote
2 answers
106 views

How important is the data to assess code coverage?

per the title how important is the data in assessing code coverage? As background, let's say that you're given 20% of the entire dataset to help trace the different pathways each row of data goes ...
user avatar
2 votes
2 answers
168 views

How to represent a data model?

Often enough, the terms data model and data format are used interchangeably, but here I disagree. Let's start with the simpler one, the data format. I don't know exact definitions, but the data format ...
user avatar
  • 237
2 votes
2 answers
1k views

Storing unchangeable data in database vs in code

Hi I am trying to create a role based dynamic route authorization system on my NodeJS ExpressJS powered API server. Scenario: There will be some roles like Admin, StandardUser etc. and they can't be ...
user avatar
  • 223
-3 votes
1 answer
73 views

Can TLV theoretically be used for browsers?

Compared with HTML (Hypertext Markup Language), TLV (Tag-Length-Value) also can be used for a data representation. If TLV is designed smart enough, it can represent hierarchies and trees with no ...
user avatar
3 votes
2 answers
97 views

How to synchronize changes between Data Engineers and Designers?

I'm processing data and writing in a database, that is used by my colleague, a designer, working on data viz. How can I efficiently, say, add a new column for my colleague to use in their ...
user avatar

1
2 3 4 5
9