Questions tagged [serialization]
Serialization is the process of converting an object into a stream of bytes in order to persist it to memory, a database, or a file. Its main purpose is to save the state of an object in order to be able to recreate it when needed.
139 questions
1
vote
3
answers
171
views
Code generation for multi-platform, multi-language library
I need to design a code generator system that produces serialisation/deserialisation libraries for multiple languages running on multiple platforms.
Hard Constraints
I need this to capture a pre-...
- 33
4
votes
3
answers
1k
views
Does it make sense to to abstract serialization/deserialization?
Serialization and deserialization in .NET applications can be achieved using several libraries, e.g.
Newtonsoft.Json
System.Text.Json
Assuming I'm developing a new application where I want to use ...
- 1,065
2
votes
2
answers
161
views
How to protect class invariants when using the extraction operator?
The C++ "standard" method for serializing and deserializing a data type is to use streams with the insertion (<<) and extraction (>>) operators. This has some flaws, but it does ...
- 789
3
votes
2
answers
214
views
Serializing key-values vs array of options
Is there a reason to encode options like this:
{
"options":[
{
"name":"opt1",
"value":"val1",
},
{
...
- 139
2
votes
1
answer
136
views
Encoding hierarchy information in objects
I am writing code that will assign entities to transaction. Each transaction has a counterparty entity.
Requirements
The entities should be (de-)serialisable (from pydantic models into JSON and vice ...
- 131
10
votes
5
answers
4k
views
How do you fix the wrong-case-sensitivity dictionary setting bug-pattern?
There is a coding anti-pattern I've noticed (while using .Net). You declare a data class, which is supposed to have a dictionary field (or get/set Property), and lets call it 'Properties', for the ...
- 389
17
votes
10
answers
7k
views
Should serialization and deserialization be "atomic" transactions?
I am wondering if serialization and deserialization of classes should always be treated as an "atomic transaction?"
What I mean is, if an error were to occur during the process of ...
- 789
1
vote
3
answers
215
views
Using source code instead of XML/JSON or other custom serialization schemes and binary file formats
For a while now I have been toying with the idea of using source code as a file storage format.
My question: How to support format version changes? (loading of older files with structural differences)
...
- 127
1
vote
1
answer
462
views
Should serializable classes have different serialization versions?
I am distributing a proprietary library that includes multiple serializable classes (serializable to a file). I want versioning to the serializable classes, i.e. when serializing I put the version ...
- 29
1
vote
2
answers
137
views
What do you call a process which transforms objects of complex types into simple objects of primitive types? [closed]
My first thought was that I'm "serializing" the complex object, but from what I understand that means I'm reducing it down to a string or binary format which could be passed over a network. ...
0
votes
2
answers
419
views
Deserialization and routing of objects
I know this question has to have been answered in countless ways before, but I would appreciate some help pointing me in the correct direction.
I am developing an embedded system that receives "...
- 789
1
vote
1
answer
165
views
Programs running under Slurm for > 24 hours are being killed by process management. What are good approaches in Python to work around this issue? [closed]
I often run data processing/machine learning/filesystem-scanning scripts that can take well over 24 hours to complete. For processes with arbitrarily low memory requirements, I can run without using ...
- 119
1
vote
1
answer
853
views
Memento pattern for Undo/Redo AND Save/Load with multiple objects
In a pretty big project, I have a hierarchy of objects, let's say:
Zone object, which contains Block objects and Connection objects.
Mainliy, the Connection object links together several Block objects,...
- 121
1
vote
1
answer
598
views
Dealing with business logic on DTOs using a DDD
I have a scenario in which an Entity defines an EntityRepresentation that is created from another end. The EntityRepresentation is a value object that implements some logic that should be performed on ...
- 131
1
vote
2
answers
2k
views
Efficient and correct way to store image data in Java? [closed]
To clarify, I'm having to develop a GUI application that stores information about clients (for a course) where one of the fields is the logo of the client. I'm currently using serialization to persist ...
- 149
1
vote
1
answer
127
views
Java, dealing with XML and JPA Annotated Classes
I use xjc to compile XSD files to Java Classes, and want to edit/extend them to make them persistable through JPA.
I cant figure out what the best "Coupling?" would be and how to organize it,...
- 13
0
votes
0
answers
92
views
Is it idiomatic to use protobufs as containers within a service?
I love gRPC, but I find every step of the protobuf process rather frustrating (particularly in Python). Even though they are structurally similar to data structures composed of lists and dicts, you ...
- 143
2
votes
0
answers
355
views
How to separate a serialization code from application in a large c++ project
I'm working on a fairly large c++ project which uses boost's serialization.
The issue that I have with the way it is currently organized is that serialization is weaved into the main source code on ...
- 129
2
votes
3
answers
264
views
How do you resolve a byte[] into a class instance in a way that doesn't couple the serialization/deserialization contexts together
Consider that you've got a POJO that you intend to serialize and send through a socket.
You can use whatever serialization strategy you wish (JSON, XML, protobuf, ..., etc) to serialize the actual ...
- 131
-3
votes
1
answer
78
views
How to exchange 1T data between multiple processes?
How to exchange 1T data between multiple processes?
Requirements:
Data < 1TB. Memory > 1T.
Multiple processes (different languages) on the same machine or different machines over network
Data ...
- 103
3
votes
3
answers
8k
views
Interfaces vs concrete types in DTOs and response classes
Had a discussion that i felt i needed some external input on. Normally always use interfaces instead of concrete types (on lists, injectable services and so forth). For example in lists i just pass ...
- 132
2
votes
0
answers
44
views
Web Service returning serialized helper objects
I had the idea to have a route in my REST API that returns serialized (PHP) helper objects. The client code then can use those objects to format some complex JSON structures for the next request. When ...
- 121
0
votes
2
answers
124
views
Serialization functions: accepting a sink versus returning the result
What are the pros/cons of having a serialization function accept a sink:
class Foo:
def toString(self, stringIOSink):
pass
def toBytes(self, byteSink):
pass
versus having the ...
- 11
0
votes
1
answer
49
views
portable hashable string representation of data object
I have a class of (flat) objects that are going to be passed around between three different parties, each possibly running a different software stack.
my_object:
item: "string_a"
money:
...
- 414
7
votes
2
answers
5k
views
Is there a canonical way to handle JSON data format changes?
Problem
Say we have a C# class with is serialized to JSON (currently via Newtonsoft's JSON.Net) and stored in a database:
public class User
{
public string authInfo;
}
If the class definition ...
- 269
10
votes
3
answers
36k
views
JSON without quotes for keys
I need a textual human readable format which is reasonably compact and version-control friendly to serialize a persistent memory heap. My Bismon system (GPLv3) has such a format (it is textual, human-...
- 32.7k
1
vote
1
answer
2k
views
Composite DTO or flattened DTO?
I'm having problem deciding which of these 2 structures to use from my DTOs:
Option 1 is:
@Data
class Lines{
private final Line[] lines;
}
@Data
class Line{
private final String[] specifiers;
...
- 367
2
votes
1
answer
281
views
Functional Approaches to Serializing Objects to Variable-length Byte Array Output
I have a large number of record types derived from a binary format specification. So far, I've already written a computation expression builder that let’s me read structures from the files easily:
...
- 145
1
vote
1
answer
256
views
Avoidance of Lost Update in DB2 for z/OS
I have a question concerning the "Uncommitted Read" Isolation Level under DB2 DBMS for z/OS.
In this article
https://www.ibm.com/support/knowledgecenter/en/SSEPEK_11.0.0/perf/src/tpc/...
- 13
1
vote
0
answers
251
views
Can I add new function for protobuf generated struct
I am using protobuf definitions to define message types.
Structure looks like
message1 {
}
message2 {
bytes msg1 = 1
}
So message2 has a field which is a byte array representation of message1. I ...
- 111
1
vote
1
answer
160
views
Dictionary Application: Suggestions for serialization format and data structure
I want to create a dictionary application.
One of the design guideline is performance in querying and showing up the definitions of words.
I would like some suggestions on two design choices:
Which ...
- 113
3
votes
2
answers
2k
views
Can I use the memento pattern with multiple relationships and persistence?
I need to implement a rollback system and the memento pattern gets mentioned regularly as a solution to this requirement, but in all cases a single object is used and it works in memory, persistence ...
- 2,256
6
votes
1
answer
6k
views
Do we really need SerializableAttribute?
I understand we need it to mark a class whose objects can be serialized, but do we strictly need this for behind the hood logic to work? We could just serialize the object using necessary methods. On ...
- 4,155
1
vote
0
answers
2k
views
Strategy to Deserialize Json of Unknown type
I find the following dilemma often. Suppose you're retrieving Json messages from a database, message bus, event store or whatever, and you need to deserialize each message into the proper type.
...
- 397
2
votes
1
answer
1k
views
Net Automatic Serialization vs Custom Serialization Difference
In C#, what is the difference between Automatic serialization and Customer Serialization? Also what is Reflection? I know this stuff is not simple, if someone can explain in lay man's term, that would ...
3
votes
3
answers
7k
views
Proper way to send custom data structures by sockets
I'm wondering what is the proper way to send structures as char array by sockets.
At this moment I have somethig like below but it isn't good solution because it causes undefined behavoiur. Is it ...
- 297
0
votes
1
answer
152
views
Common Design approach when using serialized class from XSD
I am creating a Parser assembly in C# to parse a XML structure. I have a XML schema defined for the XML files.
Using XSD tool I have created Class structures from XML Schema. The XML has various ...
- 111
2
votes
1
answer
488
views
One data structure to both parse and stringify
You can create a data structure such as a Parsing Expression Grammar (PEG) that will be used to parse:
string -> object
You can then write a function that iterates through the object's properties ...
- 2,661
3
votes
3
answers
589
views
How to separate serialization, visualization and computation from GUI editable code objects?
This might be an X Y problem, but here's my situation.
I've got a QT5 C++ code base whose task it is to allow for the configuration of a "process chain". A "process chain" here means a list of ...
- 1,310
0
votes
1
answer
95
views
How to serialze data that should be changed outside the program
Java provides methods for saving data permanently. An object that has to be saved has to implement the interface Serializable. But there is one problem with Java's object serialization: The objects ...
- 211
2
votes
2
answers
661
views
App/server communication - versioning, JSON vs. binary serialization
I apologize in advance for the length of this question; it requires some explanation. I'll try to make it as clear as possible.
I am developing and maintaining a mobile app (Xamarin, F#) that ...
- 329
1
vote
1
answer
68
views
efficient Communication in two Apps [closed]
I have two applications, first is like starting point and the main app is second one.
Whenever i have to do something, i have to login to the App1, provide some details to it, then App1 builds a ...
- 19
1
vote
0
answers
415
views
Caluclate needed heap size needed for serialization
In my current Java EE project I've got a quite common task: Load objects from database using Hibernate, transfer them in another object structure (based on a XSD) and serialize it using JAXB. To ...
- 173
0
votes
2
answers
241
views
Techniques for observing runtime state in C
What are the techniques for C programs which allow easily inspecting the state of the embedded device, through a communication link?
For example, for a microcontroller device which communicates with ...
- 265
2
votes
2
answers
277
views
Object cut in parts for technical reasons : Should it be modelized as 1 or 2 objects?
I'm making a game I have a Replay object that represents all the inputs done by a player to finish a level. It looks like that:
class Replay {
int replayId;
int playerId;
int levelId;
...
- 705
0
votes
1
answer
49
views
Ensuring no data is missed in serialization
I'm converting some code that formerly wrote structs directly into storage, to use serialization. This is because it is pretty annoying having piles of code to unpack structs that were laid out ...
- 135
5
votes
4
answers
2k
views
Do we have to write a wrapper function for few line code?
In the early time, maybe just few line code, But I have see too many repeated and messy codes involve serialize/deserialize interference the main logic. So I try to persuade the team members to wrap a ...
- 161
1
vote
2
answers
3k
views
Best way to invoke 'setter method' for first access and 'getter method' for the rest with "getter setter" pattern?
Here is a json which comes in the request param. I am constructing a class with getter and setter for accessing the values in json so that I could be able to pass the class object to different methods ...
- 366
1
vote
1
answer
96
views
Are there any cases when an abstraction should inherit ISerializable?
There is a @Mark Seemann's cite from a conversation which states that an abstraction must never implement IDisposable:
I like how @nblumhardt put it almost six years ago:
an interface [...] ...
2
votes
3
answers
3k
views
Can I serialize all responses from an API into one generic Response object in C#?
I want to consume the Tumblr API in C#. Every request to the API returns a JSON-encoded object with the same general outline:
{
"meta": {
"status": 200,
"msg": "OK"
},
"...
- 203