Questions tagged [serialization]

Serialization is the process of converting an object into a stream of bytes in order to persist it to memory, a database, or a file. Its main purpose is to save the state of an object in order to be able to recreate it when needed.

Filter by
Sorted by
Tagged with
1 vote
3 answers
188 views

Using source code instead of XML/JSON or other custom serialization schemes and binary file formats

For a while now I have been toying with the idea of using source code as a file storage format. My question: How to support format version changes? (loading of older files with structural differences) ...
0 votes
1 answer
145 views

Should serializable classes have different serialization versions?

I am distributing a proprietary library that includes multiple serializable classes (serializable to a file). I want versioning to the serializable classes, i.e. when serializing I put the version ...
  • 19
1 vote
2 answers
94 views

What do you call a process which transforms objects of complex types into simple objects of primitive types? [closed]

My first thought was that I'm "serializing" the complex object, but from what I understand that means I'm reducing it down to a string or binary format which could be passed over a network. ...
0 votes
2 answers
74 views

Deserialization and routing of objects

I know this question has to have been answered in countless ways before, but I would appreciate some help pointing me in the correct direction. I am developing an embedded system that receives "...
1 vote
1 answer
110 views

Programs running under Slurm for > 24 hours are being killed by process management. What are good approaches in Python to work around this issue? [closed]

I often run data processing/machine learning/filesystem-scanning scripts that can take well over 24 hours to complete. For processes with arbitrarily low memory requirements, I can run without using ...
1 vote
1 answer
372 views

Memento pattern for Undo/Redo AND Save/Load with multiple objects

In a pretty big project, I have a hierarchy of objects, let's say: Zone object, which contains Block objects and Connection objects. Mainliy, the Connection object links together several Block objects,...
  • 121
1 vote
1 answer
272 views

Dealing with business logic on DTOs using a DDD

I have a scenario in which an Entity defines an EntityRepresentation that is created from another end. The EntityRepresentation is a value object that implements some logic that should be performed on ...
1 vote
2 answers
649 views

Efficient and correct way to store image data in Java? [closed]

To clarify, I'm having to develop a GUI application that stores information about clients (for a course) where one of the fields is the logo of the client. I'm currently using serialization to persist ...
1 vote
1 answer
56 views

Java, dealing with XML and JPA Annotated Classes

I use xjc to compile XSD files to Java Classes, and want to edit/extend them to make them persistable through JPA. I cant figure out what the best "Coupling?" would be and how to organize it,...
0 votes
0 answers
68 views

Is it idiomatic to use protobufs as containers within a service?

I love gRPC, but I find every step of the protobuf process rather frustrating (particularly in Python). Even though they are structurally similar to data structures composed of lists and dicts, you ...
2 votes
0 answers
186 views

How to separate a serialization code from application in a large c++ project

I'm working on a fairly large c++ project which uses boost's serialization. The issue that I have with the way it is currently organized is that serialization is weaved into the main source code on ...
  • 129
2 votes
3 answers
138 views

How do you resolve a byte[] into a class instance in a way that doesn't couple the serialization/deserialization contexts together

Consider that you've got a POJO that you intend to serialize and send through a socket. You can use whatever serialization strategy you wish (JSON, XML, protobuf, ..., etc) to serialize the actual ...
  • 131
-3 votes
1 answer
76 views

How to exchange 1T data between multiple processes?

How to exchange 1T data between multiple processes? Requirements: Data < 1TB. Memory > 1T. Multiple processes (different languages) on the same machine or different machines over network Data ...
3 votes
3 answers
5k views

Interfaces vs concrete types in DTOs and response classes

Had a discussion that i felt i needed some external input on. Normally always use interfaces instead of concrete types (on lists, injectable services and so forth). For example in lists i just pass ...
  • 132
2 votes
0 answers
41 views

Web Service returning serialized helper objects

I had the idea to have a route in my REST API that returns serialized (PHP) helper objects. The client code then can use those objects to format some complex JSON structures for the next request. When ...
  • 121
0 votes
2 answers
83 views

Serialization functions: accepting a sink versus returning the result

What are the pros/cons of having a serialization function accept a sink: class Foo: def toString(self, stringIOSink): pass def toBytes(self, byteSink): pass versus having the ...
0 votes
1 answer
44 views

portable hashable string representation of data object

I have a class of (flat) objects that are going to be passed around between three different parties, each possibly running a different software stack. my_object: item: "string_a" money: ...
4 votes
2 answers
4k views

Is there a canonical way to handle JSON data format changes?

Problem Say we have a C# class with is serialized to JSON (currently via Newtonsoft's JSON.Net) and stored in a database: public class User { public string authInfo; } If the class definition ...
  • 229
8 votes
3 answers
27k views

JSON without quotes for keys

I need a textual human readable format which is reasonably compact and version-control friendly to serialize a persistent memory heap. My Bismon system (GPLv3) has such a format (it is textual, human-...
1 vote
1 answer
1k views

Composite DTO or flattened DTO?

I'm having problem deciding which of these 2 structures to use from my DTOs: Option 1 is: @Data class Lines{ private final Line[] lines; } @Data class Line{ private final String[] specifiers; ...
2 votes
1 answer
199 views

Functional Approaches to Serializing Objects to Variable-length Byte Array Output

I have a large number of record types derived from a binary format specification. So far, I've already written a computation expression builder that let’s me read structures from the files easily: ...
  • 145
1 vote
1 answer
180 views

Avoidance of Lost Update in DB2 for z/OS

I have a question concerning the "Uncommitted Read" Isolation Level under DB2 DBMS for z/OS. In this article https://www.ibm.com/support/knowledgecenter/en/SSEPEK_11.0.0/perf/src/tpc/...
1 vote
0 answers
148 views

Can I add new function for protobuf generated struct

I am using protobuf definitions to define message types. Structure looks like message1 { } message2 { bytes msg1 = 1 } So message2 has a field which is a byte array representation of message1. I ...
  • 111
1 vote
1 answer
98 views

Dictionary Application: Suggestions for serialization format and data structure

I want to create a dictionary application. One of the design guideline is performance in querying and showing up the definitions of words. I would like some suggestions on two design choices: Which ...
3 votes
2 answers
1k views

Can I use the memento pattern with multiple relationships and persistence?

I need to implement a rollback system and the memento pattern gets mentioned regularly as a solution to this requirement, but in all cases a single object is used and it works in memory, persistence ...
  • 2,126
5 votes
1 answer
5k views

Do we really need SerializableAttribute?

I understand we need it to mark a class whose objects can be serialized, but do we strictly need this for behind the hood logic to work? We could just serialize the object using necessary methods. On ...
1 vote
0 answers
2k views

Strategy to Deserialize Json of Unknown type

I find the following dilemma often. Suppose you're retrieving Json messages from a database, message bus, event store or whatever, and you need to deserialize each message into the proper type. ...
  • 339
2 votes
1 answer
980 views

Net Automatic Serialization vs Custom Serialization Difference

In C#, what is the difference between Automatic serialization and Customer Serialization? Also what is Reflection? I know this stuff is not simple, if someone can explain in lay man's term, that would ...
3 votes
3 answers
6k views

Proper way to send custom data structures by sockets

I'm wondering what is the proper way to send structures as char array by sockets. At this moment I have somethig like below but it isn't good solution because it causes undefined behavoiur. Is it ...
  • 275
0 votes
1 answer
80 views

Common Design approach when using serialized class from XSD

I am creating a Parser assembly in C# to parse a XML structure. I have a XML schema defined for the XML files. Using XSD tool I have created Class structures from XML Schema. The XML has various ...
2 votes
1 answer
369 views

One data structure to both parse and stringify

You can create a data structure such as a Parsing Expression Grammar (PEG) that will be used to parse: string -> object You can then write a function that iterates through the object's properties ...
  • 2,412
3 votes
3 answers
466 views

How to separate serialization, visualization and computation from GUI editable code objects?

This might be an X Y problem, but here's my situation. I've got a QT5 C++ code base whose task it is to allow for the configuration of a "process chain". A "process chain" here means a list of ...
  • 1,222
0 votes
1 answer
88 views

How to serialze data that should be changed outside the program

Java provides methods for saving data permanently. An object that has to be saved has to implement the interface Serializable. But there is one problem with Java's object serialization: The objects ...
2 votes
2 answers
510 views

App/server communication - versioning, JSON vs. binary serialization

I apologize in advance for the length of this question; it requires some explanation. I'll try to make it as clear as possible. I am developing and maintaining a mobile app (Xamarin, F#) that ...
  • 319
1 vote
1 answer
65 views

efficient Communication in two Apps [closed]

I have two applications, first is like starting point and the main app is second one. Whenever i have to do something, i have to login to the App1, provide some details to it, then App1 builds a ...
  • 19
1 vote
0 answers
334 views

Caluclate needed heap size needed for serialization

In my current Java EE project I've got a quite common task: Load objects from database using Hibernate, transfer them in another object structure (based on a XSD) and serialize it using JAXB. To ...
  • 153
0 votes
2 answers
178 views

Techniques for observing runtime state in C

What are the techniques for C programs which allow easily inspecting the state of the embedded device, through a communication link? For example, for a microcontroller device which communicates with ...
  • 265
2 votes
2 answers
269 views

Object cut in parts for technical reasons : Should it be modelized as 1 or 2 objects?

I'm making a game I have a Replay object that represents all the inputs done by a player to finish a level. It looks like that: class Replay { int replayId; int playerId; int levelId; ...
  • 705
0 votes
1 answer
47 views

Ensuring no data is missed in serialization

I'm converting some code that formerly wrote structs directly into storage, to use serialization. This is because it is pretty annoying having piles of code to unpack structs that were laid out ...
  • 125
5 votes
4 answers
2k views

Do we have to write a wrapper function for few line code?

In the early time, maybe just few line code, But I have see too many repeated and messy codes involve serialize/deserialize interference the main logic. So I try to persuade the team members to wrap a ...
  • 161
1 vote
2 answers
2k views

Best way to invoke 'setter method' for first access and 'getter method' for the rest with "getter setter" pattern?

Here is a json which comes in the request param. I am constructing a class with getter and setter for accessing the values in json so that I could be able to pass the class object to different methods ...
1 vote
1 answer
87 views

Are there any cases when an abstraction should inherit ISerializable?

There is a @Mark Seemann's cite from a conversation which states that an abstraction must never implement IDisposable: I like how @nblumhardt put it almost six years ago: an interface [...] ...
2 votes
3 answers
3k views

Can I serialize all responses from an API into one generic Response object in C#?

I want to consume the Tumblr API in C#. Every request to the API returns a JSON-encoded object with the same general outline: { "meta": { "status": 200, "msg": "OK" }, "...
  • 203
4 votes
2 answers
3k views

Store conditional expression in database

We have an application that allows users to enter conditionals in the form bound op x op bound2, we store this as a string, and then parse it at runtime to evaluate it. It is a decent amount of work, ...
  • 293
2 votes
1 answer
129 views

Writing a library for accessing persisted DBMS data - what should I pay attention to?

I'm working on this project which, for certain reasons, will need to read the data that's persisted by a DBMS (*). The data is not serialized objects with class information, representation of pointers ...
  • 2,447
4 votes
1 answer
3k views

Serialize Entity Framework Objects Without Lazy Loading

I have a very specific scenario that I'm not sure is possible to fully support: I have an ASP.NET application that uses Entity Framework 6.3 for it's data management. I also have a generic audit ...
0 votes
2 answers
87 views

Use XmlDeserializer in factory method?

I have the following class representing an XML configuration stored in the DB: using System.IO; using System.Xml; using System.Xml.Serialization; [XmlRoot("ModalConfiguration")] public class ...
5 votes
6 answers
833 views

Should custom deserialization happen in a constructor or in a static method?

When writing custom deserialization code, what's the better practice: a static method that creates an uninitialised object (e.g. using the default constructor) and then performs deserialization, or a ...
  • 232
1 vote
1 answer
74 views

Consistency for disk write in a model object graph

You're writing an app where the model (as in MVC) is an object graph. Occasionally, you want to save this object graph to disk. What are some best practices to ensure that the write is consistent? (...
  • 1,187
1 vote
2 answers
1k views

jersey post with a huge payload

I have a requirement to write a POST endpoint where the payload is huge(not normal). In the order 10s of Megabytes. And this endpoint will be accessed by a lot of clients (in the order of 1000s) ...