2

I've read both visitor&strategy patterns are a good choice when it comes to export data into multiple file formats. I'd like to understand how to apply both so I've created a really simple dummy snippet :

class SubWidget:
    def __init__(self, parent):
        self.parent = parent


class SubWidget1(SubWidget):
    def __init__(self, parent, a):
        super().__init__(parent)
        self.a = a

    def restore_state(self, state):
        self.a = state["a"]

    def save_state(self):
        return {"a": self.a}


class SubWidget2(SubWidget):
    def __init__(self, parent, a, b):
        super().__init__(parent)
        self.a = a
        self.b = b

    def restore_state(self, state):
        self.a = state["a"]
        self.b = state["b"]

    def save_state(self):
        return {"a": self.a, "b": self.b}


class Element:
    def __init__(self, a, b):
        self.a = a
        self.b = b

    @classmethod
    def from_state(cls, state):
        return cls(state["a"], state["b"])

    def save_state(self):
        return {"a": self.a, "b": self.b}


class Widget:
    def __init__(self):
        self.subwidgets = {
            "subwidget1": SubWidget1(self, 1),
            "subwidget2": SubWidget2(self, 2, 3),
        }
        self.elements = [Element(4, 5)]

    def save_state(self):
        return {
            "subwidgets": {k: v.save_state() for k, v in self.subwidgets.items()},
            "elements": [v.save_state() for v in self.elements],
        }

    def restore_state(self, state):
        for k, substate in state["subwidgets"].items():
            self.subwidgets[k].restore_state(substate)
        self.elements = [Element.from_state(substate) for substate in state["elements"]]


if __name__ == "__main__":
    w = Widget()
    st = w.save_state()
    w2 = Widget()
    w2.restore_state(st)

Let's assume both the number of subwidgets classes as their internal represantion may grow up and change over time. Let's also say that'd be desirable to be able to export the Elements object in different file formats.

Which design patterns would you use on this particular simple example? How can you grant your design will support different file versions over time with no sweat? Is it an antipattern having save_state/restore_state methods or from_state static class method per class?

I wanted the question to be language agnostic but I thought by providing a simple dummy python code could enrich the question content a bit. In any case, in the real world case we'd be talking about a complex widget that will have some dynamically loaded (plugins) subwidgets... as analogy think of tools such as photoshop, 3dsmax, ... on these tools you'll see multiple subtools (SubWidget) and internal model/data (elements). I know the code is an oversimplification but the goal is to know which are the right design patterns to apply on this typical case and how to apply them.

3 Answers 3

3

I think the save_state and restore_state methods are pretty good, since they prevent other objects from tearing the object apart, which would violate encapsulation and is used by the python persistence module (__getstate__ and __setstate__ are used by pickle) The problem is that the state itself has to be serialized, which is solved very nicely by the serde library in rust.

There, every object which can be serialized acts as a director and every file format as a builder which builds the serialized representation as directed by the object.

To unserialize, you could simply reverse the roles and let the data direct how to rebuild the object.

This gives the object the ability to change its layout because you just need to modify the method responsible for directing the serialization process and the builder responsible for deserialization.

To support different versions, the serialized representation could contain a version number which is used by the builder to handle old representations.

Furthermore, the file format can change too because the object and its builder work with primitives like Integers, Strings and Sequences which are handled by the file format, allowing it to change without the object and its builder noticing.

Additionally, you could use this to convert file formats easily by using the target format as the builder and the current format as the director.

1
  • Thanks for your contribution, I'll take a deep look into the approach used by serde library and check if it handles file versioning gracefully
    – BPL
    Feb 4, 2021 at 15:39
1

The example you give with save_state() and restore_state() requires that the state already exists in order to restore it - essentially a skeletal object graph must exist in some form in order to populate it. This may only require a root node, but in a complex object hierarchy, how could you know what type of node to create as the root?

You might benefit from the serialization here, it has its own issues; see this other question for another perspective.

Consider the separation of your data from the user-interface used to manipulate that data. There are many patterns that advocate the clear delineation of user-data and code to manipulate it (MV*, CQRS, React, Redux, etc).

I think your question is more about Export ("...export data into multiple file formats...") in which case, are you expecting to re-read back the file?

  • If no, consider having your native file format as rich as possible so that you can transform it in whatever way your product/customers need. This would likely be determine by the language, tools or libraries that you need to work with. If possible, humanly-readable will save you hours/days/weeks of debugging pain. Once you have this format nailed down, export becomes a Transformation operation, for which Visitor pattern is fine. See also Tell, Don't Ask.
  • If yes, you'll need to handle the fact that some file formats are 'lossy' compared to the format you're storing - consider the case of a rendered image representation of an 3D scene - clearly that one isn't going to 'import' properly without serious AI/ML work. Even JSON could be lossy.
1
  • Thanks for your answer, really nice valuable links/info you've provided here, specially that one to Fowler's site, which it's priceless :) . One of the main flaws of the dummy snippet I've posted is about versioning... think these plugins/tools will evolve but you still want them to have backwards compatibility so I need to think a proper way to handle that fact. Also... how to enforce plugin developers to support old files?
    – BPL
    Feb 4, 2021 at 15:37
0

What sort of file Format?

  • Hierarchical?
  • Block Based?
  • Record Based?
  • Relational?

And what sort of data-structure do you have? I'm presuming hierarchical based on your example.

And by multiple format, are you simply meaning JSON vs YAML? Both hierarchical formats.

Or are you talking: proprietary format A, a block based format like PNG, and JSON?

Supporting what essentially amounts to a different syntax (JSON vs YAML) is many times easier than supporting file formats which cannot natively express the data structure being stored.


Transformers

The act of saving or loading one representation (in memory) from another representation (in file) is an act of transformation. We have very good tools that deal with this problem every day. They are called Compilers.

Now compilers can be as simple as reading one representation, and writing out in another representation. The simpler transformation can certainly be done this way: like YAML <-> JSON and Assembly <-> Machine Code. Though it may not be the best way, or even pretty.

Most modern compilers avoid this limitation by being a clan of co-operating compilers. This clan is roughly broken down into three families (and maybe a few black sheep).

  • Front ends: Are compilers often called Parsers. They understand a given language and knows how to create an intermediate representation.
  • Back ends: Are compilers, and are usually just called that, but Writers might be a better term. They understand the intermediate representation and know how to write that down in another representation.
  • Middleware: Are compilers, often just called transformations though. They just understand the intermediate representation and modify it in various ways, like removing duplicates. There might be several families of middleware for different intermediate representations.

Visiting

A Parser wants to translate your object into another representation. The problem is that there is no good mechanism for this parser to read your object. The solution is to invert the responsibility, the parser inverts itself into a Writer and passes that into your object. Its now your objects responsibility to explain itself using that writer API. Which it just might not be able to do.

Now that visitor could have compiled direct to your desired output language. But as you want to trivially support multiple formats it makes more sense that it translated to an intermediate representation. Then there are a number of compilers that can take that intermediate representation as either a stream, or DOM (think java SAX an XML DOM), and convert that into some file on disc.


Loading

The reverse problem is addressed in the same way. A Parser reads the file, and this time because of a good API it can read the file directly. It creates a translation of the file in memory in some intermediate representation. That representation now serves to drive a compiler to regenerate the object. Only this time the compiler is called a builder.

There are two ways this builder can be made.

  • To be given the intermediate representation and reconstitute the object/s however it wants
  • To be streamed the representation as a series of events/calls on the interface of the builder, and somehow it caches and reworks the data back into the object/s.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.