How to represent a data model?

Question

Often enough, the terms data model and data format are used interchangeably, but here I disagree. Let's start with the simpler one, the data format. I don't know exact definitions, but the data format describes the layout of a piece of data, the meaning of the individual bytes. There are many formats available, more general ones being for example JSON and XML. However, one can represent the same piece of data in different formats. Short example - a location.

JSON

{
    "location": {
        "longitude": 41.25,
        "latitide": -120.9762
    }
}

And XML

<location>
    <longitude>41.25</longitude>
    <latitude>-120.9762</latitude>
</location>

So, the basis for those two is a common data model, which states that a location is comprised of two fields, long and lat, and both those fields are floating point values.

Now my question is, how can I formally write down such a data model, preferrably in a common format that is machine readable (preferrably - a standardized diagram is already better than a sheet of paper)?

XSD for XML and JSON Schema for JSON are mappings of a data model onto a specific data format. What I was hoping to find is one level higher, providing an abstraction of the data model to be consumed my multiple, different data formats. I don't design the application, I design the data model. If application A consumes a piece of data from a web service, JSON might be a good choice for a data format here. But an application B, which works with the same data, but only over the wire, might prefer to use a binary format. I hope it gets clear what kind of abstraction I want - not to model data for a specific format, but for all formats.

Answer 1

Data format

A data format defines a specific syntax to be used to store or exchange data. This syntax is usually defined with the help of a grammar. Some examples:

• Json syntax is defined using a railroad syntax diagram (left side of the official website) and an indented variant of a BNF like grammar rules (right side of the official web-site).
• XML syntax is defined using a rigorous BNF grammar (the terminals are defined using regular expressions).
• CSV syntax is defined by the IETF both with a textual description and with a formal ABNF grammar, yet another variant of BNF.
• In telecom industry, many data formats are themselves defined using the ASN.1 syntax.

As you rightly pointed out, the same data can be represented in different formats.

Data models

The data model defines the semantic of the data independently of its representation. But semantic has many levels:

• A first level is to identify data that belongs together and has together a meaning. For example 27, 2, 2020 are just three numbers, but if taken together it can represent a date, the 27 February 2020

• A second level is to identify tight relations between such groups of data, which can be mapped to concepts in the real world A person can for example be defined by a name, a date of birth, and a place of birth.

• A third level is to identify more relationships for example that a person may attend a course, and a course may be attended by several persons. Relationships can be annotated to express what the relationship means.

• A next level is to identify generalisation or specialisation relations between concepts.

The data modeling can be abstract and completely independent of the data representation. Typically, you would use and Entity-Relationship (ER) model or UML models (class diagrams). The first is focused on data and is heavily used in the area of RDBMS. The second is used in object oriented design and allows to address not only data but also associated behavior.

The important aspect in a data model is also to map the data to its meaning. This is done using a dictionary that defines each entity or class, the attributes and the relations.

The data schema

A schema such as XSD or JSON-Schema are in-between data format and data modelling:

• A schema defines valid combinations of elements and value constraints. It therewith extends the general XML or JSON format to make it specific to some kind of data.
• But at the same time, defining the elements that belong together de facto identifies the level one and two of our data modeling: it defines relevant data types, that could be considered independently of the representation.

Personally, I tend to see schemas more as a data format definition, since one of their main use is to have a validation of some data. Furthermore, it often happens that in a schema you would put together interrelated data, which could be modelled as distinct entities with a strong relationship and that are only put together tactically in the data format. But I understand that a schema could be considered as modeling tool, since it nevertheless allows also to work with/define/document data structures.

Machine readable models

UML is not machine readable. ER neither. In fact, the mainstream modeling languages are graphical. They could be encoded using a machine readable format, but there is no standard.

So if you want to use a machine readable model, the schemas seem to be the best option. For very strurctured data SQL is of course an alternative of choice, and many tools exist that reverse engineer SQL to draw a model.

Answer 2

There is no absolutely universal way to describe a specific data model, which is completely platform and system independent.

From a very high point of view, the UML could be used, which is a very common language to describe not only building pieces of software but also other architectural parts, however UML lacks your requirement of being machine readable, as it's mostly targeted towards human interpretation.

Mostly, when you architecture a system and your requirement is to document available data models, it's your responsibility to define the way and format how those data models are presented, along with defining the interchangeable data format used for data exchange.

Should you choose JSON or XML, you will need some form of formal documentation which tells other developers what to expect. In case of JSON, you could use JSON Schema, in case of XML, you could use WSDL. Should you be building a public API, regardless of the exchange format used, you could use a platform such as Apiary to provide the description of available operations, along their input and output data models.

The used approach depends on the kind of project you're building, and the chosen data exchange format.