Complex data structure in python: just dict, etc, or some classes?

Question

Consider a web service API that returns a complex Json object. Using the stock Python tools for the job, this will read in from the web service as a dict which contains, in turn, a mixture of arrays, dicts, strings, and numbers.

I confess: this data is born in Java. In Java, there's a data model for this data. My question is, would dyed-in-the-wool Python programmers desire a Python class data model that adds some notational niceness, or would they expect to just be handed the situation described in the first paragraph.

In code terms, what's the value of:

 text.tokens[3].part_of_speech

as opposed to:

 text['tokens'][3]['part_of_speech']

?

Based on some comments, I want to expand this a little. What I wrote above is all about consuming a complex data structure that arrives in Python as Json, and I can see that 'sugaring' it with a pile of classes with @property markers is not very helpful.

Another side of the coin is assembling one of these. As, admittedly, a person who has spent a lot more time in Java, C++, C (and Lisp?) than in Python, I would tend to see value in some API that helps me remember the names of all the bits and pieces. So maybe that suggests simply providing constants that provide the defined keys?

Answer 1

I don't know that there is one answer that everyone would agree with, but personally I would just use the built in Python classes. Reading JSON to python dictionaries and lists is a very standard thing to do, it maps very well.

I'm all and for OO programming, but I don't see the point in writing a tiny class to be a thin wrapper over existing powerful data types. When someone reads your code, it's just one more type to grok. Whereas standard data structures are, well, standard. They know it's a dict, if a programmer wants to understand the contents at some point, they can just place a breakpoint, print it out, and let the program keep running.

In addition, python has a lot of really nice syntax like dict and list comprehensions. You're probably losing as much niceness as you gain by not having to type a few extra []'s.

Representing JSON in C++ or Java is a different situation, because if the JSON structure is not known at compile time, so the types are not known in advance. So writing a custom data structure is pretty well necessary anyway. Very different in python.

Answer 2

I would use built-in lists for any lists (uniform variable-length sequences of objects).

I would use namedtuple-derived classes to store data with known fixed sets of named fields. It's like objects, but immutable.

So you'd have your text.tokens[3].part_of_speech == NOUN, but won't have text.tokens[3].part_of_speech = NOUN. Immutability is often useful at preventing a whole class of errors, though.

E.g.

from collection import namedtuple

Token = namedtuple('Token', ['part_of_speech', 'whatnot'])
Text = namedtuple('Text', ['tokens', 'metainfo', 'stats'])

# ...

json_doc = json.load(...)
tokens = []
for json_token in json_doc['tokens']:
  t = Token(json_token['part_of_speech'], json_token['whatnot'])
  # or even t = Token(**json_token)
  tokens.append(t)
...
return Text(tokens, ...)