3

Consider a web service API that returns a complex Json object. Using the stock Python tools for the job, this will read in from the web service as a dict which contains, in turn, a mixture of arrays, dicts, strings, and numbers.

I confess: this data is born in Java. In Java, there's a data model for this data. My question is, would dyed-in-the-wool Python programmers desire a Python class data model that adds some notational niceness, or would they expect to just be handed the situation described in the first paragraph.

In code terms, what's the value of:

 text.tokens[3].part_of_speech

as opposed to:

 text['tokens'][3]['part_of_speech']

?

Based on some comments, I want to expand this a little. What I wrote above is all about consuming a complex data structure that arrives in Python as Json, and I can see that 'sugaring' it with a pile of classes with @property markers is not very helpful.

Another side of the coin is assembling one of these. As, admittedly, a person who has spent a lot more time in Java, C++, C (and Lisp?) than in Python, I would tend to see value in some API that helps me remember the names of all the bits and pieces. So maybe that suggests simply providing constants that provide the defined keys?

3

I don't know that there is one answer that everyone would agree with, but personally I would just use the built in Python classes. Reading JSON to python dictionaries and lists is a very standard thing to do, it maps very well.

I'm all and for OO programming, but I don't see the point in writing a tiny class to be a thin wrapper over existing powerful data types. When someone reads your code, it's just one more type to grok. Whereas standard data structures are, well, standard. They know it's a dict, if a programmer wants to understand the contents at some point, they can just place a breakpoint, print it out, and let the program keep running.

In addition, python has a lot of really nice syntax like dict and list comprehensions. You're probably losing as much niceness as you gain by not having to type a few extra []'s.

Representing JSON in C++ or Java is a different situation, because if the JSON structure is not known at compile time, so the types are not known in advance. So writing a custom data structure is pretty well necessary anyway. Very different in python.

7
  • Do documentation strings or IDE completions make any impression on you? A person looking to write code to reference some particular item down in the data model has to have a perfect memory or constantly refer to separate doc if things are just dicts. – bmargulies May 15 '15 at 0:51
  • First, if you know that the data will always have the same structure, you can possibly use named_tuples in some places. Second, to be honest you're still thinking like a Java guy. I'm not going to write a substantial chunk of program and use auto complete to get the first pass at a program working. Instead, I'm going to call your API from the REPL. I'll interactively inspect the dictionary in one window, and write code in another. – Nir Friedman May 15 '15 at 3:12
  • Except that not all the data will show up in a sample run; it varies by inputs. Anyhow, how would you get the json reader to produce named tuples? Final note; if the coder also has to write code to Create one of these, would some sort of builder feel out of place? – bmargulies May 15 '15 at 11:05
  • If the data is different between every run, then all the more reason to just use a dictionary. Python has keyword arguments, built in dicts, and the splat operator. If builder is this: javaworld.com/article/2074938/core-java/… then I can't imagine why you would ever, ever want to use this in Python. Anyhow, I'm done discussing this. You asked for a python programmer's perspective, and you're arguing. If you stop trying to write Java in Python, I'm sure you'll soon get a grasp for the idiomatic way to do things. – Nir Friedman May 15 '15 at 14:14
  • 1
    I didn't mean to argue. I'm grateful for your views. – bmargulies May 15 '15 at 14:34
1

I would use built-in lists for any lists (uniform variable-length sequences of objects).

I would use namedtuple-derived classes to store data with known fixed sets of named fields. It's like objects, but immutable.

So you'd have your text.tokens[3].part_of_speech == NOUN, but won't have text.tokens[3].part_of_speech = NOUN. Immutability is often useful at preventing a whole class of errors, though.

E.g.

from collection import namedtuple

Token = namedtuple('Token', ['part_of_speech', 'whatnot'])
Text = namedtuple('Text', ['tokens', 'metainfo', 'stats'])

# ...

json_doc = json.load(...)
tokens = []
for json_token in json_doc['tokens']:
  t = Token(json_token['part_of_speech'], json_token['whatnot'])
  # or even t = Token(**json_token)
  tokens.append(t)
...
return Text(tokens, ...)
3
  • So, let me go see how hard it is to make the json-reading package do this. – bmargulies May 15 '15 at 19:39
  • If your JSON schema is fixed, it's not hard. See update. – 9000 May 15 '15 at 20:24
  • The lack of assignment in namedtuple has always been too annoying for me to consider using it. Python needs const so badly. – Nir Friedman May 17 '15 at 23:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.