37

Erlang and Ruby both come with functions for flattening arrays. It seems like such a simple and useful tool to add to a language. One could do this:

>>> mess = [[1, [2]], 3, [[[4, 5]], 6]]
>>> mess.flatten()
[1, 2, 3, 4, 5, 6]

Or even:

>>> import itertools
>>> mess = [[1, [2]], 3, [[[4, 5]], 6]]
>>> list(itertools.flatten(mess))
[1, 2, 3, 4, 5, 6]

Instead, in Python, one has to go through the trouble of writing a function for flattening arrays from scratch. This seems silly to me, flattening arrays is such a common thing to do. It's like having to write a custom function for concatenating two arrays.

I have Googled this fruitlessly, so I'm asking here; is there a particular reason why a mature language like Python 3, which comes with a hundred thousand various batteries included, doesn't provide a simple method of flattening arrays? Has the idea of including such a function been discussed and rejected at some point?

  • 2
    @detly: I happened to miss flattening lately when using several queries to retrieve data from different sources. Each query returns a list of dictionaries, so in the end I have a list of lists of dictionaries to be turned into a list of dictionaries. I used a loop + extend but flatten would have been much more elegant. However, I wounder if this pattern is common enough to justify having flatten in the standard library. – Giorgio Aug 24 '14 at 9:19
  • 4
    "I mean, imagine if you introduce a bug into your code that inadvertently changes the structure of your data. flatten will still work, but produce completely the wrong results.": This is one reason why I like statically-typed languages. ;-) – Giorgio Aug 24 '14 at 12:50
  • 4
  • 2
    @BryanOakley See prior comment as well (though not for multi-level lists, flattening in general is common) – Izkata Aug 29 '14 at 0:42
  • 3
    It is built-in in Mathemaica, and I use it extensively. – Per Alexandersson Jan 26 '15 at 22:06
33

Proposals for a flatten function to be added to the standard library appear from time to time on python-dev and python-ideas mailing lists. Python developers usually respond with the following points:

  1. A one-level flatten (turning an iterable of iterables into a single iterable) is a trivial one-line expression (x for y in z for x in y) and in any case is already in the standard library under the name itertools.chain.from_iterable.

  2. What are the use cases for a general-purpose multi-level flatten? Are these really compelling enough for the function to be added to the standard library?

  3. How would a general-purpose multi-level flatten decide when to flatten and when to leave alone? You might think that a rule like "flatten anything that supports the iterable interface" would work, but that would lead to an infinite loop for flatten('a').

See for example Raymond Hettinger:

It has been discussed ad nauseam on comp.lang.python. People seem to enjoy writing their own versions of flatten more than finding legitimate use cases that don't already have trivial solutions.

A general purpose flattener needs some way to be told what is atomic and what can be further subdivided. Also, it is not obvious how the algorithm should be extended to cover inputs with tree-like data structures with data at nodes as well as the leaves (preorder, postorder, inorder traversal, etc.)

  • Just to be explicit, this means that the one-level flatten function can be defined as lambda z: [x for y in z for x in y]. – Christopher Martin Apr 16 '15 at 9:24
  • 1
    "A general purpose flattener needs some way to be told what is atomic and what can be further subdivided.": This sounds like a problem that can be solved using OOP: each object could have a flatten method. The implementation of this method should recursively call flatten on its subcomponent, if the object is a composite. Unfortunately, AFAIK not every value is an object in Python. In Ruby it should work though. – Giorgio Sep 28 '15 at 11:54
  • 1
    a flatten helper for a one-level flatten rather than a continued "for in for in" is already a good enough case IMO. easily readable – dtc Jun 15 '16 at 22:01
  • 1
    @Giorgio Python shies away from such methods. Protocols are preferred, and I find they're vastly smoother to work with than an OOP design since you often don't even need to implement very much at all. – jpmc26 Dec 13 '17 at 22:58
8

It does come with such a method but it doesn't call it flatten. It's called "chain". It returns an iterator which you'd then need to use the list() function on to turn it back into a list. If you don't want to use a *, you can use the second "from_iterator" version. It works the same in Python 3. It will fail if the list input is not a list of lists.

[[1], [2, 3], [3, 4, 5]] #yes
[1, 2, [5, 6]] #no

There was at one time a flatten method in the compiler.ast module but this was deprecated in 2.6 and then removed in 3.0. Arbitrary depth recursion, necessary for arbitrarily nested lists does not function well with Python's conservative maximum recursion depth. The reasoning for compiler's removal were largely due to it being a mess. Compiler was turned into ast but flatten was left behind.

Arbitrary depth can be achieved with numpy's arrays and that library's flatten.

  • The chain.from_iterator function, as you said, can only be used to flatten two-dimensional lists. An actualy flatten function, which accepts any amounts of nested lists and returns a one-dimensional list, would still be massively useful in lots of cases (at least in my opinion) – Hubro Aug 24 '14 at 5:48
  • 1
    @Hubro: "in lots of cases" — can you name six? – Gareth Rees Sep 13 '14 at 11:24
  • 1
    @GarethRees: I gave a few examples here: programmers.stackexchange.com/questions/254279/… – Hubro Sep 14 '14 at 22:58
  • I would also go so far as to argue that if these other language do in fact provide such a feature to flatten a list in the very simple way described, that is one of the most compelling arguments in support of adding that simple ability to Python. – Bobort Sep 15 '16 at 20:50
  • Does it return an iterator or a generator? – jpmc26 Dec 13 '17 at 23:01
-2

... maybe because it's not that difficult to write one yourself

def flatten(l): return flatten(l[0]) + (flatten(l[1:]) if len(l) > 1 else []) if type(l) is list else [l]

... and then flatten all you want :)

>>> flatten([1,[2,3],4])
[1, 2, 3, 4]
>>> flatten([1, [2, 3], 4, [5, [6, {'name': 'some_name', 'age':30}, 7]], [8, 9, [10, [11, [12, [13, {'some', 'set'}, 14, [15, 'some_string'], 16], 17, 18], 19], 20], 21, 22, [23, 24], 25], 26, 27, 28, 29, 30])
[1, 2, 3, 4, 5, 6, {'age': 30, 'name': 'some_name'}, 7, 8, 9, 10, 11, 12, 13, set(['set', 'some']), 14, 15, 'some_string', 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]
>>> 
  • 7
    asker is aware of that: "in Python, one has to go through the trouble of writing a function for flattening arrays from scratch". This doesn't even attempt to address the question asked, "This seems silly to me, flattening arrays is such a common thing to do. It's like having to write a custom function for concatenating two arrays." – gnat Apr 8 '16 at 13:24
  • 1
    Out of topic... But super cool :-) !! – SeF Apr 12 '18 at 13:18
  • this answer is like telling OP he's not a good developer because he didn't know how to code the function himself. I suggest you modify the beginning of your answer because this is useful code for those who stumble upon the question, even if off-topic – Federico Bonelli Feb 13 at 12:14

protected by gnat Apr 8 '16 at 13:21

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.