10

Consider the following:

def some_function():
    return 1

def some_generator():
    yield 1

In the code above, some_function is a function, while some_generator is a generator. They look quite similar.

The problem that I have when reading code is that I need to scan through every line in a "function" looking for the yield keyword before I can determine whether it is actually a function or a generator!

It just seems to me that using a different keyword for generators would make more sense, e.g.:

gen some_generator():
    yield 1

What are the merits of using the def keyword for both generators and functions? Why has a new keyword is not been introduced to separate functions and generators?

  • I don't know the real answer, but I often start out writing functions that return lists, then convert to generators when it seems right. Having the syntax match makes this more natural. – Steven Burnap Jan 28 '15 at 3:41
  • 2
    @StevenBurnap Derek's suggestion of using something like gen instead of def wouldn't make that transformation significantly more onerous. – jamesdlin Jan 28 '15 at 9:09
  • 1
    In general, language designers usually try to avoid adding unnecessary keywords. Every keyword they add is an identifier that programs aren't allowed to use for other purposes. – jamesdlin Jan 28 '15 at 9:13
  • @jamesdlin: But the question is arguing that a new keyword would be necessary. – Giorgio Jan 17 '16 at 8:54
  • 1
    @jamesdlin: Of course you can distinguish functions and generators by looking at their body, but the question argues that using different keywords would make the code more readable. – Giorgio Jan 17 '16 at 10:13
14

"What are the merits of using the def keyword for both generators and functions?"

While they are mechanically different, in practice when I use them they are often effectively the same to me conceptually (I don't think much of calling range() vs xrange()).

In terms of understanding what the function is all about quickly though, I agree something is lost with the use of def, but things should not be too obfuscated within the function to begin with.

Even an implicit return None can confuse the intended behavior of a function after a long bit of conditionals (as in, was a return None intended as a final behavior or an oversight in the logic). But those are just my believies about it.

I don't feel my argument is particularly convincing though so I will just defer to PEP 255:

Issue: Introduce another new keyword (say, "gen" or "generator") in place of "def", or otherwise alter the syntax, to distinguish generator-functions from non-generator functions.

Con: In practice (how you think about them), generators are functions, but with the twist that they're resumable. The mechanics of how they're set up is a comparatively minor technical issue, and introducing a new keyword would unhelpfully overemphasize the mechanics of how generators get started (a vital but tiny part of a generator's life).

Pro: In reality (how you think about them), generator-functions are actually factory functions that produce generator-iterators as if by magic. In this respect they're radically different from non-generator functions, acting more like a constructor than a function, so reusing "def" is at best confusing. A "yield" statement buried in the body is not enough warning that the semantics are so different.

BDFL: "def" it stays. No argument on either side is totally convincing, so I have consulted my language designer's intuition. It tells me that the syntax proposed in the PEP is exactly right - not too hot, not too cold. But, like the Oracle at Delphi in Greek mythology, it doesn't tell me why, so I don't have a rebuttal for the arguments against the PEP syntax. The best I can come up with (apart from agreeing with the rebuttals ... already made) is "FUD". If this had been part of the language from day one, I very much doubt it would have made Andrew Kuchling's "Python Warts" page.

  • 1
    It is worth noting that the new await (and async with and async for) syntax constructs added in PEP 492, which work very similar to yield from, require the function they're used in to be declared using async def instead of a just def. – Feuermurmel Jul 15 '15 at 13:02
2
  1. Adding new keywords risks breaking existing programs. Language designers generally try to avoid adding new keywords, especially for language features added after the language has already gained some popularity. Every keyword they add is an identifier that programs aren't allowed to use for other purposes, so adding a keyword could potentially break existing programs. Language designers have to weigh the benefits of a new keyword against the costs.

  2. I doubt that defining generators separate keyword would have much benefit. To understand whether a symbol corresponds to the name of a function or to the name of a generator is something that matters to callers, and callers shouldn't have to (and sometimes can't) look at what keyword was used to implement it. That's the responsibility of better naming conventions and documentation.

1

Generators are functions that evaluate lazily. Given that they are the same thing at base, it makes sense that they would use the same keyword. One option might be to use a comment to identify which is which for a given instance:

def some_function(): #This is a function.
    return 1

def some_generator(): #This is a generator.
    yield 1
0

I would guess it is because it is more Pythonic but what do I know? ;)

I don't really think that it matters all that much. It is easier to remember for me because no need to memorize them both.

EDIT: The PEP might say, you could investigate there.

  • Thanks! In fact, the PEP does say! See PEP-255 for the pros/cons and final decision regarding a new keyword for generators. – Derek Kwok Jan 28 '15 at 3:56
  • With Pythonic meaning: "The way it is done in Python"? – Giorgio Jan 17 '16 at 8:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.