Using lookahead assertions in regular expressions

Question

I use regular expressions on a daily basis, as my daily work is 90% in Perl (legacy codebase, but that's a different issue). Despite this, I still find lookahead and lookbehind to be terribly confusing and often unreadable. Right now, if I were to get a code review with a lookahead or lookbehind, I would immediately send it back to see if the problem can be solved by using multiple regular expressions or a different approach. The following are the main reasons I tend not to like them:

• They can be terribly unreadable. Lookahead assertions, for example, start from the beginning of the string no matter where they are placed. That, among other things, can cause some very "interesting" and non-obvious behaviors.
• It used to be the case that many languages didn't support lookahead/lookbehind (or supported them as "experimental features"). This isn't the case quite as much, but there's still always the question as to how well it's supported.
• Quite frankly, they feel like a dirty hack. Regexps often already are, but they can also be quite elegant, and have gained widespread acceptance.
• I've gotten by without any need for them at all... sometimes I think that they're extraneous.

Now, I'll freely admit that especially the last two reasons aren't really good ones, but I felt that I should enumerate what goes through my mind when I see one. I'm more than willing to change my mind about them, but I feel that they violate some of my core tenets of programming, including:

• Code should be as readable as possible without sacrificing functionality -- this may include doing something in a less efficient, but clearer way as long as the difference is negligible or unimportant to the application as a whole.
• Code should be maintainable -- if another programmer comes along to fix my code, non-obvious behavior can hide bugs or make functional code appear buggy (see readability)
• "The right tool for the right job" -- I'm sure you can come up with contrived examples that could use lookahead, but I've never come across something that really needs them in my real-world development work. Is there anything that they're really the best tool for, as opposed to, say, multiple regexps (or, alternatively, are they the best tool for most cases they're used for today)?

My question is this: Is it good practice to use lookahead/lookbehind in regular expressions, or are they simply a hack that have found their way into modern production code?

I'd be perfectly happy to be convinced that I'm wrong about this, and simple examples are useful for examples or illustration, but by themselves, won't be enough to convince me.

Answer 1

I still find lookahead and lookbehind to be terribly confusing and often unreadable.

You're aware that regular expressions can be exploded and commented, right?

$foo =~ m/^
  (?=.*a)           # must contain an a somewhere
  (?=.*c)           # must contain a c somewhere
  (?=.*1)           # must contain a 1 somewhere
  (?=.*2)           # must contain a 2 somewhere
  \S+               # all non-space characters
$/x

Is it good practice to use lookahead/lookbehind in regular expressions, or are they simply a hack that have found their way into modern production code?

They are quite indispensable, to avoid catastrophic backtracking and regex-related security issues. Ideally use plain atomic groups as well.

Compare how the above expression will backtrack, as compared with the naive equivalent:

$foo =~ m/^
  \S*a\S*c\S*1\S*2\S*      # a, then c, then 1, then 2
 |
  \S*a\S*c\S*2\S*1\S*      # a, c, 2, 1
 |
  \S*a\S*1\S*c\S*2\S*      # a, 1, c, 2
 |
  \S*a\S*1\S*2\S*c\S*      # a, 1, 2, c
 |
  # ... etc
$/x

Especially with a long input and a random sequence of a, c and 2 (no 1).

Answer 2

To take a very general view, you'll just have to weigh the trade-offs against each other. On one hand you have an advanced feature which is supported only by a handful implementations, which will be hard to read unless you're very proficient with it. On the other you have a (very likely longer) piece of code using simpler constructs. Which one is the best depends on the accessibility (directly related to the original programmer's and your own proficiency) and more abstract concerns such as accuracy and speed. As a general rule, my own opinion (after using regexes for about a decade):

• Unless the code is throw-away (and we all know how that ends), use regexes sparingly. They are a bit like putting together an 8-dimensional model of the problem, and untangling them is accordingly difficult.
• Don't equate a short regex with short processing time. A five-line substr/if-else construction could very well be faster, so try it before committing to an approach.
• It often gets incredibly difficult to handle corner cases (witness all the people asking about regexes to parse HTML). Divide and conquer rather than eating the whole string regex-style.