Historical perspective
The Wikipedia article is quite detailed about the origins of regular expressions (Kleene, 1956). The original syntax was relatively simple with only *
, +
, ?
, |
and grouping (...)
. It was terse (and readable, the two are not necessarily opposed), because formal languages tend to be expressed with terse mathematical notations.
Later, the syntax and capabilities evolved with editors and grew with Perl, which was trying to be terse by design ("common constructions should be short"). This complexified the syntax a lot, but note that people are now accustomized to regular expressions and are good at writing (if not reading) them. The fact that they are sometimes write-only suggest that when they are too long, they are generally not the right tool.
Regular expressions tend to be unreadable when being abused.
Beyond string-based regular expressions
Speaking about alternative syntaxes, let's have a look at one that already exists (cl-ppcre, in Common Lisp). Your long regular expression can be parsed with ppcre:parse-string
as follows:
(let ((*print-case* :downcase)
(*print-right-margin* 50))
(pprint
(ppcre:parse-string "^(?:([A-Za-z]+):)?(\\/{0,3})(0-9.\\-A-Za-z]+)(?::(\\d+))?(?:\\/([^?#]*))?(?:\\?([^#]*))?(?:#(.*))?$")))
... and results in the following form:
(:sequence :start-anchor
(:greedy-repetition 0 1
(:group
(:sequence
(:register
(:greedy-repetition 1 nil
(:char-class (:range #\A #\Z)
(:range #\a #\z))))
#\:)))
(:register (:greedy-repetition 0 3 #\/))
(:register
(:sequence "0-9" :everything "-A-Za-z"
(:greedy-repetition 1 nil #\])))
(:greedy-repetition 0 1
(:group
(:sequence #\:
(:register
(:greedy-repetition 1 nil :digit-class)))))
(:greedy-repetition 0 1
(:group
(:sequence #\/
(:register
(:greedy-repetition 0 nil
(:inverted-char-class #\? #\#))))))
(:greedy-repetition 0 1
(:group
(:sequence #\?
(:register
(:greedy-repetition 0 nil
(:inverted-char-class #\#))))))
(:greedy-repetition 0 1
(:group
(:sequence #\#
(:register
(:greedy-repetition 0 nil :everything)))))
:end-anchor)
This syntax is more verbose, and if you look at comments below, not necessarily more readable. So don't assume that because you have a less compact syntax, things will be automatically clearer.
However, if you start having troubles with your regular expressions, turning them into this format might help you decipher and debug your code.
This is one advantage over string-based formats, where a single character error can be difficult to spot.
The main advantage of this syntax is to manipulate regular expressions using a structured format instead of a string-based encoding. That allows you to compose and build such expressions like any other data-structure in your program.
When I use the above syntax, this is generally because I want to build expressions from smaller parts (see also my CodeGolf answer). For your example, we may write1:
`(:sequence
:start-anchor
,(protocol)
,(slashes)
,(domain)
,(top-level-domain) ... )
String-based regular expressions can also be composed, using string concatenation and or interpolation wrapped in helper functions. However, there are limitations with string manipulations which tend to clutter the code (think about nesting problems, not unlike backticks vs. $(...)
in bash; also, escape characters may give you headaches).
Note also that the above form allows (:regex "string")
forms so that you can mix terse notations with trees. All of that leads IMHO to good readability and composability; it addresses the three problems expressed by delnan, indirectly (i.e. not in the language of regular expressions itself).
To conclude
For most purpose, the terse notation is in fact readable. There are difficulties when dealing with extended notations which involves backtracking, etc., but their used are rarely justified. The unwarranted use of regular expressions can lead to unreadable expressions.
Regular expressions need not be encoded as strings. If you have a library or a tool that can help you build and compose regular expressions, you'll avoid a lot of potential bugs related to string manipulations.
Alternatively, formal grammars are more readable and are better at naming and abstracting sub-expressions. Terminals are generally expressed as simple regular expressions.
1. You may prefer to build your expressions at read-time, because regular expressions tend to be constants in an application. See create-scanner
and load-time-value
:
'(:sequence :start-anchor #.(protocol) #.(slashes) ... )