I am having trouble finding a lexicon which provides terminology for the explicit patterns that are employed when parsing syntax. I am trying to write about the niggling differences between the 10+ different markdown parsing rules as they apply to nesting and how different markdown syntaxes which appear to have similar symbology have completely different parsing rules.

What I am trying accomplish is to completely disambiguate the meanings and/or terminology of some of ubiquitous parsing rules of syntax patterns that we probably don't even think about wondering what the name is. I came to realize that I don't have any idea what these patterns are called or if they even have explicit terminology. I was hoping that I could find a lexicon somewhere that might help me but I couldn't find anything that was both unambiguous or not specific to a particular language. In other words, I was looking for something "universally explicit".

Below are some examples of well-known patterns that I am trying to find clear terminology that explicitly describes those parsing patterns and/or the operators that invoke parsing rules. I did my best to give them "made up" names which I hope will convey their meaning but where I don't know the "proper" terminology.

I realize this might seem like a pedantic exercise, but I am hoping there is someone out there who not sees why this is important but also knows the answers!

  1. Grouping Operators - I believe there are only two different types these: those that have explicit symbols that denote the start and end of the group (i.e.: parentheses in arithmetic expressions) and those that use the same symbols (i.e.: single or double quotations marks in string interpolation)
    • Is there a term that distinguishes between these two types of grouping operators?
    • Other than "grouping", is there a term that explicitly means that symbol's syntactical scope is explicitly defined to be whatever is contained within those operators?
  2. Line-Scoped Operators - These are one or more symbols that appear at the beginning of a line which imply that their scope is the entire line, that is, everything up to whatever the EOL symbol is. (i.e.: - which indicates a bullet or # which indicates a level 1 heading in CommonMark.)
    • Is there a term to describe this implied behavior where the symbol(s) must be anchored ad the beginning of the line and the scope is the entire line?
  3. Adjacent Operator - The operator symbol applies to an arbitrary scope that follows it, such as a "word", or specifically up to the next bare whitespace. To me this is the hardest pattern to describe since the rules for what is explicitly "adjacent" can use any number of delimiters, have a different means of escaping the delimiter symbol, and so on. A good example of this is boundary or inline operators in regex.
    • What term best describes this pattern of operator where the scoping ends with some arbitrary symbol?
    • Also, is there a term that describes the scope of these operators? This is tricky as different syntaxes have specific terms, such as how a delimited data file would call these "fields", but I have no idea what one calls the scope of a particular class of regex operator
  4. Block Operators - Same as Grouping Operators but they ignore the EOL symbol. (i.e.:``` for code fences in GFM)
    • I think "block syntax" probably explicitly enough, except that it does differentiate the beginning and end like HTML tags to.
  5. Entity - A symbolic word which is translated to a specific unicode character, such as how &copy is rendered as ©
    • I use the term "Entity" as it applies in markdown, specifically GFM, but outside of GFM "Entity" is about as generic a term can get so hopefully there is an term that explicitly means this kind of pattern universally.

I realize that this question may be more about lexical semantics than software engineering, but since being explicit in programming (or at least fully understanding what constitutes "explicit" in a given context) is how a programmer avoids total chaos and dismal, fiery failure, I figured that this would be the place where I might actually find a lexical ninja erudite enough to actually know what some of these patterns are actually named.


1 Answer 1

  1. "foo" [foo] (a + b) <a>...</a> { stuff; } /* comments and stuff */ - these tend not to have a well known name as a group. Each has its own well known name though, often used as an adjective (quoted foo, bracketed foo, tagged foo, etc).
  2. # // -- - these are simply single line comments or rarely line modifiers. The token doesn't need to be at the start of the line, and these generally aren't treated distinctly from #1 (where newline|EOF is your end marker).
  3. ! + and ++ async - these are prefix, suffix, or infix operators depending on if they are before, after, or between non-terminals respectively. These are the most common in actual grammars since everything is reducable to that sort of simple form (https://en.wikipedia.org/wiki/Chomsky_hierarchy for the elemental language structures).
  4. { stuff; } <a> ... </a> @"..." - are just blocks. In practice these tend not to distinguished from #1, since newline is just another character that can be allowed or not in the contained non-terminal.
  5. &copy \n 0x64 :emoji: - are generally referred to as escape sequences or occasionally encoded character (or encoded int or encoded whatever).

And one other very common one you didn't mention is something delimited where you have something like a, b, c which is "comma-delimited".

Depending on your audience, these may or may not matter. #3 matters the most when talking about actual grammars. Escape sequences and delimited sequences show up in almost all languages. Quoted things, parenthesized things, and blocks are very common, but might not be called that in your context. The correct terminology will depend on who you're trying to convey information to.

  • I am trying to unambiguously describe anti-patterns within the ecosystem of markdown parsers. You actually just made me realize that the fact markdown relies on what is essentially comment syntax creates a fundamental problem with nested structures. By definition they require line breaks which obviously interrupt blocks. This might explain why the GFE spec - the closest thing to an unambiguous set of semantics for any markdown spec - reads like a laundry list of rule exceptions!
    – Darf Nader
    Commented Dec 15, 2019 at 9:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.