There are historical reasons for this guidance, that are mainly related to the lack of uniform encoding standard.
Unicode dates back to the 90s'. Before it became mainstream, there was no global standard way to encode
ä. The western character sets used to be limited to a single byte (octet to be more precise). This leaves room for only 128 characters on top of the ASCII ones. But there are many more accented local characters and greek letters than the 128 that were available.
This is why the ISO-8859 character sets were standardized in several variants. People used country-dependent settings. Suppose a German developper worked in ISO 8859-1 and had an
ß in the identifier. This was encoded as
0xDF in the source file. If a greek developer opened the same source, using an ISO 8859-7 setting, the identifier would be displayed with an
ί instead of
ß. If the greek developer would type a
ß it would lead to
0xE2 in the source file, which a French developer would then read as
â. This was a total mess guaranteed. And when an US colleague looked at it, it just showed a dot, or a question mark, or, a semi-graphic character (after 1981, when extended ASCII became popular).
The lack of uniformity triggered practical annoyances in text editors. I remember for example that the word-by-word move viewed non-ascii as a word separator. So the navigation was not as smooth as with ascii identifiers.
More seriously, there was a lacking tool support. I remember the first linker on MS-DOS had constraints about the length of the symbol identifiers and it was limited to ASCII character sets. This is btw not so old story according Wikipedia about GNU Compiler collection (see also this SO question):
Although the C++ language requires support for non-ASCII Unicode characters in identifiers, the feature has only been supported since GCC 10. As with the existing handling of string literals, the source file is assumed to be encoded in UTF-8. The feature is optional in C, but has been made available too since this change.
Momentum, prejudice and internationalization
To prevent all these nasty issues, a lot of coding standards pragmatically recommended the use of ASCII characters for identifiers. This creates some momentum.
Moreover there is a broad consensus for using English language in identifiers in the context of international projects, or open source projects looking for a broad community. Having this kind of expectations create some prejudice against opening up to unicode characters.
But not all projects have an international audience. There are lots of teams out there working in a local context and using native language in comments, in git commits and even in identifiers. A study of some 1.1 millions non-English git repos demonstrate that this is a large scale reality. Here unicode is relevant (after all, there must be a reason for new languages such as Swift, C# and others to accept unicode identifiers). Ironically, due to the historical problems, many keep using transliteration (e.g. in German
ae instead of
ä or, more ambiguous: in French
e instead of
So, still nowadays, a significant number of people have kept in mind that there was something, and just continue to promote ASCII, although Unicode identifiers are now largely supported.
Edit: All unicode chars are still not equal!
Accepting unicode characters in an identifier does not ensure that all unicode characters are equally treated in all languages that claim to support it.
Unicode characters are categorized into classes, for example spacing, punctuation, letters (aka writing alphabet) and others. Some quick experimentation on my mac:
- Swift, C++ and C# interpret spacing characters properly as token separator, whereas python considers it as an error.
- Emojis are not letters. C++ and Swift accept them in an identifier, but python and C# don't (Example:
- Characters of class "letter" are generally accepted in an identifier in all my tests in the four languages. But some Egyptian hieroglyphs are not recognized as letter by C# (Examples:
téw, the first being the problematic one)
- And good news for the mathematicians among us,
π is a valid identifier in all these languages ;-)