7

People often get excited about JuliaLang supporting Unicode function names. But it's not new at all,it's just that the Julia community decided that it was sometimes appropriate, and built tooling to make entering Unicode easy.

What was the first language to allow it? Please backup answer reference to a reliable source, e.g. release notes.

I have seen reference to it being allowed in Java 1.5, in 2004, but I could not find it in release notes.

Note: I am looking in particular for Unicode. So Mathematica's rich text files, or APL's use of non ASCII symbol and custom code-pages do not count.

3
  • 2
    I don't know if it was the 1st one, but the Delphi parser allows almost everything for any symbol names (quite horrible IMO BTW). Delphi (Pascal) is older than Julia IIRC. Dec 25, 2019 at 0:02
  • 1
    “it's just that the Julia community decided that it was sometimes appropriate”: they are right. Dec 25, 2019 at 0:21
  • I guess the biggest problem with this question is that Unicode is extremely new compared to most programming languages. Almost nothing interesting has happened in programming language design since 1991. Even Java's design had started by then. And what we nowadays consider as Unicode only appeared with Unicode 2.0 in 1996. For example, Algol allows pretty much everything in an identifier, even whitespace! But, of course, Unicode didn't appear until 30 years later. Dec 25, 2019 at 23:45

2 Answers 2

6

Perl’s 1990s Unicode Support in Function Names

I can’t say that Perl was “the first” programming language to allow Unicode in function names — this seems unlikely to me — but I can definitively document when it first began to do so.

First Repository Commit: 834a4ddd8309fbf6aabbbc51bb6fcbe056e7963f on 1998-10-23

It was in this commit from 1998 that Larry Wall fixed the bugs standing in the way of using “UTF-8 identifiers” in Perl:

commit 834a4ddd8309fbf6aabbbc51bb6fcbe056e7963f
Author: Larry Wall <[email protected]>
Date:   Fri Oct 23 18:00:41 1998 +0000

    Program with utf8 identifiers fails to compile

Hindsight shows that it proved a blessing that it took us that long, for it was to our great good fortune that we held off (some might say dilly-dallied) for years before finally adding any Unicode support to the core Perl distribution until an earlier commit of Larry’s in July of 1998:

commit a0ed51b321531af4b47cce24205ab9656f043f0f
Author: Larry Wall <[email protected]>
Date:   Fri Jul 24 05:44:33 1998 +0000

    Here are the long-expected Unicode/UTF-8 modifications.

Notice that that was a couple years after Unicode 2.0’s release in 1996. Delaying Unicode strings in the core language allowed us to avoid the sea of troubles that flooded the early-adopter languages that had jumped right into Unicode 1.0’s 16-bit characters, and which all gut stuck with that. By waiting to get our feet wet in that ancient sea until “modern” Unicode’s 21-bit code point repertoire, we were able to use UTF-8 as our internal representation for all our strings.

First Development Release: version 5.005_54 on 1998-11-30

Tracking Perl’s historical minor releases towards the end of the last millennium can be tricky to the uninitiated owing to the peculiar version-numbering conventions it was still at that time using for its parallel development and maintenance tracks.

First Stable/Production Release: v5.6 on 2000-03-22

Suffice it to say that being able to use Unicode in function names first appeared in the 5.005_54 minor development release, which was building to eventually producing the v5.6 release in March 2000. The v5.6 release notes contain this passage:

Unicode and UTF-8 support

Perl now uses UTF-8 as its internal representation for character strings. The utf8 and bytes pragmas are used to control this support in the current lexical scope. See perlunicode, utf8 and bytes for more information.

By that time, Unicode itself was up to its v3.0 release.

Lessons Learned: Always Throw the First One Out!

Even though Perl never got caught by the 16-bit trap (still!) afflicting so many others, its initial internal model for Unicode support quickly proved itself to be problematical one, so we rewrote all those internals for the v5.8 release on 2002-07-18.

Fortunately this internals rewrite affected very few actual end-users, because it was restricted to the internal C API. As far as end-programmer use, the crucial logical separation of an abstract code point (a 21-bit number) on one hand and the hidden underlying physical memory layout inherent to UTF-8 representation on the other was preserved, and so folks writing Perl scripts (rather than folks updating the perl compiler and interpreter) nearly never need to think of anything but the abstract characters (more like Go’s runes), not machine layouts in bits and bytes.

Reworking our initial representational model not only fixed a broken design, it also made it easy to role in updates from the Unicode Consortium; by Perl v5.8.1 of 2003-09-25 we supported v4.0 of the Unicode Character Database, and have tracked updates to the UCD and the many closely related annexes and technical reports (like TR#18 on Unicode Regular Expressions) fairly tightly ever since then.

Then and Now

And sure, there are still various “Unicode-y” things we don’t do it as well in Perl as we could. Examples that quickly come to mind include normalizing the identifiers for variables and function names through something like NFKC normalization, or making better guarantees about how Unicode in module names get represented externally in the filesystem.

Even so, I still find it easier to use Unicode in Perl than in any other programming language of its venerable vintage. After all, they all had to have Unicode support tacked on after the language was first created, and that’s always its own kind of nightmare. Many Unicode-related tasks that are hard in languages originally designed a long time ago but which like Perl and Python are still around today are sometimes easier in “recent” languages designed with abstract Unicode support built into their design from the git-go, as the saying runs. But that’s a topic for another day.

4

Java supported unicode in identifiers from its first release (1996). See The Java Language Specification 1.0: http://titanium.cs.berkeley.edu/doc/java-langspec-1.0/3.doc.html#40625 :

Letters and digits may be drawn from the entire Unicode character set, which supports most writing scripts in use in the world today, including the large sets for Chinese, Japanese, and Korean. This allows Java programmers to use identifiers in their programs that are written in their native languages.

The first spec refers to Unicode 1.1.5. later updated to unicode 2.0.

Unicode 1.0 was released in 1991, five years before than Java 1.0, so it is theoretically possible some other languages supported unicode earlier. I don't know of any though.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.