7

http://babelstone.blogspot.com/2005/11/how-many-unicode-characters-are-there.html says there are 1 million unicode characters and around 240k of which are already assigned.

1 million > 240k > 65k

However,

http://msdn.microsoft.com/en-us/library/microsoft.visualbasic.strings.chrw says that chrw accept 65k characters.

Not only that chrw accept integer. Integer is 4 bytes in vb.net right and can store way more than 65k characters.

The numbers do not match up and so what am I missing?

Improve this question

1 Answer 1

Reset to default
10

Chances are that this method only supports the Basic Multilingual Plane of Unicode. That Plane contains the lower 64k of codepoints and can be represented with a 16 bit data type.

There was a time when the BMP was all the Unicode standard defined and at that time many languages and/or runtimes added "Unicode support". They thought that 16 bit will always be enough and therefore "Unicode" equals "16 bit characters" in many places (even though this is wrong these days). To be fair: The Unicode consortium also thought that 16 bit ought to be enough for everybody.

Unicode 2.0 however introduced additional planes and it was clear that 16 bit are no longer enough for representing every possible Unicode codepoint.

The "solution" to this is usually to use UTF-16 instead of UCS-2. I'm not only faulting .NET for this: Java has fallen into the same trap, having a 16-bit char data type and now having to support String instances that need 2 "characters" to represent a single codepoint.

Improve this answer
5
  • So soon we will have chrVeryWide (someNumber)? Or is that function already there?
    – user4951
    May 23, 2012 at 8:10
  • @JimThio: I don't know .NET well enough to know if its there and a quick peek into the String class documentation you linked to doesn't reveal anything obvious. In Java there's now a int equivalent for most methods dealing with a char where the word Char/Character is replaced by Codepoint in the name. So String.charAt() is replaced by String.codePointAt().
    – Joachim Sauer
    May 23, 2012 at 9:26
  • 2
    yep, UTF-16 is a "historical accident" and should be considered broken. benlynn.blogspot.co.uk/2011/02/utf-8-good-utf-16-bad_07.html
    – gbjbaanb
    May 23, 2012 at 13:19
  • 2
    @gbjbaanb: actually I'd argue that the historical accident is UCS-2. UTF-16 is an attempt to make the best out of it. It's a "UCS-2 compatible encoding" in the same way that UTF-8 is a "ASCII compatible encoding". So UTF-16 is the middle ground: it's not perfect but it allows you to continue using most of the tools that expect/handle UCS-2. UTF-8 is almost certainly the long term solution to all the encoding problems.
    – Joachim Sauer
    May 23, 2012 at 13:23
  • Note: Java fallen into the trap because it was designed before UTF-16. .NET, on the other hand, was designed far later, so I would say that it was just a bad design choice that preferred compatibility with Windows rather than the rest of the world. And yes, it's a bad design choice because, for instance, all what ASP.NET does is converting all its strings back and forth from UTF-16 to UTF-8... in addition to higher memory consumption.
    – ybungalobill
    Jun 22, 2012 at 11:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.