23

Unicode has maybe 50 spaces

\u0009\u000A-\u000D\u0020\u0085\u00A0\u1680\u180E\u2000-\u200A\u2028\u2029\u202F\u205F\u3000][\u0009\u000A-\u000D\u0020\u0085\u00A0\u1680\u180E\u2000-\u200A\u2028\u2029\u202F\u205F\u3000

and 6 line breaks

not only CRLF, LF, CR, but also NEL (U+0085), PS (U+2029) and LS (U+2028).

Maybe I could understand most of the spaces and PS ("Paragraph separator"), but what are "Next Line" and "Line separator" good for?

It all looks like invented by a very big committee where everybody wanted their own space and the leaders were granted one line break each. But seriously, how do you deal with it when your programming language doesn't support it (or does it wrong as e.g. Java does)?

8
  • 1
    How does Java do it "wrongly"? Commented Jan 30, 2011 at 1:19
  • Nearly completely, s. stackoverflow.com/questions/4304928/…
    – maaartinus
    Commented Jan 30, 2011 at 1:23
  • 2
    @maaartinus: (I can't believe I'm defending Java of all things) Java's character classes are documented to apply to a specific set of characters. Unicode supplies more characters which look like they fit into these character classes, but Unicode does not define regular expression languages; only character encodings. Java behaves completely correctly according to it's spec -- that is to match typical whitespace. If you want it to match everything in the Unicode standard that might be seen as empty space then you'll have to write that yourself. Commented Jan 30, 2011 at 1:29
  • 2
    Thx for the info. However, but they're free to create a Pattern.compile2010 method returning regexes working according to last years definition. They're also free to create a method Pattern.compileLatestUTS which would explicitly state, that the meaning would change according to new specification.
    – maaartinus
    Commented Feb 9, 2011 at 14:51
  • 2
    Looks like Java eventually did fix/modernize their regex implementation, using an opt-in flag to prevent backwards compatibility problems: stackoverflow.com/a/4307261/1172352
    – peterflynn
    Commented May 19, 2016 at 0:24

1 Answer 1

17

Maybe I could understand most of the spaces and PS ("Paragraph separator"), but what are "Next Line" and "Line separator" good for

NEXT LINE (U+0085) is often used as the newline character on EBCDIC systems (as 0x15). It's like CR+LF, but as one character.

LINE SEPARATOR (U+2028) and PARAGRAPH SEPARATOR (U+2029) are explained in section 5.8 of the Unicode standard, which describes them as a plain-text version of HTML <br> and <p>, to disambiguate these functions of "newline". But in practice, these characters don't get used much.

3
  • 2
    Good explanation, however for me it means: one line break per committee leader.
    – maaartinus
    Commented Feb 9, 2011 at 12:30
  • 6
    @maaartinus Nope. One line break per all the previous conflicting standards, and 2 more unambiguous ones from the Unicode standard.
    – Milind R
    Commented Dec 11, 2014 at 18:19
  • 10
    xkcd.com/927
    – dan04
    Commented Dec 12, 2014 at 1:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.