0

Java, by default, uses UTF-16 to represent characters in the String data type.

I inherited a JavaFX project which currently has some Strings in UTF-8 and others in UTF-16. This is causing bugs (in pop-ups for example) and I'm at a stage where I must provide some uniformity and choose between one and the other. Do note that for the pop-ups, I must use UTF-8 because UTF-16 doesn't show the characters correctly (I'm not sure why this happens, nor is that the focus of this question).

If Java used UTF-8 by default, I would absolutely use it as well because it is the de facto encoding for the foreseeable future. However, since Java uses UTF-16 by default, I was thinking of changing everything to UTF-16 to be consistent with the language, and then if need be when creating these pop-ups convert to UTF-8. Since there are many pitfalls associated with this encoding, of which a good summary is [1], I'm scared that I'm making the wrong decision.

So, which encoding should I use to store my String variables?

A similar question [2] was asked but for PHP and not between UTF-16 and UTF-8. I believe this qualifies as a different question due to Java natively using UTF-16.

[1] - Should UTF-16 be considered harmful?

[2] - Should I convert the whole project to UTF-8?

10
  • 3
    Java strings are inherently UTF-16. It is not something you have control over. But when reading and writing files you might use utf-8 encoding for the files, and it would be a good idea to consistently use UTF-8.
    – JacquesB
    Commented May 4 at 18:05
  • 6
    This sounds like you're suffering mojibake/double-encoding issues. Fix those. It is not meaningful to talk about the encoding of a String, because conceptually Strings are sequences of already-decoded chars (which correspond to Surrogate Halves, not Unicode Codepoints/Scalars). Java code cannot determine or control how those Strings are actually represented in memory (but having read the OpenJDK source code, I know that Strings may be represented as Latin-1 or UTF-16 depending on contents).
    – amon
    Commented May 4 at 19:47
  • 4
    @chilliefiber Are you talking about the constructors which take a byte array and a charset parameter? In this case the encoding applies to the input byte array, not the resulting string.
    – JacquesB
    Commented May 4 at 20:23
  • 4
    The internal representation of characters inside the Java virtual machine is not relevant to how you choose to store your text strings outside of it. The world is moving towards UTF-8 (since ASCII strings are valid UTF-8 too). Go with that. Commented May 4 at 22:44
  • 1
    I'm sorry to everyone but I just realised my question was made under false assumptions. I saw that conversion to UTF-8 being done in the code and assumed it was a problem because the XML was stored in UTF-8 and so were the source files but Java uses UTF-16. In reality the javac command was being issued with a third encoding (don't ask me why) and that was the root of the issues Commented May 5 at 0:29

1 Answer 1

1

Java uses UTF-16 internally. But nobody needs to care about that, except for a tiny bit of efficiency.

UTF-8 is much more what everyone uses as the standard for external representation.

0

Not the answer you're looking for? Browse other questions tagged or ask your own question.