Should putting html tags into localization files be avoided?

Question

We are localizing all parts of our website to many languages. We use XML localization files. I think this scenario is so common, and even there should be a standard solution to this, but still I couldn't find any good advice, and every developer here has different opinion about it, so I'm asking you.

Suggest the following example:

If you have question, please ask our <a href="blabla" target="_blank" title="Our nice Customer Support">Customer support</a> or write an email to <a href="mailto:XXX">Jane Doe </a> our specialist.

Or a long, formatted text:

 <p>A very long, marketing blah-blah. A very long, marketing blah-blah. A very long, marketing blah-blah concluding in a list:</p>
 <ul>
   <li>1</li>
   <li>2</li>
   <li>3</li>
</ul>

1) Would you put html tags into the localization XML file?

My concern is the View is separated into 2 files: your page and your localization file. People will forget to check the localization file. Also, there's logic and style embedded in html as well (see target="_blank", or the fact that the above mentioned list is unordered...)

2) Or splitting it to smaller parts?

<msg id="IfYouHaveQuestion">If you have question, please ask our</msg>

<msg id="CustomerSupport">Customer support</msg>

<msg id="OrWrite">or write an email to</msg> ...

Now, View can contain all the markup and style. It's easy to change it, flexible.

But.

There's absolutely no guarantee that the word order will be the same in all languages. Also, this would make the translator's work a nightmare, making it a puzzle.

3) Or introduce BB-style markdown?

<msg id="HaveQuestion">If you have question, please ask our [link url="{customersupportlink}" title="{{CustomerSupportTitle}}">Customer support[/link] or write an email ...</msg>

But probably this is over-complicating the issue, and also we have to write our own parser for this (though, I think it wouldn't be so hard). And probably does not solve the long, formatted text problem.

4) ??? (Your golden solution here) :)

Answer 1

It seems there is no mainstream choice, so here is my suggestion :

Localisation files could be used more like semantic data than just text strings. It seems reasonnable to expect identifiying a list , tagging a paragraph, a name or a part of a phrase being part of the localization team work. So it could contain semantic (but no logical) html tags and use semantic span tags ( like <span id="seo-name"> ) in localization files. Note : I here suggest span, wich is a valid html tag and so could be manipulated easily as a DOM element, but nothing stop you to use your own tags to parse.

Doing so you can in your view logic code, when extracting the text from the localization file, identify the seo-name, and adding the html link tags properly.

You may even, since some prior answer have made a security point about leting potentialy unknown people writting html code on your website, have a security parser wich check only limited safe tags (<p>, <ul>, <ol>, <li>, <span id="blabla">, ...) are present in localization files.

An exemple to illustrate :

If you have question, please ask our <a href="blabla" target="_blank" title="Our nice Customer Support + boilerplate SEO bullshit">Customer support</a> or write an email to <a href="mailto:XXX">Jane Doe </a> our specialist.

Could became in the file with thoses convention :

If you have question, please ask our <span id="customer-support">Customer support</span><span id ="customer-support-description">Our nice Customer Support + boilerplate SEO bullshit</span> or write an email to <span id="specialist-name">Jane Doe </span> our specialist.

Wich (I think) ins't really difficult to localize in french for (bad since closely-related) exemple by :

Si vous avez des question, n'hésitez pas à rencontrer notre <span id="customer-support">Service client</span><span id ="customer-support-description">Notre super service client + habituel blabla commercial</span> ou à contacter par email notre spécialiste <span id="specialist-name">Jane Doe </span>.

And a ugly controller pseudocode exemple :

$customerServiceParagraph = getLocalizedText("customerServiceContact",$lang);
$customerSupportDescription = getTextContentByElementIdThenDelete("customer-support-description"));
linkify($customerServiceParagraph,"customer-support",$customerSupportDescription,"blabla","_blank");
mailto_ify($customerServiceParagraph, "specialist-name","XXX" );

I'm far from being a localization expert but think the ordered or unordered list choice is a matter of cultural convention, and so is also a part of localization team work, even if I agree thoses tags are a legacy of dirty old styling/semantic tags collection of prior html versions.

Answer 2

The fact that solution 1 and solution 2 are both less than ideal got me thinking*. Here's an attempt at solution 4 (the "golden" one) that is not just a compromise between 1 and 2.

The goals of this approach are:

• Translators work only with natural language
• Translators translate meaningful whole sentences, not chopped up parts of speech
• Programmers control the markup
• The architecture respects the DRY principle

The basic idea is to ask the translators to provide both a complete phrase or sentence, and additional guide phrases that the programmer will use to add markup dynamically.

So looking at your example** where the desired output was

If you have a question, please ask our
  <a href="..." target="_blank" title="Customer support agent">Customer support agent</a>    
or write an email to our specialist, 
  <a href="mailto:...">Jane</a>.

The translator would provide this XML:

<msg id="IfYouHaveAQuestion_body">
  If you have a question, please ask our Customer support agent or write an email to our specialist, Jane.
</msg>
<msg id="IfYouHaveAQuestion_link">Customer support agent</msg>
<msg id="IfYouHaveAQuestion_email">Jane</msg>

Without needing to use markup, the translator is using the second and third strings to specify which parts of the first string should be marked up. Given those hints it's not hard for a coder to assemble the desired output with basic string manipulation methods.

*Solution 3 is not much different than solution 1, but it does have the advantage that BBCode is a more restricted form of markup than HTML.

**Unlike this example, the second example is not a hard case: the list markup should not go into the localization resources.

Answer 3

Generally speaking you should avoid placing these HTML tags in your localised strings. It violates the DRY principle and there is at least one good reason to adhere to it here: Imagine you have not only got - let's say - English and French, but also a plethora of other languages. Now you find one HTML tag to cause problems. You will have to change all of your localisation files - for virtually no reason. If you went with the DRY principle from the beginning you will only have to change one single file at best. Unfortunately the same applies to other markups.

Regarding your problem of the word order this is of course a bit more complex. I still think you should explode the strings as much as possible, but I also think that there is a huge gain in labels of localisation strings which are more or less self explanatory. Therefor you maybe should ditch DRY and consistency to some extent and introduce HTML tags in l13n strings when it is only avoidable with a great effort and the markup is unlikely to change. Otherwise use plain strings.

Another thought: The method I described above only holds if you can absolutely trust your resources. If there is only the slightest chance that someone may inject malicious code via your l13n resources. Use another markup you validate before inserted (ok, should work with HTML either, but I believe it's easier with a more restricted markup) or - if your project allows - a formatter like textile.

Answer 4

With PHP gettext implementations we use placeholder string substition for placing varying content inside translatable texts.

For instance, in WordPress (__() is a WP specific gettext function used to extract strings from source) you can do this:

$content = sprintf( __( 'Send us a message or %1$sview our contact information%2$s.' ), '<a href="...">', '</a>' );

Which gives is a .pot source string

Send us a message or %1$sview our contact information%2$s.

After PHP parses and executes sprintf you get the following output (in the wanted language of course):

Send us a message or <a href="...">view our contact information</a>.

This way all the markup and other non-translatable data resides inside the application logic and stays out of the translation string data. The only gotcha is to instruct translators to actually copy over the placeholders properly.

Depending on your implementation you could use placeholders too. A custom implementation of sprintf wouldn't be that difficult either. So in essence your "BB-code" variant would be closest.

Answer 5

I echo Neil's and other's comments. Just as you also said in your second point, you never, ever want to split the message into chunks if you are localizing the software. It is a translator's nightmare (although it happens regularly, so it wouldn't surprise them) and it will delay the translation more than you can ever imagine because of the time it will take for the translators and QA specialists to fix the problems due to different word arrangement and order in other languages. It will also negatively affect other parts of the process, including the translation memory, which will wreak havoc on future iterations of the text strings. Tags are an unresolved problem with localization. We've used several methods to deal with it including protecting the tags and using hidden placeholders, and then reconstructing the file after translation, but no method is perfect. Ideally, you should try to minimize tagging within the text as much as possible. However, at the same time, translators and translation companies are accustomed to dealing with some level of tagging within the text, so it isn't the end of the world if you have some tagging in your files.