One of the biggest challenge of localization is not to extract all the translatable resources and include them in one single interface but it is about providing the context is which each strings will appear in order for the translator to give the best wordings.

This is what this post is about.

I am going to show you how to add context screens in iL10Nz for any loaded resources.

1. First you need to login to iL10Nz. You ccan either use the demo account or create your own account.
2. Double click on the project you want to open or create your own project.
3. The project will expand and you will find the __SCREENS__ folder that contains the context screens.
4. Select the __SCREENS__ folder and right click
5. Select Upload files to upload in the project the screen. It can be .gif, .png, .jpeg.
6. Double click again on the __SCREENS__ folder to see the image you just uploaded.
7. Now you need to link a string to this screen
8. Open a translation grid by click in the Language node in the project for a specific file and release.
9. Select one string
10. Open the Right Panel (Translation helper)
11. Select the Context panel
12. Select the image from the Screen drop down and click on Add project screens
13. The image appears in the preview panel when you select this string.

In this way, you can then link an image representing a context screen into the project for each string to help the translator.

Next step into this developement will be add multiple strings to one screen.

In the back office,  iL10Nz is using the OKAPI framework based on a WEBService.

OKAPI framework is a great Java tool developped by people from the localization arena and provided great support. Many components are included in this framework such as Rainbow, CheckMate etc.. but from my usage, I only use the Core part which are the different classes.

I have built my own interface to provide WebService support to some specific functionalities in order to connect with iL10Nz parsers in PHP. OKAPI has a great list of filters that can handle the industry file format such XLIFF, TMX, etc.. but it can be easily extended to manage other formats. To find more on OKAPI, go to http://okapi.opentag.com/

The OKAPI filter behavors can be modified at run time using some configuration settings included in a .conf file  in iL10Nz so the user can customize how the file is parsed.

Here is an example of the po configuration file in iL10Nz

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<configuration><filterconf>#v1
bilingualMode.b=true
makeID.b=true
useLD.b=true
localizeOutside.b=true
useCodeFinder.b=true
codeFinderRules.count.i=3
codeFinderRules.rule0=%(([-0+#]?)[-0+#]?)((\d\$)?)(([\d\*]*)(\.[\d\*]*)?)[dioxXucsfeEgGpn]
codeFinderRules.rule1=(\\r\\n)|\\a|\\b|\\f|\\n|\\r|\\t|\\v
codeFinderRules.rule2=\{\d.*?\}
codeFinderRules.sample=%s, %d, {1}, \n, \r, \t, etc.
codeFinderRules.useAllRulesWhenTesting.b=true
</filterconf>
</configuration>

To change the configuration, select the Project in iL10Nz. In the Menu select Projects / Customize Parsers

Filter Configuration

When you customize the configuration file, the parser and the conf file are copied in the __PARSERS__ folder in the project folder.

The OKAPI filters are used when parsing the source file after uploading it, during the rebuild, import translation. If there is an exception raises in OKAPI, the error message is sent back to the parser to be displayed to the UI and a SendErrorReport dialog will popup to send the details of errors to the iL10Nz team in order to trouble shoot and fix the issue.

In the area of videos, the SRT are used display subtitles in your video. The format of this file type is quite simple and straightforward.

In this post, i am going to show how to use iL10Nz to localize your SRT files.

First, here is an example of the file format

1
00:00:03,583 --> 00:00:07,385
Écoute Sam, on a sauvé beaucoup de gens,
Mais on ne peut pas tous les sauver.

2
00:00:08,171 --> 00:00:09,171
Je suis Meg.

3
00:00:11,758 --> 00:00:13,182
Ronald Reznick.

4
00:00:15,428 --> 00:00:17,979
Voici l'agent spécial
Victor Henrickson.

You can easily identify the different sections of this format.

- Subtitle index

- Subtitle duration Start –> Stop

- Subtitle (can be in multiline)

I have customize a OKAPI filter to handle the parsing of this file in iL10Nz so now the SRT extension is well recognized in the application.

1/ Open a project in iL10Nz by selecting the menu Open project or double clicking directly on the project tree

2/ Click on the project

3/ Right click with the mouse and select Upload file

Upload file

Choose the releasename you want to set. By default, the release name is the timestamp.

4/ After the file is uploaded, you need to load the resources. This steps trigerred the parsing by iL10Nz and update the database.

Load resources

When the resouces of the SRT file have been loaded in iL10Nz. you can open the translation view by double clicking on the project tree.

You can see that the subtitle, index, duration have been added in the translation database.

5/ When the subtitles have been translated, you can then select in the project menu Rebuild file then Download file in order to get the localized SRT file.

You can use directly the file in your Video application

Limitation found:

-  Use SRT file in UTF8 in order to add extension characters.

- Add the new line at the end of the file in order to get the last subtitle

In my daily work, I often have to work on excel to process important amount of  data.

When many strings are stored in one column, I have the need to extract the unique value for one column as if I was doing a SELECT DISTINCT in one database in SQL.

Here is how to extract the unique list of strings in one Excel column:

You need to go in the main menu Data / Filter / Advanced Filter

The Advanced Filter dialog box will appear and you need to check the Unique Records only option

Then you will able to copy the list in a particular location.

This way you won’t miss any records when  doing this filtering.

Here is a very good article about the manipulation of date in javascript and how to internationnalize the script.

User Interface

Does the user interface work in all the chosen locales?

Address Information

Is the address format customizable for all locales? It is not an acceptable practice to assume that all addresses are formatted like yours. In many cases the address format is critical to how the mail is handled and whether it gets where it is going today or next week.

Date and Time Information

Is time formatted and displayed correctly in all locales? Many countries show time on a 24 hour clock basis. There may be no AM or PM designation so don’t rely on it being there. Even if there is, it may not be the generally accepted way to display the time.

Are dates formatted and displayed correctly in all locales? The use of only numerics in a date is called the “short” format in Java (and the OS). However no locale uses a 4 digit year format in their short form. In the short form the major differences in locales (even within the same language variant) is the ordering of month, day and year. In en_US it is MM/DD/YY but in en_GB it is DD/MM/YY. Locale sensitive implementation does make the assumption that the person knows how to enter the date in their own locale. 01/02/2000 would be January 2 in the US but February 1 in the UK.

It is trivial to support locale sensitive date display and input. You can even handle dates entered in the format of a 4 digit year and provide the user with a prompt display that makes the day/month positioning a little more clear. It will be more work (but should be doable) to support a 4 digit year on display. It cannot be done directly with a DateFormat object because the format patterns dictate order as well as style so you will have to build up an appropriate pattern for the locale.

Currency Information

Are currencies formatted and displayed correctly in all locales? This is an offshoot of number handling with the added twist of the currency character. If you use the Format objects for currency the assumption is that you will also be typing in the currency character at the same time (i.e., $1,000,000.54). The symbol is extra work and not something you would necessarily want to store in a database with the numeric value. For input it is probably best to use the same code as for number handling.

The added twist introduced by the currency symbol is position. The symbol is either a prefix to the value or postfix to the value. For example, $100.54 in the US and 100,54 F in France. This means any displays of currency or requests for input of currency needs to be dynamic about the position of the currency symbol for the specified locale.

Can multiple currencies be displayed at any point? In several locales it is required, by law, that currency be displayed in at least two ways. For example, in Europe, you would display the currency in the local currency as well as the Euro.

Numeric Information

Are measurements formatted correctly in all locales?

Like dates, the input and display of numbers is locale sensitive. The thousands separator and decimal separator are the obvious items. Display of numbers in the locale sensitive manner is very straightforward. Unfortunately, locale sensitive input seems to be a different story. The same basic type of formatting objects that exist for dates also exist for numbers but the parsing for input is more limited when it comes to numeric values. For example, in the fr_FR locale (France) you would express 1 million and 54 cents as: 1 000 000,54

The default parsing behavior in Java, however, leverages the mechanisms in the Character class (specifically isDigit()). If the non-breaking space is not a digit the above entry would come out of the stock parsing mechanism as 1. As you can see, there is a slight loss of precision. If this happens, it means a little extra work to assemble the entered String into something that can be handled by java.lang.Double for storage.

Additional logic is needed if you want to protect the user from themselves. Double entry of the decimal separator is not flagged as an error and yields up a truncated value.

Text Formatting

Does line breaking work correctly in all locales (including double-byte locales)? White space is not good enough. French, for example, has some very specific rules on when an article can begin or end a line of text.

Does word wrapping work correctly in all locales (including double-byte locales)? Make sure that code which handles word wrapping can correctly identify appropriate break points for words. There are specific rules for where to break Asian character strings. You should not break a concept. Some concepts can have more than one character in their representation.

Is vertical writing and printing supported? This is not as common with electronic information as with printed material but may be a consideration for some forms of information.

Are the right paper sizes supported? Many countries use the A4 paper size as their default. Make sure you can print correctly on all paper sizes for the locales.

Is right-to-left writing and printing supported? This can become an issue in the Middle East. If you are going to handle RTL language be aware that there can be some LTR strings in the middle of an RTL string.

Is there room for expansion of text both horizontally and vertically? Most western European languages will expand horizontally. Most Asian languages will expand vertically. French and German can expand up to 50% horizontally. More if idioms are used in English. Asian characters will fit nicely and be readable in buttons which are about 14 pixels high. You must be careful with Asian languages that you don’t cut off the top or bottom of the characters since you can change the meaning of the translation if you do.

Cultural Issues

Are colors acceptable in all locales? Colors play a big part in some cultures. Make sure that the colors you choose are not offensive in any of the locales. For example, in English red means stop and green means go, but in some cultures red means royalty and green means nothing.

Are Icons acceptable and relevant in all locales? Icons based on common English words do not translate well or may not mean the same thing in other languages. For example, using a picture of a log to represent a log file has no meaning in most languages other than English.

Is highlighting appropriate in all locales? In Japanese and Chinese the italic fonts do not look good. In some cases it makes the text almost illegible.

Keyboard Issues

Is it possible to use the short-cut key combinations in all locales? Make sure that the key combinations are available on all the keyboards. Some letters on the English keyboard are not available on other keyboards. Some letters on the English keyboard may not be in the same position on other keyboards. Make sure that short-cut keys are really an aid to the user and not just a convoluted set of keys to press.

Coding Practices

Have the appropriate coding practices been followed in producing the product?

Externalize Translatable Information

Are all strings externalized? It is not a good practice, long term, to simply pile all resource strings into one giant .properties file. There needs to be a strategy for separating items by type (errors, etc.) as well as by Internationalization effort. There are several types of strings in a product:

1. Company Owned and Translatable. An example would be error messages. The proper model is not “change the .properties” entry. The proper model is to provide an extensible error mechanism to allow the addition of more messages.
2. Company Owned and not Translatable. An example would be any string used as a key in a lookup or logic point.
3. Non-localizable resource strings. An example would be any HTML tags that are not locale sensitive (which is most of them).
4. User Modifiable and Translatable. An example would be the explanatory text at the top of the a UI page and any error messages that cannot be locked up and owned by the product.
5. User Modifiable and not Translatable. An example would be HTML tags that are locale sensitive or that are likely to need modification by the user. For example, font-face needs to be configurable to allow for items like Japanese characters.

Consider designing and implementing a generic resource/string manager. When your needs go beyond the default “1 world per VM” functionality in Java, treating a suite of products like an Application will not result in an International friendly product. This generic resource/string manager, no matter who implements it, could be a platform level service. Possibly a standalone servlet if it makes sense.

Is the text externalized from the images? Text included on images can be a very labor intensive item in localization. The text should be isolated from the graphic by either removing it completely or by placing it on a layer of the graphic which is easily accessible.

Are characters processed correctly for all relevant locales (Unicode)? You must use the appropriate functions and operations to get the next character in a string. Incrementing a pointer by 1 is not an option when dealing with double-byte characters. Characters can be mixed single-byte and double-byte characters in a single string.

Are all fonts externalized? Hard coded font faces in generated HTML is problematic at best. With the set chosen you are probably covered for European locales but not for Pacific Rim. In the long run, hard coding a font face is not International friendly and you will need to investigate mechanisms for generating locale specific HTML from servlets and JSPs. Over the short haul you could just make sure that the font tags are in localizable resources and that the fonts specified are appropriate for the locale. The user can (at least in Netscape) override the font choices you make so that may provide you with another short term solution.

Is string concatenation avoided? String concatenation is a bane to localizers. When faced with a partial string the translator does not know how to handle the translation because there is no way to know what comes next. Even if the translator works directly in the source files (not a good idea) the fragments of strings can be a big problem for word order. This only gets worse when the fragments are used more than once to make up different strings.

Is all character encoding externalized?

Do all error messages have a unique identifier? This is good, if for no other reason than it makes life a little easier when trying to debug a user’s issue. If the user can tell you which message number is shown you can find it and determine what it says without relying on the user’s ability to translate it. Generally, when a user tries to translate a message on the fly you get something like “well the message has something to do with the program not working right”. Well that is true of all error messages so it really did not help you much. If the user can say that message number E1042 followed by message I2388 is shown on the console then you have exactly what you need to find out where the issue occurred (if you do a good job of recording errors). This also means that you can not do the sloppy general catchall messages so popular with hackers.

Databases and Files

Is there a database storage schema for multilingual deployment?

Are all strings or files being parsed as true Unicode?

Are there any limitations in file operations, such as constraints in regards to valid file and directory names?

Processing at the Client

Does adjusting regional settings on the client side cause any problems? The problems could occur on either the client side or the server side of the application, so remember to check it both ways.

Are there any modules or fragments of code, which do not fully support Unicode?

Have accessibility issues been addressed? Sometimes addressing issues dealing with accessibility can lead you to novel ways to handle linguistic and internationalization issues.

Code Organization

Is there one set of source code for all language editions? One set of source code and a build environment to handle the various language editions is better, in the long run, than having a separate set of code for each language.

Installation

Does the product correctly install and uninstall in all the chosen locales?

Can the application be installed in all locales?

Does the installation procedure allow for a choice of languages for the user? The language of the installer does not need to be the same as the target locale. Consider that someone may be installing the product in a locale where they are not proficient in the language of the locale. It is best to give the install instructions in a language that the person installing the product understands.

Can the application be completely un installed in all locales? Again consider that the person who is un installing the product may not be proficient in the language of the locale.

International Functionality

Does the product provide all of its intended functions in all of the chosen locales?

Loading Data

Does the data load work in all chosen locales (including double-byte locales)? Data load here refers to the loading of data from a file. This can be very useful for testing and quick setup even if it is not useful to the customer.

Reloading data is also another concern. You should be able to remove the data and then load it again. Again this is handy for those folks who must test the system multiple times.

Is input from the most appropriate Input Method Editor supported for the most relevant locales (including double-byte locales)?

Does input of data in forms work in all relevant locales (including double-byte locales)?

Does input of date and time information work in all relevant locales (including double-byte locales)?

Does input of numeric information work in all relevant locales (including double-byte locales)? Is the thousands separator accepted correctly? Is the decimal point marker accepted correctly?

Does input of currency information work in all relevant locales (including double-byte locales)? Is the currency symbol accepted correctly and in the correct position? In some locales the currency symbol follows the numbers.

Multilingual Support

Does the application support multiple languages on the same application server (multilingual)?

Where are you? v. Where is the Server/JVM? What matters is the locale of the end user. There isn’t a good way to get this info with 100% accuracy but you can ask the browser for the list of languages and order of preference that has been specified by the end user. You have to assume that they have also configured and installed the appropriate fonts to support their languages. Since this list can be very long it is probably a good idea to try to accommodate their primary choice and fall back to English after that. Ignore any ‘q’ values specified and if the browser tells you “*” for the language, use English as well.

Capturing this information at login and storing it with the session is good enough for most applications.

There are International tags in HTML that can be used to facilitate multi-lingual support. You should look at these tags to see how they can be used to your advantage. Not everyone reads from left-to-right, etc.

Sorting and Searching

Does sorting work in all relevant locales (including double-byte locales)? Sorting can be very tricky in some locales. Use well supported collators when appropriate or use a sorting or collating product to help.

Does searching work in all relevant locales (including double-byte locales)? Like collating, searching can be very tricky. Watch out for things like wild cards which operate on a byte basis. The smallest unit of text is a character (not a byte).

Reference: Thanks to Bill Rich from Wilandra Consulting

Internationalization of Products

This set of check lists is intended to help you in deciding which steps to take to make your part of the products ready for the international market. There are no hard and fast rules included in this document. These are only guidelines and suggested areas to examine.

Product Goals

• Enable international users to be served in their own localeEach user will have a unique locale.

The applications must be able to identify the predefined locale of each user at login time, and maintain the locale throughout the session across all related products. Additionally, a fall back procedure for unregistered users should be made, where the user is asked to specify a desired language and country, and/or it is attempted to detect the user locale from the browser.

• Support multiple locales on one application server

In addition to being able to serve multiple locales, the products need to enable this to happen from one installation, as opposed to multiple mono-locale installations of multiple application servers. This requires further internationalization of some or all of the products. The presentation layer needs to be entirely separated from the application logic in order to make this work. The other side that needs to be considered is the Database Architecture. There needs to be a design that can store identical items in several locales and serve the appropriate locale without a negative performance impact.

• Multi Currency SupportFor the European market this is a requirement.

The products need to support (at least) dual currency on a line by line basis. You should also consider how to set the exchange rate conversion between currencies as either static or dynamic, if needed. The dynamic setting requires an API to a real-time exchange rate service.

• Enable International Search Functionality

All products offering search functionality must be able to accomplish this in all locales.

• Speed up time to market for localized products

By isolating locale specific textual data which constitute the applications’ user interface, such as page headers and trailers, user messages, form button labels, dates, currencies, etc into resource bundles, the localization effort will be far more expedient. This allows products to be localized for any given locale easily and quickly and allows the company to pull in the timeline for international releases, and build more revenue sooner.

• Avoid negatively impacting performance

A database architecture could negatively impact performance, if it is not appropriate. The proper level of due diligence must be invested in designing the appropriate way to use a database in the product.

Organization

This document is divided into several parts. Each part addresses a different area of concern when internationalizing the products. By using the navigation bar to the left you can explore each of the areas and their check lists.

There are many locales in the international arena. Only the x, y, z, and US English locales have been chosen as potential markets for the products.

The areas of concern are:

International Functionality

Does the product provide all of its intended functions in all of the chosen locales?

Installation

Does the product correctly install and uninstall in all the chosen locales?

Coding Practices

Have the appropriate coding practices been followed in producing the product?

User Interface

Does the user interface work in all the chosen locales?