Making a Drupal Module Fully Translatable

Drupal 6. Please note that most of this is sound advice but that some of it is still being debated as far as what is best practice, specifically how to ensure that exportable structures are translatable. I encourage you to leave and read the comments.

The Tools

Drupal's Core Function: `t()`

If you have written a module, you should be familiar with t(). Almost every interace string you write in your module should be wrapped with t(). This function creates a mechanism so that the core module, locale, can offer translations for non-English languages. Without using the t() function, Drupal would have no idea what strings are translatable and this would be very limiting for sites that were not in English (or not just in English).

The limiting nature of t() is that there is no identification on strings. This means that Drupal is not really keeping track of changes in a string and there is no way to remove old strings that are no longer needed on the site. This is a bad thing for user-defined strings, strings that are entered into the interface, for example the title of a menu item, since it can change often.

The i18n Module

The i18n module (i18n is used based on the number of characters in the word internationalization), offers a set of modules to make Drupal a much better platform for multilingual sites. It offers the ability to translate some of the main structures of Drupal, like menus, taxonomies, blocks, and variables, as well as a more usable interface for translating.

The i18n module also offers a mechanism for translating strings that are based on identifiers for strings. The i18nstrings($name, $string, $langcode = NULL) function allows for better management of user-defined strings.

The Problems

Translatable Strings in Code

Though this problem is solved in core, it is still valid to point out in all of this. Interface strings (and other messages) that are defined in code, need to be translatable.

User-Defined Strings

Interface strings and messages that are inputted by the user need to be able to be translated. These almost always live in the database. The main issue here is that there needs to be a mechanism to identify each inputted string so that changes can be maintained properly.

Denoting Translatability

With modules that utilize a flexible plugin architecture or other dynamic system, one big issue is how to track what fields (or other data points) need to be translatable. This is very important to ensure that making something translatable is not hard-coded.

Exported Data

Recently (well, Views has done this well for a while), there has been a lot of work towards making Drupal structures that have the ability to be imported and exported. The main benefit of this is that it allows for setting-type structures to live in and be maintained in code. In turn, this means that a data structure that may contain interface strings and messages can be stored in either code or in the database.

The Solutions

The `t()` Function

The Drupal core t() function handles the ability to translate static strings within module code. For instance:

  '#description' => t('This is an example for the t() function.'),

i18n Module

The i18n module provides the i18nstrings() function to allow a module to translate its user-defined strings (see below about Drupal variables). Please see Translating user defined strings for module developers for more information and example code. Do keep in mind that you will want to create a wrapper function, unless you want your module to depend on the i18n module. For example:

function yourmodule_translate($name, $string, $langcode = NULL, $textgroup = 'yourmodule') {
  // Check for existence of i18nstrings function, then
  // translate if available.
  return function_exists('i18nstrings') ? 
    i18nstrings($textgroup . ':' . $name, $string, $langcode) : 
    $string;  
}

The i18n module also provides a way to more easily translate variables that your module may store. The mechanism that tells the i18n about your variables is a variable itself, $conf['i18n_variables'], usually set in the settings.php file. Your module can circumvent this, by utilizing hook_init(). For example:

function yourmodule_init() {
  // Get i18n variable array.
  global $conf;
  
  // Add your module's translatable variables
  $conf['i18n_variables'][] = 'yourmodule_variable_1';
  $conf['i18n_variables'][] = 'yourmodule_variable_2';
}

Schema or Plugin Notation

For Views 3, there is a discussion on putting in a plugin system for translators. This is pretty awesome, though may be overkill for most modules. But more specifically this issue also deals with how to denote things as translatable so that they can be automatically ran through the given translation system.

The best solution here is hard to define. It is very dependent on architecture. If you are using a database table for specific data structures (like with CTools), it would probably be best to simply provide a 'translatable' => TRUE field to the schema that can be referenced when displaying values. For example:

       'description' => array(
         'type' => 'text',
         'not null' => TRUE,
         'description' => 'Layer description.',
         'translatable' => TRUE,
       ),

But this still poses some questions like: How to handle serialized data? Handling customized forms?

Managing `t()` in Exports

This is the area that seems to most questionable in my mind. Most of these ideas and the proposed solution is taken from a discussion for the OpenLayers module.

The problem with exports is that they are exported as code and can be used either as code or imported back into the system via the interface. At least this is the common use of exportable structures. So, we need a way that ensures that structures in code are translatable and structures in the database are translatable, but without overlapping the two.

My proposed solution is then this:

Provide export with the t() embedded into it; this will be determined by the mark of translatability. This ensures that structures that live as code are translatable.
On import, strip out the t().
Finally, on display, check where the structure lives, and if in database, then translate (taking into account everything above).

You might say that: Why not just translate on display? The problem here, as I see it, is that there are probably already structures living code that have not been exported, like default Views or OpenLayers layers, and taking these out of the usual Drupal t() paradigm is dangerous. Also, this means that structures are translatable without a third-party module.

In Conclusion

I hope this makes everyone think about ensuring that their modules are fully translatable because the world is full of non-English internet users.

I'll also be the first to admit that I am not doing this will all my modules. It took me a long time to research most of this information. I think it is important to start to work towards making these methods standardized and more obvious to module developers.

It's also important to look towards what is happening in Drupal 7 and how this might influence these structures. Honestly, I am in the dark as far as what changes are happening in D7 as far as localization goes.

Jake wrote 13 weeks 2 days ago

Good post!

yhahn wrote 13 weeks 2 days ago

Great writeup zzolo. I think

Great writeup zzolo. I think this gives a great overview of what some of the challenges are to translatability and what some of the key points to consider are when designing your module.

I would like to make some observations about your suggestions about managing exports, coming from the perspective of someone who has worked on exportables and the marathon Views 3.x localization patch.

Your suggestion looks like this:

Wrap translatables inline with t() in exports
Only localize translatables at runtime if an object lives in the database
Strip out instances of inline t() in exports when importing objects

From working with exportables that need translation in Open Atrium, what I've found to be best practice is actually slightly different and is what is currently implemented in the Views 3.x patch:

Always localize translatables at runtime for an object
Don't wrap translatables inline with t() but instead provide a separate $translatables export array which can be easily detected/parsed by potx and other string extractors

This greatly simplifies logic needed at various points in the system. For example, you don't need to strip inline instances of t() as there are no inline instances in exports. In addition, there is also no need to check whether an object is in the database or an export before localizing at runtime.

In the long run, there is no question that better infrastructure is needed in core. In addition, I think eventually modules implementing exportables will begin to use a pattern similar to that in Views for packing/unpacking properties so that metadata for tagging translatable strings can be used. If a best practice becomes widespread enough I can see CTools providing a base class for exportables implementing a lot of this infrastructure.

Gábor Hojtsy wrote 13 weeks 1 day ago

yhahn says it well

I think yhahn has good thinking around how this should be handled. Passing around English and translating on demand is way more manageable then trying to track whether what you have is not English and not translating it (especially, if you'd need it another language).

zzolo wrote 13 weeks 1 day ago

Thanks for the comments

Thanks for the comments. @yhahn, I think that part was added after I read most of that ticket, though this is what @tmcw was proposing for OpenLayers. I do think that both methods are hacks and that the real problem is infrastructure to support this.

I do agree that the method you describe is a little simpler, but I don't actually like the method of translating all strings at runtime because it totally strips the use of t() in code. This means that if someone writes an exportable from scratch, they will have to be aware to not wrap what they normally would in t(). Also, this method duplicates strings in code. But, as I stated above, both are hacks, and I am sure that you have thought about this more than I have.

@Gábor, I am confused on how my proposed method creates the situation of translating non-English strings. There is no logic to determine language in my proposal. The idea is to create an "automated" way to provide t() to coded exports.

eojthebrave wrote 13 weeks 1 day ago