Client-side translation module design

Hello all,

Currently, we don’t have a uniform way to access translations client-side.

We have two ways to fetch translation from javascript currently:

  1. This code duplication is not good
  2. The code is also strictly bound to the XWiki backend (the /rest/wikis/xwiki/localization/translations), and we’d like to generalize to make it usable in Cristal
  3. It lacks TypeScript type definition

In the case of the requirejs translation module, the keys are accessible via an object with get methods allowing access to the translation values, possibly with parameters.
In the case of Live Data, the fetched translation keys are loaded into a vue-i18n instance, the standard library to integrate translation in Vue components.

On top of to those architectural limitations, there is also a multiple performance issues.

Delay before display

Displaying a UI element is done in two steps:

  1. The JavaScript code for the UI element is fetched (e.g., from a WebJar), then interpreted by the browser and used in the UI.
  2. But, before making the UI element visible, a second request needs to be done on the translations rest endpoint to get the translations for the current locale

The first call is fine, and thanks to the ES modules, the content is cached with a long lifetime, so this is only impacting the first display of a UI element. The main concern here is to keep the size of the UI element bundle as small as possible (but this is out of scope of this proposal).
The second call to load the translations is more of an issue because it cannot be cached (translations can change server-side) and is delaying the time when the UI element can be presented to the user (as before it would be displaying translation keys).

Duplicated calls

When two live data are displayed on the same page, they both load the translations again.
This is the same query, with the same parameters done over and over again each time Live Data is used.

Goals

  • Abstract away the way translation values are resolved
  • Abstract away the way translation values are consumed

Proposed solutions

Draft of a translation module API.

/**
 * A request for translation values.
 */
type TranslationRequest = {
  /**
   * A shared prefix for all keys.
   */
  prefix?: string;
  
  /**
   * A set of key to translate.
   */
  keys: string[]
  
  /**
   * An optional locale, by default the local is found from the environment (e.g., the html element lang attribute).
   */
  locale?: string
}

/**
 * A key/value map where the keys are 
 */
type Translations = {[ket:string]:string}

/**
 * Resolve the translations with a given implementation (e.g., from a local dom, or from a remote rest endpoint)
 */
type Resolver = (request: TranslationRequest): Promise<Translations>

/**
 * Takes a set of resolvers and return a method taking a translation request and returning a set of translations.
 */
type Translator = (resolvers: Resolver[]): (request: TranslationRequest) => Promise<Translations>;

I propose to have a translation module that allows easily chaining translations resolution.
The translation module would keep track of already resolved translations and avoid reloading a translation key that is already known. This cache would last for the page duration (i.e., not kept on page refresh).
The translation resolvers would be tried in order until all requested translation keys are resolved (or the resolvers are all called.

I’m considering the following resolvers:

  1. directly from the DOM (e.g., by looking for hidden div elements with translations serialized in json in the content)
  2. from the XWiki REST endpoint
  3. from arbitrary sources by extension

The advantage of the first resolver is performance; if the translations are pre-rendered at page load, the translations can be loaded very efficiently. But this is not optimal if:

  1. The translations are pre-rendered several times
  2. The translations are pre-rendered but not used (e.g., because the UI element using them is only displayed conditionally).

The second resolver is good for UI elements that are only loaded conditionally (e.g., the notifications pane).

The arbitrary resolvers can be added incrementally in XS (e.g., a resolver that would call several resolvers in parallel) or externally when the module is used outside XWiki.

I am still looking for a way to elegantly:

  1. Handle translations resolution errors (e.g., a translation keys is not found)
  2. Bind the translation values to different translation libraries (e.g., vue-i18n, the translation methods from the requiresjs translation modules)

This is still a brainstorming text to make public what I have in mind. This is also the opportunity to raise your voice if you see missing use cases or requirements that you consider interesting to consider for the design of this module.

Thanks!

You can find a first working implementation in XWIKI-24046: Implement a module based client-side API and implementation for translations by manuelleduc · Pull Request #5311 · xwiki/xwiki-platform · GitHub

  • Definition of a @xwiki/platform-localization-api API defining the business concepts (translation, query…)
  • @xwiki/platform-localization-default to help initialize a chain of translators
  • Two resolvers
    1. a DOM based, looking up for a div element by its id and expecting for a json value in the content (currently unused)
    2. a XWiki localization rest endpoint based resolver, fetching translations from the server (with a cache + handling of concurrence to minimize the amount of back and forth with the server)
  • An adapter for Vue (not used yet but should help to localize Vue components later)
  • A re-implementation of the existing requirejs localization module using the reusable API
  • Update of the Live Data implementation to use the new API (this should remove so code duplication and speed up the performance of LD rendering)

What’s now missing is a Java component to allow rendering to define the keys they want to see inject in the dom (which should make the DOM resolver useful, and spare us a lot of localization rest endpoint call, mainly from components that are not loaded lazily).

That seems very dangerous as it means that any user can override translations via the page content or even comments, creating a very easy XSS vector if translations aren’t properly escaped on use. I would go with a script element which is designed for this purpose and for which we have proper protections in place. See MDN on Embedding data in HTML.

Would be nice if we could parse the JavaScript code on the server side to extract the translation keys when the JSX is enabled/registered (cached) and automatically inject them.