Time is not translated across the website

[hb20007]

Exact URL where the bug happened

https://lichess.org/@/RebeccaHarris

Steps to reproduce the bug

1. Go to the profile page of a user who has spent time on TV.
2. Change the language.
3. You will notice that everything on the page is translated except for the message showing the time spent playing and on TV.

What did you expect to happen?

For example, when I changed the language to Greek, I expected to see:

Χρόνος που διατέθηκε παίζοντας: 13 μέρες, 9 ώρες και 48 λεπτά
Χρόνος στην Τηλεόραση: 6 μέρες, 2 ώρες και 41 λεπτά

What happened instead?

Χρόνος που διατέθηκε παίζοντας: 13 days, 9 hours and 48 minutes
Χρόνος στην Τηλεόραση: 6 days, 2 hours and 41 minutes

[Arihol]

Time isn’t translated in other areas as well. Tournament duration, simul host extra time etc.

[hb20007]

@Arihol You are right; I updated the issue title.

Also, I took a look here: #3382 I see that time translations are not mentioned, but maybe it falls under "Messages from server & presets in chat". If it does, maybe we can close this issue since it would be covered in the other one.

[Arihol]

@hb20007 I’m not sure it’s restricted to messages from the server. For example, in the tournament duration setting the list is predetermined and it’s still not translated.

[yafred]

It is translated (switch to French, German, Italian, Spanish) ... but not in all languages

It is based on joda time (... again) period formatter

See https://github.com/lichess-org/lila/blob/master/modules/i18n/src/main/PeriodLocales.scala

[yafred]

import org.joda.time.Period;
import org.joda.time.format.PeriodFormat;

class Test {
	public static void main(String[] args) {
		Period period = new Period(6, 3, 0, 0, 0, 0, 23, 0);
		System.out.println("fr: " + PeriodFormat.wordBased(new java.util.Locale("fr")).print(period));
		System.out.println("de: " + PeriodFormat.wordBased(new java.util.Locale("de")).print(period));
		System.out.println("nl: " + PeriodFormat.wordBased(new java.util.Locale("nl")).print(period));
		System.out.println("el: " + PeriodFormat.wordBased(new java.util.Locale("el")).print(period));
		System.out.println("he: " + PeriodFormat.wordBased(new java.util.Locale("he")).print(period));
		System.out.println("ar: " + PeriodFormat.wordBased(new java.util.Locale("ar")).print(period));
	}
}

Result:

fr: 6 années, 3 mois et 23 secondes
de: 6 Jahre, 3 Monate und 23 Sekunden
nl: 6 jaar, 3 maanden en 23 seconden
el: 6 years, 3 months and 23 seconds
he: 6 years, 3 months and 23 seconds
ar: 6 years, 3 months and 23 seconds

[Arihol]

Is it a fixed output or can it at least be adjusted for RTL languages? I think most users understand basic English time phrases but the RTL glitch is annoying:)

[hb20007]

@yafred Thanks for the info.

@Arihol I tried to check the documentation: https://www.joda.org/joda-time/apidocs/org/joda/time/format/PeriodFormat.html#wordBased-java.util.Locale-

This point seems relevant:

The words are configured in a resource bundle text file - org.joda.time.format.messages. This can be added to via the normal classpath resource bundle mechanisms.
You can add your own translation by creating messages_.properties file and adding it to the org.joda.time.format.messages path.

The available languages by default are English, Danish, Dutch, French, German, Japanese, Polish, Portuguese and Spanish.

I think it should be translated to all languages available on lichess, even though most people understand English. But yes, especially for RTL languages.

[yafred]

@Arihol @hb20007 I found how to add these resources ...

I would need a (proper) translation of
messages.properties.txt

I will submit a PR with Greek and Hebrew as a starting point

[Arihol]

You’re awesome! I’d love to help with the Hebrew translation, just tell me how:)

should I simply upload a text file with the translations in the attached format?

[yafred]

@Arihol yes, just modify the file hereabove and drag it back in a comment

As it is a one of, I don't think we need to care much about the process

[hb20007]

@yafred, @Arihol, here's the one for Greek:
messages.properties.txt

[Arihol]

Hebrew.txt

Here's the one for Hebrew

[yafred]

Here is how it looks now:

Defaulted to English

Updated for Greek

Updated for Hebrew

Are you OK with it ?

[Arihol]

Can I send an updated file with a hyphen (-) instead of a space?

Will this work with singular forms? For example, it should be
דקה 1
with the number one after the word, unlike English.

Btw, in Arabic I’m not sure this solution will work because there are different plural forms:
٣ دقائق
١٣ دقيقة

[hb20007]

@Arihol Why do you want a hyphen instead of a space?

Regarding the singular Hebrew form word order and the Arabic different plural forms, I saw an example in the documentation for Polish where they use regular expressions to have more nuanced translations. Maybe we need something like that for Hebrew and Arabic.

[yafred]

The change has been merged.
I suggest we wait and see how it looks on the live site when it's updated (it is possible we have to tune Hebrew)

@Arihol do you mean that the order (number and word) is different between singular and plural form in Hebrew ?

[hb20007]

@yafred Btw, what do you think is the best way to manage these translations? We need some kind of list of which languages have been translated and which are pending.

[yafred]

I'm not sure what the best way to manage these translation is.

For the time being, getting the message.properties via github issues is fine because these files should not change in time.

However, crowdin has the big advantage that is reaches out to translators.

Any idea is welcome

[Arihol]

@hb20007 in Hebrew ‘and’ isn’t a word, it’s a prefix, so we use hyphens before numbers with this word and other prefixes.

In singular forms, the word order is different:
“Three minutes” but “minute one”. Technically there are special forms for “two hours” etc but it’s completely optional and unnecessarily complicated.

@yafred I’ll wait for the change to be deployed, and it looks awesome for the time being so thank you!

[Arihol]

Btw, if you manage the translations through a GitHub ‘mega issue’ where people upload files for their language maybe there can be a link to the time formatting file and the GitHub issue on Crowdin (in the timeago file for example) to make it more accessible to translators.

[hb20007]

@Arihol, I am willing to create a new issue with a checklist of all languages and update the checklist regularly when we get new translations. Where exactly is the timeago file which you mentioned?

[Arihol]

https://crowdin.com/translate/lichess/170/en-he?filter=basic&value=0

[hb20007]

@Arihol Perhaps it is possible to create a new file like this one for the Joda time translations? We will need to put the whole string with the comma etc. but it will work, right?

[Arihol]

It works with the timeago strings, which require more or less the same tweaks and adjustments. @yafred what do you think?

[yafred]

Yes, timeago is handled by lichess code + crowdin

We could consider replacing joda lib (specially formatting) with a lichess module if we consider that joda (and its replacement java.time) are not 100% matching our needs

Quite a big entreprise though

[hb20007]

@ornicar Who do we contact for creating the file on crowdin for these translations? I tried to do it but it seems I do not have the option.

[Arihol]

@hb20007 thank you! Would you like me to send an updated file with the hyphenated form? The singular form tweak is relevant in many languages (Arabic is a glaring example) so I assume it’s more complex to fix

[hb20007]

@Arihol I believe the singular form tweak can be done using regex substitution. I am looking at the examples in the documentation and also trying to find a way to test it. If we can figure it out, then maybe we can do this together with the hyphen fix.

[hb20007]

I found that there are a few more (undocumented but working) translations on the joda-time repository: https://github.com/JodaOrg/joda-time/tree/main/src/main/java/org/joda/time/format Also, I see that the last release was 5 days ago so it seems that they are accepting changes.

In project/Dependencies.scala, we are using almost the latest version of joda-time so I assume that bumping the version is possible.

All this said, I think it is better to submit the translation files to the joda-time repository instead of lila. Then, hopefully we can get a new joda-time release with the translations and use it on lila.

[hb20007]

@Arihol Also, it seems that it is better to encode non-Latin characters as Unicode escapes to avoid issues: JodaOrg/joda-time@0daee12 So, I am thinking to also encode the Greek one and submit it to joda-time.

[hb20007]

@yafred Do you mind reverting your changes? I think it's better to add these translation files to joda-time directly: https://github.com/JodaOrg/joda-time/tree/main/src/main/java/org/joda/time/format

[yafred]

@hb20007 let's see if they accept the PR for Slovak ...

If they do, I agree this is the best course of action.

I also noticed the regexp in the link you mentioned above, I agree this should be applicable to Hebrew

[Arihol]

Do you think a similar pull request could fix #11569?

[yafred]

If you know how to fix it ... definitely.

If not, you can still open an issue

[Arihol]

@yafred I’ll do some research about it and see

[hb20007]

@yafred The Slovak as well as some other PRs for translations were accepted on joda-time. Also, one of them, Finnish, was accepted even though the author did not use Unicode escapes. It seems UTF-8 encoding is enough. In this case, I believe we can submit Greek as-is and then submit Hebrew once we figure out the regular expression.

[yafred]

@hb20007 That is great ... I suggest we remove the translations from lila at the same time we bump joda-time (I guess it will be 2.11.3)

[Arihol]

Can you please wait for the Hebrew translation before reverting the change? It isn’t perfect but it’s better for the time being than no translation at all

[yafred]

@Arihol of course

[hb20007]

@Arihol I checked the documentation again. Unfortunately, I do not think it is possible to switch the order of the word with the number. I see that with regex, we can only define a list of words and choose the right word based on a condition. Also, I thought about removing the number "1" in this case but even this does not seem to be possible.

I have created the pull request for Hebrew on the joda-time repository: JodaOrg/joda-time#651

[Arihol]

@hb20007 Excellent, thank you. I saw a similar comment by Niklas on Crowdin about the translation of timeago to Arabic. It makes sense and the singular issue is common in many websites.

@yafred I created an issue in the Joda repo about the date formatting issue (#11569). Based on the comments there and the relevant formatting files the language-specific locales are imported from Java and Joda doesn’t control them. Is there a way to override this on Lila or otherwise change the Hebrew locale formatting?

[hb20007]

@yafred I see that there is a new joda-time release with the translations: https://github.com/JodaOrg/joda-time/releases/tag/v2.12.0

Do you want to bump the version and remove the translations from lila?

[yafred]

@hb20007 Done in #11711

[yafred]

I'd like to understand the problem behind the rejected PR's

@niklasf what do you mean by 'not properly decoded' in JodaOrg/joda-time#504 ?

[niklasf]

It's a bit strange, but those .properties files use latin-1 to describe utf-8 encoded strings.

So for example you would want to write:

PeriodFormat.year=\ χρόνος

But that's not possible to represent in latin-1, so requires escaping:

PeriodFormat.year=\ \u03c7\u03c1\u03cc\u03bd\u03bf\u03c2

[hb20007]

@yafred Btw, the issue with the rejected PRs was later fixed in this PR: JodaOrg/joda-time#653

[yafred]

There is something I must be missing :(

Where do you see latin-1 ?
Are you sure your editor is not forcing latin-1 ?

UTF-8 is one encoding of Unicode with all its codepoints; Latin1 encodes less than 256 characters.

So there is no way latin-1 should be used in translation files

Using escaped unicode characters make the files non human-readable

[hb20007]

@yafred, As I understand it, they need to use Latin-1 because of some compatibility issues with Android. I agree that this makes the file non human-readable, which is why I tried Unicode at the beginning, which got rejected.

[Arihol]

@hb20007
It seems the hyphen is misplaced in the encoded version

The hyphen should separate the ו from the number after it

[hb20007]

@Arihol,
I remember we had a confusion about it here: JodaOrg/joda-time#651
I made a new PR with the fix: JodaOrg/joda-time#667
Thanks for reporting!

[Arihol]

Thank you!