Translator version 0.2.6 Beta 2

Adds ETA

Translator version 0.2.5 Beta 2

Fix Beta 1 of version 0.2.5

Translator version 0.2.5 Beta 1

Fixes more issues and implements a handy way to refer to languages.

Full Changelog: v0.2.3b1...v0.2.5b1

Translator version 0.2.3 Beta 1

Fix various bugs with previous version. Why take the risk to encounter them when you have this stable version now?

This version keeps all the improvements of previous versions without the bugs. What are you waiting to upgrade your Translator?

Full Changelog: v0.2.2b1...v0.2.3b1

Translator version 0.2.2 Beta 1

Such speed wow!

Full Changelog: v0.1.3b4...v0.2.2b1

Translator version 0.1.3 Beta 4

This is the lowest stable version you should use if you seriously care about your translations.

Updated command interface, fix load data from files, fix translation buffer, fix UX, logs and various improvements towards stability, efficiency with large translation.

This is a test to translate MLS and Wikipédia (~13 GB) from english to french.

❯ ls
 en2fr.eng_Latn.fra_Latn.tmp.cache   en2fr.txt   LICENSE   mls_lm.txt   wiki_en_lower.txt
❯ cat LICENSE | head -n 12
Creative Commons Legal Code

CC0 1.0 Universal

    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
    HEREUNDER.
❯ LANG="en_US.UTF_8" translate --version
Translator version: 0.1.3b4
❯ file en2fr.txt
en2fr.txt: Unicode text, UTF-8 text
❯ du -h *.txt
32K     en2fr.txt
13G     mls_lm.txt
240M    wiki_en_lower.txt
❯ translate -d . -S en2fr.txt eng_Latn fra_Latn
ℹ Preparing to translate...
ℹ No sentence was given but directory was provided.
ℹ Using cuda:0 to translate sentences in eng_Latn to fra_Latn from text files in directory '.' by batches of size 128.
ℹ Loading all sentences...
ℹ Found 2 text files.
Found cached dataset text (/mnt/Data_II/Données/Translation/en2fr.eng_Latn.fra_Latn.tmp.cache/text/default-fbf3e5d3bc11ec4e/0.0.0/cb1e9bd71a82ad27976be3b12b407850fe2837d80c22c5e03a28949843a8ace2)
ℹ RAM memory used by dataset: 8374.3671875 MB
ℹ Translating 13694289051 sentences...
ℹ Loading translated sentences...
Downloading and preparing dataset text/default to /mnt/Data_II/Données/Translation/en2fr.eng_Latn.fra_Latn.tmp.cache/text/default-28ba15b78d688bb3/0.0.0/cb1e9bd71a82ad27976be3b12b407850fe2837d80c22c5e03a28949843a8ace2...
Downloading data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 6492.73it/s]
Extracting data files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1287.78it/s]
Dataset text downloaded and prepared to /mnt/Data_II/Données/Translation/en2fr.eng_Latn.fra_Latn.tmp.cache/text/default-28ba15b78d688bb3/0.0.0/cb1e9bd71a82ad27976be3b12b407850fe2837d80c22c5e03a28949843a8ace2. Subsequent calls will reuse this data.
ℹ Translated 512 sentences already.
ℹ Took 0.5572784369869623 second(s) to load translated sentences.
ℹ Loading translated sentences...
ℹ Filtering untranslated sentences...
ℹ Took 896.2490978440037 second(s) to compute untranslated sentences.
ℹ Translating untranslated sentences...
^C
⚠ You are about to loose your progress!
ℹ Partial translation has been saved.

Full Changelog: v0.1.1b2...v0.1.3b4

Translator version 0.1.1 Beta 2

This version of Translator is fast and efficient. It leverages the GPU and CPU to load and translate sentences as quickly as possible.
For large files translation, cache and interruption buffer have been implemented so that you never loose any progress on your translation work.

Full Changelog: v0.0.1b3...v0.1.1b2

Translator version 0.0.1 Beta 3

This version introduces the first stable beta of Translator. It speeds batch translations by leveraging GPU. Improving translation time by a factor of at least ten.

Release version Alpha 8 (0.0.0a8)

Alpha 8 is the last stable Alpha version.

It will be the latest stable version until version Beta 1.

You can always grab main but if you intend to translate seriously, better use a stable version.

Full Changelog: https://github.com/wasertech/Translator/commits/v0.0.0a8