loctool, ilib-po: Add support to the loctool for saving intermediate files in po format #31

ehoogerbeets · 2024-11-27T07:13:17Z

ilib-po updates
- Added the ability to encode ResourceArray resources in a po file in a reversible way
- Added support for encoding unique keys for resources that are different than the English string
- Added support for datatypes and project names for each resource and as the default for the whole file
- Fixed a bug where you could not have resources with non-json format comments on them. Now it only parses the json format if the comment starts with '{'
loctool updates
- Added command-line switch --intermediateFormat to switch between xliff or po intermediate files
- Added IntermediateFile class with a factory method getIntermediateFile() that returns either a PO or Xliff intermediate file. The factory method can also determine the intermediate file type from the file name extension (.po or .xliff) of the requested file.
- Use the new intermediate file support to save the extracted files and the new strings files in either format
- Use the new intermediate file support to read in translation files in either format

changeset-bot · 2024-12-11T06:26:05Z

🦋 Changeset detected

Latest commit: b79d491

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
ilib-po	Minor
loctool	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

.changeset/ten-students-agree.md

packages/ilib-po/package.json

wadimw · 2024-12-16T17:10:55Z

packages/ilib-po/tsconfig.build.json

Is the formatting-only change intentional?

packages/ilib-po/types/ilib-tools-common.d.ts

wadimw · 2024-12-18T15:47:36Z

packages/ilib-po/src/Parser.ts

+            defaultDataType: string | undefined = this.datatype,
+            datatype: string | undefined,
+            defaultProject: string | undefined = this.projectName,
+            project: string | undefined;


Please remove default* local variables and use the instance-level value directly whenever you intend to fall back to it.

Can't do that -- these ones have a purpose that is different than the instance-level values. The default data type and project name can be defined in the headers of the po file and can be different than the ones specified parameters to the POFile constructor. We still need a place to store that. The default data type and project are parsed out of the header of the po file on lines 204-211. If they are not there in the file, they fall back to the values for the instance. Maybe I should call them defaultFileDataType and defaultFileProject instead?

Oh, I missed that. So basically, the value attached to the produced resource instance fallback should be 1. from unit 2. from file header 3. from parser instance right? In that case, maybe use the word file instead default (rather than both)

wadimw · 2024-12-18T16:09:18Z

packages/loctool/lib/Project.js

+        var fileFormat = this.settings.intermediateFormat || "xliff";
+        var extractedPath = path.join(dir, base + "-extracted." + fileFormat);


Extension string should be separate from the format identifier

Right now, they are exactly the same, so it is a little silly to introduce a mapping right now.

packages/loctool/test/Project.test.js

packages/loctool/package.json

packages/ilib-po/src/Generator.ts

wadimw · 2024-12-18T16:19:42Z

packages/ilib-po/src/Generator.ts

+
+    /**
+     * The name of the project that the resources belong to
+     */


Please document the default value

... where the key is different than the source string

- includes a factory method to create a new intermediate file - in the po file format generator, if the comments don't have the path, use the resource path to create the colon comment - added @jest/types so that the IDE stops complaining

- it is usually noted in the header. If it is not there, default to English

- both reading and writing with both PO and xliff files

- getIntermediateFile went into its own factory file - the subclasses of IntermediateFile get their own file - IntermediateFile.js is now just a simple abstract class

- controlled by the command-line argument

- each resource can have its own datatype represented as a #d comment - each file can have a default datatype given in the "Data-Type:" header

- each resource can have its own project name represented as a #p comment - each file can have a default project name given in the "Project:" header

- based on the intermediateFormat settings - pass the data type and project to the generator - all unit tests pass

- add src to the files in the package

- workspace:* dependencies mean that when you create a tgz file for testing, then you have to make a tgz file for a whole tree of as-yet unpublished and unrelated changes in other parts of the monorepo in order to satisfy all the dependencies. This is onerous for a simple little test. The solution is to use dependencies like workspace:^1.11.0 when you know you only need the older 1.11.0 of the other package for testing.

- moved release notes into the changesets - change dependencies into workspace:^

ehoogerbeets added the do not merge label Nov 27, 2024

ehoogerbeets requested review from nmkedziora and wadimw November 27, 2024 07:13

ehoogerbeets self-assigned this Nov 27, 2024

ehoogerbeets force-pushed the loctool-add-po-support branch from 145560e to 4d4ab4c Compare December 5, 2024 02:16

ehoogerbeets changed the title ~~Add support to the loctool for saving intermediate files in po format~~ loctool, ilib-po: Add support to the loctool for saving intermediate files in po format Dec 5, 2024

ehoogerbeets force-pushed the loctool-add-po-support branch from f0a2ad0 to 36d7c09 Compare December 5, 2024 18:36

ehoogerbeets marked this pull request as ready for review December 7, 2024 01:42

ehoogerbeets removed the do not merge label Dec 7, 2024

ehoogerbeets force-pushed the loctool-add-po-support branch 2 times, most recently from e8ddbab to 022fa23 Compare December 11, 2024 06:26

This was referenced Dec 12, 2024

Fix incomplete package entrypoint definitions #42

Merged

Fixed the jest version of many packages to 29.7.0 so that they won't have a problem with ilib-po #36

Merged

ehoogerbeets force-pushed the loctool-add-po-support branch from 9a7cd29 to 4bfd09f Compare December 13, 2024 19:38

wadimw reviewed Dec 18, 2024

View reviewed changes

ehoogerbeets force-pushed the loctool-add-po-support branch from 4fca28f to d056255 Compare December 18, 2024 23:27

wadimw approved these changes Dec 19, 2024

View reviewed changes

ehoogerbeets added 12 commits December 19, 2024 10:30

Add utility functions to convert between common and loctool resources

1f5b716

Added ResourceArray support to ilib-po

20d580a

Add support in the generator for proper keys

4afd262

... where the key is different than the source string

Add the ability to glean the target locale from the file contents

1c636be

- it is usually noted in the header. If it is not there, default to English

Unit tests work for intermediate files

4cf5c17

- both reading and writing with both PO and xliff files

Split the IntermediateFile file for better OO design

8702f3b

- getIntermediateFile went into its own factory file - the subclasses of IntermediateFile get their own file - IntermediateFile.js is now just a simple abstract class

Add the ability to write out intermediate files as po or xliff

3b9e1b6

- controlled by the command-line argument

Checkpoint commit. Not all tests work yet.

0b2ef45

Added support for datatypes

eded43a

- each resource can have its own datatype represented as a #d comment - each file can have a default datatype given in the "Data-Type:" header

Add support for project names

20e882b

- each resource can have its own project name represented as a #p comment - each file can have a default project name given in the "Project:" header

Add ability to write to po or xliff intermediate files

1598968

- based on the intermediateFormat settings - pass the data type and project to the generator - all unit tests pass

ehoogerbeets added 12 commits December 19, 2024 10:30

Redoing the lock file to make it work properly

abbedd0

Fix typescript problems

c4c02ae

Remove all the debugger statements

5435ac7

Add a few more files to clean up

0515e7f

Updated ilib-po docs

799db37

Remove the unnecessary .gitignore

39abd50

- add src to the files in the package

auto reformatted

a472bac

Added changesets

ef12d40

- moved release notes into the changesets - change dependencies into workspace:^

Address review comments

df2af38

Address last review comment

bbaba6e

Regenerate the docs on top of the rebased code

b79d491

ehoogerbeets force-pushed the loctool-add-po-support branch from 7eee01d to b79d491 Compare December 19, 2024 18:32

ehoogerbeets merged commit 0672f4c into main Dec 19, 2024
1 check passed

ehoogerbeets deleted the loctool-add-po-support branch December 19, 2024 18:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

loctool, ilib-po: Add support to the loctool for saving intermediate files in po format #31

loctool, ilib-po: Add support to the loctool for saving intermediate files in po format #31

ehoogerbeets commented Nov 27, 2024 •

edited

Loading

changeset-bot bot commented Dec 11, 2024 •

edited

Loading

wadimw Dec 16, 2024

wadimw Dec 18, 2024

ehoogerbeets Dec 18, 2024

wadimw Dec 19, 2024

wadimw Dec 18, 2024

ehoogerbeets Dec 18, 2024

wadimw Dec 18, 2024

		var fileFormat = this.settings.intermediateFormat \|\| "xliff";
		var extractedPath = path.join(dir, base + "-extracted." + fileFormat);

loctool, ilib-po: Add support to the loctool for saving intermediate files in po format #31

loctool, ilib-po: Add support to the loctool for saving intermediate files in po format #31

Conversation

ehoogerbeets commented Nov 27, 2024 • edited Loading

changeset-bot bot commented Dec 11, 2024 • edited Loading

🦋 Changeset detected

wadimw Dec 16, 2024

Choose a reason for hiding this comment

wadimw Dec 18, 2024

Choose a reason for hiding this comment

ehoogerbeets Dec 18, 2024

Choose a reason for hiding this comment

wadimw Dec 19, 2024

Choose a reason for hiding this comment

wadimw Dec 18, 2024

Choose a reason for hiding this comment

ehoogerbeets Dec 18, 2024

Choose a reason for hiding this comment

wadimw Dec 18, 2024

Choose a reason for hiding this comment

ehoogerbeets commented Nov 27, 2024 •

edited

Loading

changeset-bot bot commented Dec 11, 2024 •

edited

Loading