Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ilib-lint: Add the ability to filter errors from xliff files #62

Open
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

ehoogerbeets
Copy link
Contributor

No description provided.

@ehoogerbeets ehoogerbeets self-assigned this Feb 4, 2025
Copy link

changeset-bot bot commented Feb 4, 2025

🦋 Changeset detected

Latest commit: 3ee822f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages
Name Type
ilib-lint Minor
ilib-lint-common Minor
ilib-lint-python-gnu Patch
ilib-lint-python Patch
ilib-lint-react Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@ehoogerbeets ehoogerbeets marked this pull request as ready for review February 10, 2025 22:17
Copy link
Contributor

@wadimw wadimw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed, pushing whatever comments I have so far

*/
write() {
if (this.filePath && this.isDirty()) {
const dir = path.dirname(this.filePath);
makeDirs(dir);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tentative: either fix doc for write, or I'd prefer to get rid of this completely - IMO it does not make sense for why would a file write operation to create directories in the first place

Copy link
Contributor

@wadimw wadimw Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also even if we keep it, no need to roll your own recursion:

Suggested change
makeDirs(dir);
fs.mkdirSync(path.dirname(this.filePath), {recursive: true});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the config file says to put the output in directory x/y/z and it doesn't exist, it should create it. The author of the config file intended that directory to be there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs fix needed

The path to the file must exist first.

Comment on lines +61 to +63
* @param {Array.<Result>|undefined} results the results of the rules that were applied earlier
* in the pipeline, or undefined if there are no results or if the rules have not been
* applied yet
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: undefined is redundant with empty array

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note/question: why does the transformer need to know about past results?
nit: description for results is confusing, I'd just say results of linting this @link representation

Copy link
Contributor

github-actions bot commented Feb 13, 2025

Jest Code Coverage

Code Coverage for changed files • (0%) 
File% Stmts% Branch% Funcs% LinesUncovered Line #s
   FileType.js96.5991.2210096.47164, 290–291
   LintableFile.js71.4259.6479.1674.7380, 90, 123–135, 209–233, 245–253, 262
   ParserManager.js77.4187.555.558027, 129–133, 153
   PluginManager.js92.856293.3392.7763–67, 266, 314
   Project.js65.9649.7276.7465.8269, 178, 229–230, 269–271, 296, 321, 328–329, 425–426, 493–494, 540, 602–732
   SerializerManager.js8077.2757.1485.1866, 138–151
   TransformerManager.js8079.1657.1485.1865, 137–150
   BuiltinPlugin.js100100100100 
   ErrorFilterTransformer.js100100100100 
   LineSerializer.js50100505047–50
   XliffParser.js100100100100 
   XliffSerializer.js100100100100 
   StringFixCommand.js100100100100 
   StringParser.js750807567–70
   StringSerializer.js57.1405057.1450–52
   DeclarativeResourceRule.js81.3979.0666.6684.21110–111, 114–117
   LineRegexpChecker.js9490.6288.8895.7468, 112
   ResourceDNTTerms.js90.4786.9510089.4765, 78, 133, 136
   ResourceEdgeWhitespace.js96.7795.4510096.55112
   ResourceICUPluralTranslation.js91.7688.6310093.5879–80, 165, 181, 213
   ResourceRule.js10092.590100100, 129, 142
   ResourceSourceChecker.js83.337510083.3372–74, 92
   ResourceTargetChecker.js83.337510083.3373–75, 94
Title Lines Statements Branches Functions
ilib-lint-python Coverage: 73%
73.43% (141/192) 60.46% (52/86) 62.06% (18/29)
ilib-lint-python-gnu Coverage: 76%
75.15% (118/157) 71.18% (42/59) 60.97% (25/41)
ilib-lint-react Coverage: 96%
95.88% (303/316) 84.91% (152/179) 91.04% (61/67)
ilib-lint Coverage: 86%
85.02% (1658/1950) 75.59% (914/1209) 86.88% (338/389)
loctool Coverage: 81%
81.53% (6638/8141) 74.29% (3370/4536) 72.78% (757/1040)
Title Tests Skipped Failures Errors Time
ilib-lint-python 52 0 💤 0 ❌ 0 🔥 4.019s ⏱️
ilib-lint-python-gnu 35 0 💤 0 ❌ 0 🔥 2.703s ⏱️
ilib-lint-react 111 12 💤 0 ❌ 0 🔥 11.148s ⏱️
ilib-lint 700 0 💤 0 ❌ 0 🔥 10.039s ⏱️
loctool 2296 81 💤 0 ❌ 0 🔥 1m 19s ⏱️

@wadimw wadimw force-pushed the linterFilterErrors branch 2 times, most recently from cef2dd3 to 7208faf Compare February 14, 2025 19:55
- filters out Resource instances that have error Results after linting
- the file stats are already encoded in the intermediate representation
- describe the new command-line parameters for writing files out,
  applying auto fixes, and overwriting source file
- add description to the configuration to help people understand
  how the configuration works
- update configuration to allow for transformers and serializers
  to be defined in the file type
- no code change so far
Will need some more work here...
- throws exceptions now if you have a plugin for a different representation
- more work on the serializer at the end (not working yet)
- updated copyright years for changed files to 2025
- ilib convention is to put tests in a top-level test directory, not
  interspersed throughout the source code in many different __test__
  directories
- SourceFile can now make the intermediate directories before writing
  out the file
- Added unit tests for SourceFile.write
@ehoogerbeets
Copy link
Contributor Author

Ready for re-review

sourceFile: jest.fn(() => "sourceFile")
});

const transformed = eft.transform(representation, []);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about the case when results is undefined?
If I understand correctly, this is possible, as per the ilib-lint docs:

Rules interpret the intermediate representation of a file produced by a Parser and produce a single Result instance, an array of Result instances, one for each problem found, or undefined if there are no problems found.

}
const resources = ir.getRepresentation();
const idsToExclude = results.filter(result => result.id && result.severity === 'error').map(result => result.id);
const filteredResources = resources.filter(resource => !idsToExclude.includes(resource.getKey()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test for filtering out. This is one of the crucial aspects of the mechanics, and it is essential to test it explicitly so that errors can be identified if the filter logic is messed up.

- getFormatters - return an array of classes that inherit from the Formatter class
- getTransformers - return an array of classes that inherit from the Transformer class
- getSerializers - return an array of classes that inherit from the Serializer class
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getFixers
getFormatters
getTransformers
👇
Does the order matter? If so, what's the correct order?
Should we add information about the ordering here?

* by type
* @private
*/
byType = {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need three caches that store the same information? Since we already have serializerCache, we can retrieve the serializer's name and description by calling serializer.getName() and serializer.getDescription() anytime.

Having only one cache simplifies the system, reduces redundancy, and makes it easier to maintain, as we only need to manage a single source of truth for the data.

*/
add(serializers) {
if (!serializers || !Array.isArray(serializers)) return;
for (const ser of serializers) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be simplified if you used only serializerCache for caching serializers (and their details).

*
* @returns {Object} the serializer names and descriptions
*/
getDescriptions() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By having only one cache, you could use it here like this (pseudocode):

getDescriptions() {
 return this.serializerCache.map(serializer => serializer.getDescription())
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants