RULEAPI-826 Compute rule / product mapping #4575

frederic-tingaud-sonarsource · 2024-12-12T17:14:50Z

No description provided.

rspec-tools/rspec_tools/coverage.py

marco-antognini-sonarsource · 2024-12-13T11:08:34Z

rspec-tools/rspec_tools/coverage.py

+def collect_coverage_per_product(rules_dir):
+  git_repo = checkout_repo('sonar-enterprise')
+  bundle_map = get_packaged_plugins(git_repo)
+  tags = git_repo.tags
+  tags.sort(key = lambda t: t.commit.committed_date)
+  versions = [tag.name for tag in tags if is_interesting_version(tag.name)]
+  plugin_versions = {}
+  for version in versions:
+    plugin_versions[version] = get_plugin_versions(git_repo, version)
+  build_rule_per_product(rules_dir, bundle_map, plugin_versions)


I'm concerned we do not sort things consistently with collect_coverage_for_all_versions (committed date vs tag).
Maybe the rest of the code handles this nicely, in which case I would remove this sort and test the implementation with a shuffle to make sure it's resilient.

I also think we can have a lightweight test here that covers scenarios like:

SQS 2025.1 is released with plugin X.1 and thus S1

(Assume 2025.1 is an LTA)

SQS 2025.2 is released with plugin X.2 and drops S1

bugfix SQS 2025.1.1 is released still with X.1 (or X.1.1) and with S1

or this one:

SQS 2025.1 is released with plugin X.1 and no S1

(Assume 2025.1 is an LTA)

SQS 2025.2 is released with plugin X.2 and introduces S1

bugfix SQS 2025.1.1 is released still with X.1 (or X.1.1) and thus without S1

In terms of test implementation, I would suggest:

pass the git_repo as a parameter

create a fake sonar-entreprise for a couple of scenarios -- we just need tags & specific files.

create a fake plugin coverage JSON file for those scenarios

a big of pytest magic should be enough -- we may not even need to mock/patch anything

I appreciate we do not want complicated unit tests for each function (although it would be nice), but we can strike a balance between having nothing and something that cover the main corner cases from end-to-end.

Tests will be covered by RULEAPI-828

rspec-tools/rspec_tools/coverage.py

rspec-tools/rspec_tools/repo_plugin_mapping.json

rspec-tools/rspec_tools/coverage.py

marco-antognini-sonarsource · 2024-12-16T09:04:41Z

rspec-tools/rspec_tools/coverage.py

+    product_per_rule_per_lang['SonarQube Server'] = {
+      'minimal-edition': EDITIONS[0],
+      'since-version': lowest_server_version(plugin_versions, plugin, version)
+      }


We'll get a hard error (thanks to the sys.exit) if a rule is released for the first time in a Community Build (i.e., it is not yet released in a SQS version).

Oh, good point!
One cool thing we could do is handle it gracefully and return something like "Coming soon" or "Next release" in that case.

I would suggest keeping this as-is for now and fixing this with RULEAPI-830 and a proper test (i.e., after RULEAPI-828) to focus on the essentials right now.

I made the change because it is a scenario that is supposed to happen very quickly after the release of the first LTA, as SQ Community Builds have a shorter cycle than SQ Enterprise.
I also realized that there was a bug in how we handled "master" branches, which once fixed, would trigger the same exception.

marco-antognini-sonarsource · 2024-12-16T14:21:45Z

rspec-tools/rspec_tools/coverage.py

@@ -49,6 +49,12 @@

 RULES_FILENAME = 'covered_rules.json'

+DEPENDENCY_RE = re.compile(r'''\bdependency\s+['"](?:com|org)\.sonarsource\.[\w.-]+:(?P<plugin_name>[\w-]+):(?P<version>\d+(\.\d+)+)['"]''')


Right now, in the code, we say major >= 8 however this regex does not match for all plugins.

Example:
https://github.com/SonarSource/sonar-enterprise/blob/f869d840f12a4dfa69533e17d3b27558617a7f82/build.gradle#L128C35-L128C41

9.x looks compatible though: https://github.com/SonarSource/sonar-enterprise/blob/ce743843018827cb30c05735e7971523de34bb48/build.gradle#L171-L209

Starting 10.0 we have an entry that does not match the regex but we should not care about this one so that's fine: https://github.com/SonarSource/sonar-enterprise/blob/75dae6a655f49ac78b0a786f6c19b15085561af9/build.gradle#L218

10.1 introduced a second one, but again we should be good. https://github.com/SonarSource/sonar-enterprise/blob/53c01c35c264c7e3d76cf5fb955de406f36b115e/build.gradle#L254-L255

I've pushed 3b74531 to:

support >=9 only

detect missing versions if variables are re-introduced for plugins in the future.

If we need older coverage, we can accommodate that in a later PR. @frederic-tingaud-sonarsource can you validate this is OK for now?

Thanks for catching these! It looks good to me.

Loosen regex to detect unexpectedly missing version in the future.

…es in mapping

sonarqube-next · 2024-12-17T11:00:38Z

Quality Gate failed for 'rspec-tools'

Failed conditions
28.4% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

sonarqube-next · 2024-12-17T11:00:45Z

Quality Gate passed for 'rspec-frontend'

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

frederic-tingaud-sonarsource · 2024-12-17T17:29:12Z

Your changes look good to me. Feel free to merge.

marco-antognini-sonarsource · 2024-12-18T15:09:06Z

rspec-tools/rspec_tools/coverage.py

@@ -300,7 +300,7 @@ def lowest_version(plugin_versions, plugin, version, skip_suffix):
      pvv = plugin_versions[t][plugin]
      if comparable_version(pvv) >= comparable_version(version):
        return t
-  sys.exit(f'failed finding the oldest version of {plugin} containing {version} from {plugin_versions}')
+  return "Coming soon"


I'm not sure we want a "coming soon" fake version. It would be better to not give such version (i.e., make both SonarQube Community Build and SonarQube Server optional).

See also my comment here: https://sonarsource.atlassian.net/browse/SONARSITE-2248?focusedCommentId=709499

marco-antognini-sonarsource · 2024-12-18T16:04:28Z

rspec-tools/rspec_tools/coverage.py

+        # The rule has an "until", therefore it does not exist anymore
+        # and should not appear in the product mapping.
+        continue
+      target_repo, version = since.split(' ')


We should probably skip the master version here since it's not available in the product anyway and we do not look at master versions of sonar-enterprise.

marco-antognini-sonarsource · 2024-12-18T16:25:12Z

rspec-tools/rspec_tools/coverage.py

I had a look at the data we produce and there is something fishy. I'll hold off merging this because I don't want the script to cause trouble during the holidays when we are not there -- it would not be a nice Xmas gift.

Looking at S100 (very old rule), we say:

for C/C++/ObjC: introduced in 9.0 Dev edition ✅

for Apex: enterprise edition (probably correct), introduced in 10.7: does not match reality SQS report it being there since Nov 02, 2018.

for C#: Dev edition (sounds correct) but introduced in SQS 10.8 / sqcb-24.12 but SQ reports the rule being introduced in 2015...

I don't know where we do things incorrectly...

RULEAPI-816 Create a rule / product mapping file

5f99700

frederic-tingaud-sonarsource requested a review from marco-antognini-sonarsource December 12, 2024 17:14

frederic-tingaud-sonarsource added the rspec system Related to handling of rspecs (scripts, doc...) label Dec 12, 2024

marco-antognini-sonarsource requested changes Dec 13, 2024

View reviewed changes

frederic-tingaud-sonarsource added 2 commits December 13, 2024 18:37

Apply suggestions

755ffff

Merge remote-tracking branch 'origin/master' into ft/rule-mapping

4b0392e

marco-antognini-sonarsource reviewed Dec 16, 2024

View reviewed changes

rspec-tools/rspec_tools/coverage.py Outdated Show resolved Hide resolved

marco-antognini-sonarsource reviewed Dec 16, 2024

View reviewed changes

rspec-tools/rspec_tools/coverage.py Show resolved Hide resolved

marco-antognini-sonarsource reviewed Dec 16, 2024

View reviewed changes

rspec-tools/rspec_tools/coverage.py Outdated Show resolved Hide resolved

marco-antognini-sonarsource reviewed Dec 16, 2024

View reviewed changes

rspec-tools/rspec_tools/coverage.py Outdated Show resolved Hide resolved

marco-antognini-sonarsource reviewed Dec 16, 2024

View reviewed changes

marco-antognini-sonarsource changed the title ~~RULEAPI-816 Create a rule / product mapping file~~ RULEAPI-826 Compute rule / product mapping Dec 16, 2024

marco-antognini-sonarsource mentioned this pull request Dec 16, 2024

RULEAPI-827 Update coverage nightly script to also generate the product mapping #4576

Open

marco-antognini-sonarsource added 4 commits December 16, 2024 11:58

Update rspec-tools/rspec_tools/coverage.py

1abfb8f

Simplify regex: drop unnecessary groups

e056c0f

Make comparable_version more linear

f5d9bbf

Simplify how regex groups are accessed

69127ba

marco-antognini-sonarsource reviewed Dec 16, 2024

View reviewed changes

marco-antognini-sonarsource and others added 2 commits December 16, 2024 15:38

Only support tags >= 9

3b74531

Loosen regex to detect unexpectedly missing version in the future.

Fix version for unreleased rules and handle gracefully unreleased rul…

af5edc7

…es in mapping

marco-antognini-sonarsource requested changes Dec 18, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RULEAPI-826 Compute rule / product mapping #4575

RULEAPI-826 Compute rule / product mapping #4575

frederic-tingaud-sonarsource commented Dec 12, 2024

marco-antognini-sonarsource Dec 13, 2024 •

edited

Loading

marco-antognini-sonarsource Dec 16, 2024

marco-antognini-sonarsource Dec 16, 2024 •

edited

Loading

frederic-tingaud-sonarsource Dec 16, 2024

marco-antognini-sonarsource Dec 16, 2024 •

edited

Loading

frederic-tingaud-sonarsource Dec 17, 2024

marco-antognini-sonarsource Dec 16, 2024

marco-antognini-sonarsource Dec 16, 2024

marco-antognini-sonarsource Dec 16, 2024

marco-antognini-sonarsource Dec 16, 2024

frederic-tingaud-sonarsource Dec 16, 2024

sonarqube-next bot commented Dec 17, 2024

sonarqube-next bot commented Dec 17, 2024

frederic-tingaud-sonarsource commented Dec 17, 2024

marco-antognini-sonarsource Dec 18, 2024

marco-antognini-sonarsource Dec 18, 2024

marco-antognini-sonarsource Dec 18, 2024

		@@ -49,6 +49,12 @@

		RULES_FILENAME = 'covered_rules.json'

		DEPENDENCY_RE = re.compile(r'''\bdependency\s+['"](?:com\|org)\.sonarsource\.[\w.-]+:(?P<plugin_name>[\w-]+):(?P<version>\d+(\.\d+)+)['"]''')

RULEAPI-826 Compute rule / product mapping #4575

Are you sure you want to change the base?

RULEAPI-826 Compute rule / product mapping #4575

Conversation

frederic-tingaud-sonarsource commented Dec 12, 2024

marco-antognini-sonarsource Dec 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marco-antognini-sonarsource Dec 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marco-antognini-sonarsource Dec 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqube-next bot commented Dec 17, 2024

Quality Gate failed for 'rspec-tools'

sonarqube-next bot commented Dec 17, 2024

Quality Gate passed for 'rspec-frontend'

frederic-tingaud-sonarsource commented Dec 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marco-antognini-sonarsource Dec 13, 2024 •

edited

Loading

marco-antognini-sonarsource Dec 16, 2024 •

edited

Loading

marco-antognini-sonarsource Dec 16, 2024 •

edited

Loading