Add sample integ tests for latest systemd unit file #17410

RajatGupta02 · 2025-02-21T07:44:43Z

Description

Added few integ tests for the latest systemd unit file that has to be shipped from version 3.0 onwards.

Supporting References

Related PR: #17107

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

RajatGupta02 · 2025-02-21T07:45:21Z

@peterzhuamazon Here are the sample tests to verify the behavior

github-actions · 2025-02-21T08:11:27Z

❌ Gradle check result for beaa600: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

reta · 2025-02-21T13:12:26Z

@RajatGupta02 I am not sure how exactly this test is supposed to be running, should we run it in Docker with RPM distribution installed? Thank you.

peterzhuamazon · 2025-02-21T15:08:05Z

@RajatGupta02 I am not sure how exactly this test is supposed to be running, should we run it in Docker with RPM distribution installed? Thank you.

It will be onboard to opensearch-build repo and run within the framework there.
Assuming a cluster is already deployed with deb/rpm in a docker on Jenkins.

peterzhuamazon · 2025-02-24T22:26:26Z

Hi @RajatGupta02 @kumargu I made a change to this PR, specially adding sudo for su command, as we are assuming a non-root user to run the test.

I also have changes to build repo which we can build the new deb/rpm with service file and env file from your previous PR (#17107):

Support new OS core systemd and integTests in 3.0.0 and manifest updates opensearch-build#5345

There might be some changes required for this line, as our docker container is assuming to have 4 CPUs, I wonder if you can use workers by default to avoid overflowing the system.

:qa:systemd-test:integTest.maxParallelForks (49) is larger than max-workers (4), forcing it to 4

cc: @getsaurabh02

The result log:


> Task :buildSrc:reaper:compileJava FROM-CACHE
> Task :buildSrc:reaper:processResources NO-SOURCE
> Task :buildSrc:reaper:classes UP-TO-DATE
> Task :buildSrc:reaper:jar
> Task :buildSrc:compileJava FROM-CACHE
> Task :buildSrc:compileGroovy FROM-CACHE
> Task :buildSrc:generateVersionProperties FROM-CACHE
> Task :buildSrc:pluginDescriptors
> Task :buildSrc:processResources
> Task :buildSrc:classes
> Task :buildSrc:jar
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.12
  OS Info               : Linux 5.10.234-205.895.amzn2int.x86_64 (amd64)
  JDK Version           : 23 (Eclipse Temurin JDK)
  JAVA_HOME             : /opt/java/openjdk-23
  Random Testing Seed   : 6B21ADE5FD63681D
  In FIPS 140 mode      : false
=======================================
> Task :libs:opensearch-cli:processResources NO-SOURCE
> Task :libs:opensearch-geo:processResources NO-SOURCE
> Task :libs:opensearch-plugin-classloader:processResources NO-SOURCE
> Task :libs:opensearch-compress:processResources
> Task :libs:opensearch-core:processResources
> Task :libs:opensearch-common:processResources
> Task :libs:opensearch-secure-sm:processResources NO-SOURCE
> Task :libs:opensearch-task-commons:processResources NO-SOURCE
> Task :libs:opensearch-telemetry:processResources NO-SOURCE
> Task :server:extractProto
> Task :libs:opensearch-x-content:processResources
> Task :libs:opensearch-plugin-classloader:compileJava FROM-CACHE
> Task :libs:opensearch-geo:compileJava FROM-CACHE
> Task :libs:opensearch-secure-sm:compileJava FROM-CACHE
> Task :client:sniffer:processResources NO-SOURCE
> Task :libs:opensearch-geo:classes UP-TO-DATE
> Task :libs:opensearch-geo:jar
> Task :libs:opensearch-nio:processResources NO-SOURCE
> Task :libs:opensearch-secure-sm:classes UP-TO-DATE
> Task :libs:opensearch-secure-sm:jar
> Task :qa:systemd-test:processTestResources NO-SOURCE
> Task :client:rest:compileJava FROM-CACHE
> Task :client:rest:processResources NO-SOURCE
> Task :client:rest:classes UP-TO-DATE
> Task :test:framework:processResources
> Task :client:rest:jar
> Task :test:telemetry:processResources NO-SOURCE
> Task :client:sniffer:compileJava FROM-CACHE
> Task :client:sniffer:classes UP-TO-DATE
> Task :client:sniffer:jar
> Task :libs:opensearch-common:compileJava FROM-CACHE
> Task :libs:opensearch-common:classes
> Task :libs:opensearch-common:jar
> Task :libs:opensearch-cli:compileJava FROM-CACHE
> Task :libs:opensearch-task-commons:compileJava FROM-CACHE
> Task :libs:opensearch-cli:classes UP-TO-DATE
> Task :libs:opensearch-cli:jar
> Task :libs:opensearch-telemetry:compileJava FROM-CACHE
> Task :libs:opensearch-task-commons:classes UP-TO-DATE
> Task :libs:opensearch-task-commons:jar
> Task :libs:opensearch-telemetry:classes UP-TO-DATE
> Task :server:extractIncludeProto
> Task :libs:opensearch-nio:compileJava FROM-CACHE
> Task :libs:opensearch-nio:classes UP-TO-DATE
> Task :server:generateProto FROM-CACHE
> Task :server:generateModulesList
> Task :server:generatePluginsList
> Task :libs:opensearch-telemetry:jar
> Task :libs:opensearch-nio:jar
> Task :server:processResources
> Task :libs:opensearch-core:compileJava FROM-CACHE
> Task :libs:opensearch-core:classes
> Task :libs:opensearch-core:jar
> Task :libs:opensearch-compress:compileJava FROM-CACHE
> Task :libs:opensearch-compress:classes
> Task :libs:opensearch-x-content:compileJava FROM-CACHE
> Task :libs:opensearch-x-content:classes
> Task :libs:opensearch-compress:jar
> Task :test:telemetry:compileJava FROM-CACHE
> Task :test:telemetry:classes UP-TO-DATE
> Task :libs:opensearch-x-content:jar
> Task :test:telemetry:jar
> Task :server:compileJava FROM-CACHE
> Task :server:classes
> Task :test:framework:compileJava FROM-CACHE
> Task :test:framework:classes
> Task :test:framework:jar
> Task :server:jar
> Task :qa:systemd-test:compileTestJava
> Task :qa:systemd-test:testClasses

> Task :qa:systemd-test:integTest

BUILD SUCCESSFUL in 17s
52 actionable tasks: 31 executed, 21 from cache

Signed-off-by: Rajat Gupta <[email protected]>

github-actions · 2025-03-03T21:09:05Z

❌ Gradle check result for e0e7c61: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Rajat Gupta <[email protected]>

github-actions · 2025-03-03T21:23:53Z

❌ Gradle check result for c1f2677: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Rajat Gupta <[email protected]>

github-actions · 2025-03-04T08:36:11Z

❌ Gradle check result for ccc1b7b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

peterzhuamazon · 2025-03-04T16:23:26Z

Seems passing:


> Task :qa:systemd-test:test

BUILD SUCCESSFUL in 45s
55 actionable tasks: 33 executed, 22 from cache

Signed-off-by: Rajat Gupta <[email protected]>

github-actions · 2025-03-04T17:29:57Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-04T17:54:38Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

RajatGupta02 · 2025-03-04T17:59:15Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Some unrelated tests are failing. Can we re-run this?

github-actions · 2025-03-04T18:14:21Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-04T18:48:28Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-04T19:33:24Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-03-05T02:14:37Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

RajatGupta02 · 2025-03-05T12:36:56Z

@reta The tests pass locally, and gradle testing convention precommit checks pass now, also I do not see any test failures with Gradle Check (Jenkins) workflow so I think we can merge this?

github-actions · 2025-03-05T19:58:04Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

RajatGupta02 · 2025-03-05T20:35:26Z

❌ Gradle check result for 1ff7d59: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

I don't see any test failures here: https://build.ci.opensearch.org/job/gradle-check/54199/testReport/
But on checking the console logs I see this:

> Task :qa:systemd-test:test
WARNING: Using incubator modules: jdk.incubator.vector

SystemdIntegTests > testReadOnlyPaths FAILED
    java.lang.RuntimeException: Failed to check read permission
        at __randomizedtesting.SeedInfo.seed([F77812C9D3894EB1:8D37CAE348EFD649]:0)
        at org.opensearch.systemdinteg.SystemdIntegTests.executeCommand(SystemdIntegTests.java:90)
        at org.opensearch.systemdinteg.SystemdIntegTests.checkPathReadable(SystemdIntegTests.java:68)
        at org.opensearch.systemdinteg.SystemdIntegTests.testReadOnlyPaths(SystemdIntegTests.java:107)

SystemdIntegTests > testSystemCallFilter FAILED
    java.lang.RuntimeException: Failed to read Seccomp status
        at __randomizedtesting.SeedInfo.seed([F77812C9D3894EB1:AB3BF7899BF60264]:0)
        at org.opensearch.systemdinteg.SystemdIntegTests.executeCommand(SystemdIntegTests.java:90)
        at org.opensearch.systemdinteg.SystemdIntegTests.testSystemCallFilter(SystemdIntegTests.java:139)

SystemdIntegTests > testReadWritePaths FAILED
    java.lang.AssertionError: Path should exist: /var/log/opensearch
        at __randomizedtesting.SeedInfo.seed([F77812C9D3894EB1:3939490521CA8BF5]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.opensearch.systemdinteg.SystemdIntegTests.testReadWritePaths(SystemdIntegTests.java:116)

Signed-off-by: Rajat Gupta <[email protected]>

peterzhuamazon · 2025-03-05T21:19:59Z

Also passes: ./gradlew qa:systemd-test:integTest --tests org.opensearch.systemdinteg.SystemdIntegTests --info


> Task :qa:systemd-test:integTest

[Incubating] Problems report is available at: file:///tmp/tmpq3kia7hz/OpenSearch/build/reports/problems/problems-report.html

BUILD SUCCESSFUL in 1m 31s
55 actionable tasks: 35 executed, 20 from cache

github-actions · 2025-03-05T21:22:53Z

❌ Gradle check result for d3b1c0a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

RajatGupta02 · 2025-03-05T21:24:30Z

❌ Gradle check result for d3b1c0a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

console log:

FAILURE: Build failed with an exception.

* What went wrong:
Gradle build daemon has been stopped: since the JVM garbage collector is thrashing

Not related to our change this time!

github-actions · 2025-03-05T22:17:09Z

❕ Gradle check result for d3b1c0a: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This was referenced Feb 21, 2025

[RELEASE] Release version 3.0.0 opensearch-project/opensearch-build#3747

Open

Support new OS core systemd and integTests in 3.0.0 and manifest updates opensearch-project/opensearch-build#5345

Merged

Rename test file

e0e7c61

Signed-off-by: Rajat Gupta <[email protected]>

Add test script

c1f2677

Signed-off-by: Rajat Gupta <[email protected]>

Extend LuceneTestCase class

ccc1b7b

Signed-off-by: Rajat Gupta <[email protected]>

Rajat Gupta and others added 2 commits March 4, 2025 09:08

Remove test bash script

8d5e2fc

Signed-off-by: Rajat Gupta <[email protected]>

Merge branch 'main' into test-integ-branch

1ff7d59

Signed-off-by: Rajat Gupta <[email protected]>

kumargu approved these changes Mar 4, 2025

View reviewed changes

RajatGupta02 force-pushed the test-integ-branch branch from c66fb4a to 1ff7d59 Compare March 5, 2025 20:47

Modify build.gradle

d3b1c0a

Signed-off-by: Rajat Gupta <[email protected]>

peterzhuamazon mentioned this pull request Mar 5, 2025

Update OS core systemd related test commands and switch to Ubuntu2404 Agents opensearch-project/opensearch-build#5367

Draft

peterzhuamazon approved these changes Mar 5, 2025

View reviewed changes

andrross approved these changes Mar 5, 2025

View reviewed changes

peterzhuamazon merged commit 02055cc into opensearch-project:main Mar 5, 2025
32 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add sample integ tests for latest systemd unit file #17410

Add sample integ tests for latest systemd unit file #17410

RajatGupta02 commented Feb 21, 2025 •

edited

Loading

RajatGupta02 commented Feb 21, 2025

github-actions bot commented Feb 21, 2025

reta commented Feb 21, 2025

peterzhuamazon commented Feb 21, 2025

peterzhuamazon commented Feb 24, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 4, 2025

peterzhuamazon commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

RajatGupta02 commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Mar 5, 2025

peterzhuamazon commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

Add sample integ tests for latest systemd unit file #17410

Add sample integ tests for latest systemd unit file #17410

Conversation

RajatGupta02 commented Feb 21, 2025 • edited Loading

Description

Supporting References

Check List

RajatGupta02 commented Feb 21, 2025

github-actions bot commented Feb 21, 2025

reta commented Feb 21, 2025

peterzhuamazon commented Feb 21, 2025

peterzhuamazon commented Feb 24, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 3, 2025

github-actions bot commented Mar 4, 2025

peterzhuamazon commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

RajatGupta02 commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 4, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Mar 5, 2025

peterzhuamazon commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Mar 5, 2025

github-actions bot commented Mar 5, 2025

RajatGupta02 commented Feb 21, 2025 •

edited

Loading