xlinux failing to read system test tools.jar

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_x86-64_linux_Nightly_testList_2/591/console
https://openj9-jenkins.osuosl.org/job/Test_openjdk8_j9_sanity.system_x86-64_linux_Nightly_testList_1/591/console

02:03:39  build-java-8:
02:03:39      [javac] Compiling 16 source files to /home/****/workspace/Test_openjdk8_j9_sanity.system_x86-64_linux_Nightly_testList_1/aqa-tests/system/aqa-systemtest/openjdk.test.debugging/bin
02:03:39      [javac] error: error reading /home/****/workspace/Test_openjdk8_j9_sanity.system_x86-64_linux_Nightly_testList_1/aqa-tests/systemtest_prereqs/tools/tools.jar; error in opening zip file
02:03:39      [javac] 1 error

@llxia @AdamBrousseau

[AdamBrousseau]

systemtest_prereqs/tools/tools.jar
Which comes from the systemtest dependency job

02:01:45  Exception: org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
[Pipeline] echo
02:01:45  Cannot run copyArtifacts from systemtest.getDependency. Skipping copyArtifacts...

Perhaps related to the other Jenkins issues we've seen last night. FTR Lan has disabled TRSS monitoring for now to see if that is causing these jenkins exceptions.

@llxia Is the build supposed to fallback to getting these dependencies from another place if copyArtifacts fails? Or should we be aborting at that point?

[pshipton]

Similar failure in an internal build on Windows
19:35:39 [javac] error: error reading C:\Users\jenkins\workspace\Test_openjdk17_j9_special.system_x86-64_windows_testList_3\aqa-tests\systemtest_prereqs\tools\tools.jar; zip END header not found

[llxia]

Is the build supposed to fallback to getting these dependencies from another place if copyArtifacts fails? Or should we be aborting at that point?

The fallback logic is in TKG, and it is used for non-system tests. The system tests have its own logic in STF. I am not sure if the correct fallback logic is implemented in STF.

To avoid confusion, maybe we should abort if copyArtifacts from systemtest.getDependency fails.

[cjjdespres]

I've been seeing this in the JITServer nightly tests recently with some regularity, e.g. in https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk11_j9_extended.system_s390x_linux_jit_Personal_testList_1/1103/

03:51:28  build-java-11:
03:51:28      [javac] Compiling 16 source files to /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_jit_Personal_testList_1/aqa-tests/system/aqa-systemtest/openjdk.test.debugging/bin
03:51:29      [javac] error: error reading /home/jenkins/testDependency/system_lib/tools/tools.jar; zip END header not found
03:51:29  
03:51:29  BUILD FAILED

[llxia]

The logic has changed since the initial reporting of this issue. We are no longer using Jenkins copyArtifacts in the test pipeline. The dependent jars are pre-staged on the machines. And the test pipeline will only download them if they do not exist.

In this case, I suspect a bad tools.jar got pre-staged on the machine.

00:02:02.309  /home/jenkins/workspace/Test_openjdk11_j9_extended.system_s390x_linux_jit_Personal_testList_1/../../testDependency/system_lib/tools/tools.jar exists, not downloading. 

ubu22s390x-svl-rt7-1 is offline. I cannot clean up testDependency on the machine atm.

[llxia]

tools.jar is removed from ubu22s390x-svl-rt7-1.

Close this issue. Please reopen if the issue still occurs.

[github-actions]

Issue Number: 18090
Status: Closed
Actual Components: comp:test, test failure
Actual Assignees: No one :(
PR Assignees: No one :(