harmony testGolden javax.security.auth.callback.serialization.PasswordCallbackTest ComparisonFailure: null expected:<[abc]> but was:<[ ]>

[pshipton]

Internal build
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode112 -Xgcpolicy:gencon -Xjit:count=0 -Xnocompressedrefs
fyrec50y

Failed 1/30 in a grinder (fyrec50z)

j> 15:03:40 20241004 15:03:40 <Primary|SimpleDriver|org.apache.harmony.auth.tests.javax.security.auth.callback.serialization.PasswordCallbackTest|3288|Default Invocant> ERROR: Test testGolden(org.apache.harmony.auth.tests.javax.security.auth.callback.serialization.PasswordCallbackTest) failed
j> 15:03:40 Throwable trace:
j> 15:03:40 junit.framework.ComparisonFailure: null expected:<[abc]> but was:<[   ]>
j> 15:03:40 	at junit.framework.Assert.assertEquals(Assert.java:81)
j> 15:03:40 	at junit.framework.Assert.assertEquals(Assert.java:87)
j> 15:03:40 	at org.apache.harmony.auth.tests.javax.security.auth.callback.serialization.PasswordCallbackTest.assertDeserialized(PasswordCallbackTest.java:51)
j> 15:03:40 	at org.apache.harmony.testframework.serialization.SerializationTest.verifyGolden(SerializationTest.java:519)
j> 15:03:40 	at org.apache.harmony.testframework.serialization.SerializationTest.verifyGolden(SerializationTest.java:492)
j> 15:03:40 	at org.apache.harmony.testframework.serialization.SerializationTest.testGolden(SerializationTest.java:147)

[github-actions]

Issue Number: 20307
Status: Open
Recommended Components: comp:test, comp:vm, comp:gc
Recommended Assignees: pshipton, jasonfengj9, chengjin01

[pshipton]

@hzongaro @r30shah fyi. I haven't seen this failure before (although I don't always do the best job of triaging harmony failures since there are a bunch of machine issues).

[r30shah]

I see this one failing 1/50 times (build_info.php?build_id=79356), need to investigate this (May be 31-bit z/OS issue only??), I will do initial triage before getting one of us to work on it.

[pshipton]

Internal build
[zOS S390 64bit Compressed Pointers] 80 Load_Level_2.harmony.5mins.Mode610 -Xcompressedrefs -Xjit -Xgcpolicy:gencon
fyrec510

[pshipton]

It's not 32-bit only and it failed without -Xjit:count=0 so I've added it to the next milestone.

[r30shah]

Thanks Peter for sharing, I think this will help nailing down the cause. @matthewhall2 Let's look into this together.

[pshipton]

Internal build
[zOS S390 64bit Compressed Pointers] 80 Load_Level_2.harmony.5mins.Mode610
fyrec60h

[pshipton]

Internal build
[Linux S390 64bit] 80 Load_Level_2.harmony.5mins.Mode109

[pshipton]

Internal build
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode187

[pshipton]

Internal build
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode112

[pshipton]

Internal build
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode110

[pshipton]

Internal build
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode187

[r30shah]

Looking into this, given the assertion failure, to effectively investigate, I need to reproduce this reliably. I did picked up the last failing build from previous comment and launched grinder of 100x
I can not reproduce this in 100x. Rerunning the s390x Linux with latest build 100x internal link . This one does not fail as well. I will check if I can reproduce this on specific set of machines (Though I checked couple of last reported failures - I see the grinder did launch some jobs on those machines so that seems unlikely).

[pshipton]

It does fail in the builds fairly frequently. If it can't be reproduced via grinder, is there anything that can be done to the test to gather more information when it does fail?

[r30shah]

It does fail in the builds fairly frequently. If it can't be reproduced via grinder, is there anything that can be done to the test to gather more information when it does fail?

I am looking into that direction. In past, I have dealt with such intermittent issue by changing the test to invoke a debug agent that can keep running the test to help identify the method causing the issue.

[pshipton]

http://vmfarm.rtp.raleigh.ibm.com/job_output.php?id=96186470
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode112

[pshipton]

I haven't seen this in a couple of weeks, removing the blocker label.

[hzongaro]

@r30shah, are you still looking at this one for 0.49, or should it move out to 0.51?

[r30shah]

Hi @hzongaro I am looking into this but given the low failure rate, couldn't get much update. I am going to work on this with changing the test to trigger agent for more debugging, we can move it to 0.51.

[pshipton]

http://vmfarm.rtp.raleigh.ibm.com/job_output.php?id=100359145
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode112
fyrec20f

[pshipton]

http://vmfarm.rtp.raleigh.ibm.com/job_output.php?id=100922439
[zOS S390] 80 Load_Level_2.harmony.5mins.Mode112
fyrec605