tests for modified java-openliberty stack fail

[tnixa]

Describe the bug
I modified the java-openliberty stack by adding in a blockchain dependency and a couple java files and when i do an appdsody run I am noticing some test failures. It seems the tests are trying to reach endpoints before they are available...

[Container] [INFO] -------------------------------------------------------
[Container] [INFO]  T E S T S
[Container] [INFO] -------------------------------------------------------
[Container] [INFO] Running it.dev.appsody.starter.EndpointTest
[Container] [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.06 s <<< FAILURE! - in it.dev.appsody.starter.EndpointTest
[Container] [ERROR] testResourceEndpoint  Time elapsed: 1.969 s  <<< FAILURE!
[Container] org.opentest4j.AssertionFailedError: Incorrect response code from http://localhost:9080/starter/resource ==> expected: <200> but was: <404>
[Container] 	at it.dev.appsody.starter.EndpointTest.assertResponse(EndpointTest.java:81)
[Container] 	at it.dev.appsody.starter.EndpointTest.checkEndpoint(EndpointTest.java:51)
[Container] 	at it.dev.appsody.starter.EndpointTest.testResourceEndpoint(EndpointTest.java:44)
[Container]
[Container] [INFO] Running it.dev.appsody.starter.HealthEndpointTest
[Container] [INFO] [WARNING ] CWMH0053W: The readiness health check reported a DOWN overall status because the following applications have not started yet: [starter-app]
[Container] [ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 0.262 s <<< FAILURE! - in it.dev.appsody.starter.HealthEndpointTest
[Container] [ERROR] testLivenessEndpoint  Time elapsed: 0.194 s  <<< ERROR!
[Container] java.lang.IndexOutOfBoundsException: Index: 0
[Container] 	at it.dev.appsody.starter.HealthEndpointTest.checkHealthEndpoint(HealthEndpointTest.java:67)
[Container] 	at it.dev.appsody.starter.HealthEndpointTest.testLivenessEndpoint(HealthEndpointTest.java:46)
[Container]
[Container] [ERROR] testReadinessEndpoint  Time elapsed: 0.065 s  <<< FAILURE!
[Container] org.opentest4j.AssertionFailedError: Incorrect response code from http://localhost:9080/health/ready ==> expected: <200> but was: <503>
[Container] 	at it.dev.appsody.starter.HealthEndpointTest.assertResponse(HealthEndpointTest.java:96)
[Container] 	at it.dev.appsody.starter.HealthEndpointTest.checkHealthEndpoint(HealthEndpointTest.java:59)
[Container] 	at it.dev.appsody.starter.HealthEndpointTest.testReadinessEndpoint(HealthEndpointTest.java:52)
[Container]
[Container] [INFO]
[Container] [INFO] Results:
[Container] [INFO]
[Container] [ERROR] Failures:
[Container] [ERROR]   EndpointTest.testResourceEndpoint:44->checkEndpoint:51->assertResponse:81 Incorrect response code from http://localhost:9080/starter/resource ==> expected: <200> but was: <404>
[Container] [ERROR]   HealthEndpointTest.testReadinessEndpoint:52->checkHealthEndpoint:59->assertResponse:96 Incorrect response code from http://localhost:9080/health/ready ==> expected: <200> but was: <503>
[Container] [ERROR] Errors:
[Container] [ERROR]   HealthEndpointTest.testLivenessEndpoint:46->checkHealthEndpoint:67 ? IndexOutOfBounds
[Container] [INFO]
[Container] [ERROR] Tests run: 3, Failures: 2, Errors: 1, Skipped: 0
[Container] [INFO]
[Container] [ERROR] Integration tests failed: There are test failures.
[Container]
[Container] Please refer to /project/user-app/target/test-reports/it for the individual test results.

To Reproduce
Steps to reproduce the behavior:

1. Go to https://github.ibm.com/icpa-refarch-blockchain/java-openliberty-stack
2. clone the repo
3. appsody stack package
4. `appsody init dev.local/java-openliberty-stack
5. appsody run

Expected behavior
The tests were not failing prior to adding in the additional code for blockchain

Environment Details (please complete the following information):

• OS: [e.g. iOS] macOS

If applicable please specify:

• CLI version: 0.5.9
• Stack you are using (including the version): java-openliberty

[scottkurz]

That's interesting... I wonder what the Open Liberty "contract" is with respect to availability of health check endpoints vs. whatever "app started" events the liberty-maven-plugin dev mode is listening for.

It seems there's a good chance this is more an issue between the liberty-maven-plugin and the Open Liberty runtime than the stack layer per se.

That said, we can try to research more to find the right way to direct this.

I'm curious if you've ever seen this having made the change initially, before appsody run ? Or do you only see it on a dynamic change made after appsody run? Or both? I'd read into your writeup that you made the change before appsody run but just asking you to confirm.

Thanks.

[chilanti]

@scottkurz - please let us know what you find out. We are observing startup times that aren't acceptable:

[Container] [INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://3430d04c5499:9080/
[Container] [INFO] [AUDIT   ] CWWKZ0001I: Application starter-app started in 182.711 seconds.
[Container] [INFO] [AUDIT   ] CWWKT0017I: Web application removed (default_host): http://3430d04c5499:9080/
[Container] [INFO] [AUDIT   ] CWWKZ0009I: The application starter-app has stopped successfully.
[Container] [INFO] [AUDIT   ] CWWKZ0022W: Application starter-app has not started in 29.965 seconds.
[Container] [INFO] [AUDIT   ] CWWKT0016I: Web application available (default_host): http://3430d04c5499:9080/
[Container] [INFO] [AUDIT   ] CWWKZ0003I: The application starter-app updated in 127.945 seconds

(this is the starter app with a minor minor change BTW - it's also unclear why the app stops and restarts... and I've verified that the app is not responding until we see that "updated" message at the end).

[scottkurz]

Opened: OpenLiberty/ci.maven#766 for some help on this from the liberty-maven-plugin side.

[scottkurz]

A workaround you could try is running in --interactive mode: appsody run --interactive which doesn't run the tests right away but holds off until you hit <Enter>.

[scottkurz]

@tnixa .. I don't have auth to do this, but maybe it'd help to re-title this as "ITs using MP Health endpoints fail in development mode".