Future Testing Requirement Details

This page fleshes out requirement details for testing Momentum model components.

Compute-intensive testing on a cloud platform

Rationale

Many tests of simulation models require significant computational resources and run over a longer period of (wall-clock) time, making it unpractical or even impossible to integrate them in GitHub Actions or similar CI services. The public cloud is a possible choice for running such tests in an automated fashion.

Advantages

• A given test can run on the most appropriate set of resources to satisfy its needs around processor type and capability, memory requirements, and storage system requirements.
• A large variety of architectures and compilers can be used for testing to ensure portability and improve overall test coverage.
• Developers from across the Momentum partnership can be given access, which is typically much more difficult to arrange on in-house resources at partner sites.
• Test outcomes can be made visible to developers much more easily, who can take action and gather information required to debug a problem.
• Cloud services come with sophisticated automation tools and services, which is ideal for integrating test systems with the GitHub repository.
• Developers from across the partnership can use the cloud service to access a wide range of tests, while they have to rely on their site support to enable running these tests on in-house systems otherwise.

Disadvantages

• Cloud resources can be expensive, depending on the type and level of utilisation.
• Using cloud services can add a lot of complexity around security, automation, vendor-specific constraints, etc.

Requirement Details

• The cloud services should be accessible by developers from across the Momentum partnership.
• The cloud service needs to integrate with GitHub so that tests can be triggered automatically.