Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test (e2e) : add more assertions to verify crc status output #4604

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rohanKanojia
Copy link
Contributor

@rohanKanojia rohanKanojia commented Feb 10, 2025

Description

⚠️ This test is expected to fail on main branch, it should be merged after #4603

Add stricter assertions to verify output of crc status.

We only verify that the CRC status output contains the keyword Running or Stopped, depending on the cluster state. However, we should also verify other attributes, such as diskSize, ramsSize, openshiftStatus, cacheDir, etc.

Relates to: PR #4603

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change
  • Chore (non-breaking change which doesn't affect codebase;
    test, version modification, documentation, etc.)

Proposed changes

Add another step in basic test scenario to verify output of crc status -ojson. Parse JSON output into a map and verify that values are as expected or within allowed limit.

Testing

I just verified that Basic scenario E2E test is passing when #4603 is applied.

Contribution Checklist

  • I Keep It Small and Simple: The smaller the PR is, the easier it is to review and have it merged
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Which platform have you tested the code changes on?
    • Linux
    • Windows
    • MacOS

Copy link

openshift-ci bot commented Feb 10, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link

openshift-ci bot commented Feb 10, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign adrianriobo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rohanKanojia rohanKanojia force-pushed the pr/crc-status-test-assertions branch from 933b731 to 7433400 Compare February 10, 2025 05:02
@rohanKanojia rohanKanojia marked this pull request as ready for review February 10, 2025 06:05
@openshift-ci openshift-ci bot requested review from adrianriobo and gbraad February 10, 2025 06:05
@rohanKanojia
Copy link
Contributor Author

E2E test failure is expected as it captures the failure reported in #4603

  �[31mScenario: CRC start usecase�[0m �[1;30m# test/e2e/features/basic.feature:32�[0m
    �[31mAnd checking the CRC status JSON output is valid�[0m �[1;30m# test/e2e/features/basic.feature:46�[0m
      �[31mError: �[0m�[1;31mfailure in asserting 'ramSize' field of crc status json output, expected less than or equal to 11274289152 bytes, actual : 32680947712 bytes�[0m

Now that #4603 is merged it should pass after rebasing this PR.

@rohanKanojia rohanKanojia force-pushed the pr/crc-status-test-assertions branch from 7433400 to c54baee Compare February 10, 2025 09:21
@@ -74,4 +75,4 @@ Feature: Basic test
And kubeconfig is cleaned up
# cleanup
When executing crc cleanup command succeeds
And executing "man -P cat crc" fails
# And executing "man -P cat crc" fails
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why those changes?

Copy link
Contributor Author

@rohanKanojia rohanKanojia Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry. It looks like I forgot to uncomment it after testing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I lost the changes while pushing from another machine. I'm seeing outdated version of the changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated it. I think it should be okay now.

@rohanKanojia rohanKanojia force-pushed the pr/crc-status-test-assertions branch 2 times, most recently from eadc046 to 9c4c8c9 Compare February 10, 2025 09:30
Add stricter assertions to verify output of crc status

Signed-off-by: Rohan Kumar <[email protected]>
@rohanKanojia rohanKanojia force-pushed the pr/crc-status-test-assertions branch from 9c4c8c9 to 40d9c08 Compare February 10, 2025 09:37
Copy link

openshift-ci bot commented Feb 10, 2025

@rohanKanojia: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/security 40d9c08 link false /test security
ci/prow/integration-crc 40d9c08 link true /test integration-crc
ci/prow/e2e-crc 40d9c08 link true /test e2e-crc

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@praveenkumar
Copy link
Member

/assign @adrianriobo
/assign @albfan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants