Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RHELC-1606] Fix of multiple backup of one file #1267

Closed
wants to merge 1 commit into from

Conversation

hosekadam
Copy link
Member

File can be present multiple times in the output of 'rpm -Va' command. The file is backed up only once.

Also, the unit test was updated to handle this situation.

Jira Issues:

Checklist

  • PR has been tested manually in a VM (either author or reviewer)
  • Jira issue has been made public if possible
  • [RHELC-] or [HMS-] is part of the PR title
  • GitHub label has been added to help with Release notes
  • PR title explains the change from the user's point of view
  • Code and tests are documented properly
  • The commits are squashed to as few commits as possible (without losing data)
  • When merged: Jira issue has been updated to Release Pending if relevant

@hosekadam hosekadam added the kind/bug-fix A bug has been fixed label Jun 18, 2024
Copy link

codecov bot commented Jun 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.01%. Comparing base (ab3bbdd) to head (0269506).
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1267      +/-   ##
==========================================
- Coverage   95.83%   92.01%   -3.83%     
==========================================
  Files          55       55              
  Lines        4756     4757       +1     
  Branches      836      836              
==========================================
- Hits         4558     4377     -181     
- Misses        112      284     +172     
- Partials       86       96      +10     
Flag Coverage Δ
centos-linux-7 ?
centos-linux-8 91.95% <100.00%> (+<0.01%) ⬆️
centos-linux-9 92.00% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@Venefilyn Venefilyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems mostly fine!

Comment on lines 360 to 370
# Remove the original file
try:
os.remove(original_file_path)
removed_paths.append(original_file_path)
# FileNotFound on Python 3+, due compatibility with Python 2.7 using OSError
except OSError:
# If the path is present multiple times in the 'rpm -Va' output
# it's possible, the path was already removed. If not, that's fail.
if original_file_path not in removed_paths:
assert False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use with pytest.raises(OSError):? What about IOError?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Venefilyn I checked this and the comments ### to have this prepared. So both of the commits are ready and we can discuss which we will merge.

Comments are updated, but I don't think this can be changed. I'm also checking if the path was really removed and the exception doesn' have to be raised each time.

@@ -297,37 +304,45 @@ def test_backup_package_file_complete(
backup_package_files_action,
global_backup_control,
):
# Prepare the rpm -va ouput to the PRE_RPM_VA_LOG_FILENAME file
### Prepare the 'rpm -Va' ouput to the PRE_RPM_VA_LOG_FILENAME file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I'm a bit unclear about the triple hashes. What is the significance of # vs ###?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably just some typo or re-commenting the same comment two more times

Copy link
Member

@Venefilyn Venefilyn Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought so too but then saw a comment else where in the PR with

### Prepare the files from the 'rpm -Va' output by creating them
# Use only unique paths, since one path can be present multiple times in the output
# https://issues.redhat.com/browse/RHELC-1606

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan was to create some headings to each part of the test. But I think it wasn't good plan :D I'll remove that, since it's unclear

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's fine but was curious of the reason why. Wonder if we can make it more clear somehow..

I think when it's like this of a heading and comments underneath I'd go with this or something like it

# Heading
# 
# Text here
# Text here

@Venefilyn Venefilyn added the tests/tier0 PR ready to run the essential test suit. Equivalent to `/packit test --labels tier0`. label Jun 19, 2024
@has-bot
Copy link
Member

has-bot commented Jun 19, 2024

/packit test --labels tier0


Comment generated by an automation.

@bookwar
Copy link
Contributor

bookwar commented Jun 25, 2024

/packit test --labels tier0

@Venefilyn
Copy link
Member

Venefilyn commented Jun 25, 2024

@hosekadam I had forgotten that I even reviewed this, but I created a solution too without realizing we had a solution created already. What is stopping us from doing something like this?

Venefilyn@1e48049

@hosekadam
Copy link
Member Author

@Venefilyn looking at it, I would say they are really similar. Maybe the mine solution is easier to read (but it's probably because I did that :D) - like just adding the backed up path to the list of already backed up paths (which we already use, just extending it).

And yours where we have in the paths to back up just unique paths. Maybe the memory usage will be lower in this case.

Just one case came to my mind - what if the status or type of file would differ, but the path would be the same? In that case it would be added to the list, right? But maybe this is a too much corner case...

But I would definitely use the unit tests from this PR, which checks the complete process of the backup and rollback, I think the unit test is good because it tests the real usage (and I spent some time on it :D). But I understand, with your solution it's probably not needed so much.

@Venefilyn
Copy link
Member

what if the status or type of file would differ, but the path would be the same?

Good question. Feels like that would never occur though 🤔

@bookwar @r0x0d WDYT

@hosekadam hosekadam force-pushed the fix-multiple-file-backup branch 2 times, most recently from eea36c4 to 5a049c4 Compare June 25, 2024 12:59
@hosekadam
Copy link
Member Author

/packit test --labels tier0

@r0x0d
Copy link
Member

r0x0d commented Jun 25, 2024

what if the status or type of file would differ, but the path would be the same?

Good question. Feels like that would never occur though 🤔

@bookwar @r0x0d WDYT

I would merge the two PRs in this one. I prefer @Venefilyn solution as it is simpler and does not require always updating a list after each new backed up file. Ideally, we should receive the data in that function already prepared to parse and backup.

But I agree with @hosekadam to keep his unit_tests as it is testing the whole situation. I would try to look for a way to split the test or simplify if possible. The test itself is very difficult to read 🤔.

My vote here is then to pick @Venefilyn solution and @hosekadam unit_tests.

File can be present multiple times in the output of 'rpm -Va' command.
The file is backed up only once.
@hosekadam hosekadam force-pushed the fix-multiple-file-backup branch from 5a049c4 to 0269506 Compare July 8, 2024 16:11
Venefilyn added a commit to Venefilyn/convert2rhel that referenced this pull request Jul 9, 2024
Taken from Adam's PR oamg#1267

Co-authored-by: Adam Hosek <[email protected]>
Venefilyn added a commit to Venefilyn/convert2rhel that referenced this pull request Jul 9, 2024
Taken from Adam's PR oamg#1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
Venefilyn added a commit to Venefilyn/convert2rhel that referenced this pull request Jul 9, 2024
Taken from Adam's PR oamg#1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
Venefilyn added a commit to Venefilyn/convert2rhel that referenced this pull request Jul 9, 2024
Taken from Adam's PR oamg#1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
Venefilyn added a commit to Venefilyn/convert2rhel that referenced this pull request Jul 9, 2024
Taken from Adam's PR oamg#1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
Venefilyn added a commit to Venefilyn/convert2rhel that referenced this pull request Jul 19, 2024
Taken from Adam's PR oamg#1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
Venefilyn added a commit that referenced this pull request Jul 22, 2024
Taken from Adam's PR #1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
Venefilyn added a commit that referenced this pull request Jul 23, 2024
Taken from Adam's PR #1267

Co-authored-by: Adam Hosek <[email protected]>
Signed-off-by: Freya Gustavsson <[email protected]>
@Venefilyn
Copy link
Member

We decided to merge this and my PR so closing this one

@Venefilyn Venefilyn closed this Jul 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug-fix A bug has been fixed tests/tier0 PR ready to run the essential test suit. Equivalent to `/packit test --labels tier0`.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants