Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OpenStack] VM Import Controller uses too much memory during QCOW2 conversion phase #6674

Open
belgaied2 opened this issue Sep 25, 2024 · 2 comments
Open

[OpenStack] VM Import Controller uses too much memory during QCOW2 conversion phase #6674

belgaied2 opened this issue Sep 25, 2024 · 2 comments
Assignees
@votdev @irishgordo
Labels
area/vm-importer issues related to vm-import-controller addon kind/bug Issues that are defects reported by users or that we know have reached a real release reproduce/always Reproducible 100% of the time severity/2 Function working but has a major issue w/o workaround (a major incident with significant impact)
Milestone
v1.5.0

Comments

@belgaied2
Copy link

After successful creation of the image in OpenStack, VM Import Controller initiates the download of the image in Harvester.

It seems to happen in two steps:

  • Download the original QCOW2 image from OpenStack to memory ([]byte)
  • run the command qcow2-img convert to RAW.

Seems to be happening here.

This results in memory usage proportional to the QCOW2 image. More exactly, the qcow2-img convert command seems also to use memory, causing the total memory usage to reach : 2x , so for an image of 500Gi, VM Import Controller would need 1TiB of RAM!!

The first step can definitely be avoided.
@votdev votdev transferred this issue from harvester/vm-import-controller Sep 30, 2024
@votdev votdev added the area/vm-importer issues related to vm-import-controller addon label Sep 30, 2024
@votdev votdev self-assigned this Sep 30, 2024
@votdev votdev added kind/bug Issues that are defects reported by users or that we know have reached a real release severity/2 Function working but has a major issue w/o workaround (a major incident with significant impact) reproduce/always Reproducible 100% of the time labels Sep 30, 2024
votdev added a commit to votdev/vm-import-controller that referenced this issue Sep 30, 2024
@votdev
VM Import Controller uses too much memory during QCOW2 conversion phase
397aae6 
- Create the volume image using RAW disk format instead of QCOW2, so no conversion is required after downloading. This will reduce memory consumption.
- Fix a variable name shadowing issue.
- Improve logging.

Related to: harvester/harvester#6674

Signed-off-by: Volker Theile <[email protected]>
@votdev votdev mentioned this issue Sep 30, 2024
Merged
@votdev votdev added this to the v1.5.0 milestone Sep 30, 2024
votdev added a commit to votdev/vm-import-controller that referenced this issue Sep 30, 2024
@votdev
VM Import Controller uses too much memory during QCOW2 conversion phase
8130b5a 
- Create the volume image using RAW disk format instead of QCOW2, so no conversion is required after downloading. This will reduce memory consumption.
- Download and write the image file in chunks (32k by default), so the whole file doesn't need to be downloaded completely and stored in memory before it is written to disk.
- Fix a variable name shadowing issue.
- Improve logging.

Related to: harvester/harvester#6674

Signed-off-by: Volker Theile <[email protected]>
votdev added a commit to votdev/vm-import-controller that referenced this issue Sep 30, 2024
@votdev
VM Import Controller uses too much memory during QCOW2 conversion phase
808a50b 
- Create the volume image using RAW disk format instead of QCOW2, so no conversion is required after downloading. This will reduce memory consumption.
- Download and write the image file in chunks (32KiB by default), so the whole file doesn't need to be downloaded completely and stored in memory before it is written to disk.
- Fix a variable name shadowing issue.
- Improve logging.

Related to: harvester/harvester#6674

Signed-off-by: Volker Theile <[email protected]>
@harvesterhci-io-github-bot
Copy link
Collaborator

harvesterhci-io-github-bot commented Sep 30, 2024 •
edited by votdev
Loading

Pre Ready-For-Testing Checklist

  • If labeled: require/HEP Has the Harvester Enhancement Proposal PR submitted?
    The HEP PR is at:

  • Where is the reproduce steps/test steps documented?
    The reproduce steps/test steps are at: VM Import Controller uses too much memory during QCOW2 conversion phase vm-import-controller#51

  • Is there a workaround for the issue? If so, where is it documented?
    The workaround is at:

  • Have the backend code been merged (harvester, harvester-installer, etc) (including backport-needed/*)?
    The PR is at: VM Import Controller uses too much memory during QCOW2 conversion phase vm-import-controller#51

    • Does the PR include the explanation for the fix or the feature?

    • Does the PR include deployment change (YAML/Chart)? If so, where are the PRs for both YAML file and Chart?
      The PR for the YAML change is at:
      The PR for the chart change is at:

  • If labeled: area/ui Has the UI issue filed or ready to be merged?
    The UI issue/PR is at:

  • If labeled: require/doc, require/knowledge-base Has the necessary document PR submitted or merged?
    The documentation/KB PR is at:

  • If NOT labeled: not-require/test-plan Has the e2e test plan been merged? Have QAs agreed on the automation test case? If only test case skeleton w/o implementation, have you created an implementation issue?

    • The automation skeleton PR is at:
    • The automation test case PR is at:

  • If the fix introduces the code for backward compatibility Has a separate issue been filed with the label release/obsolete-compatibility?
    The compatibility issue is filed at:

@harvesterhci-io-github-bot
Copy link
Collaborator

Automation e2e test issue: harvester/tests#1558

Open
1 task
votdev added a commit to votdev/vm-import-controller that referenced this issue Sep 30, 2024
@votdev
VM Import Controller uses too much memory during QCOW2 conversion phase
d917f12 
- Create the volume image using RAW disk format instead of QCOW2, so no conversion is required after downloading. This will reduce memory consumption.
- Download and write the image file in chunks (32KiB by default), so the whole file doesn't need to be downloaded completely and stored in memory before it is written to disk.
- Fix a variable name shadowing issue.
- Improve logging.

Related to: harvester/harvester#6674

Signed-off-by: Volker Theile <[email protected]>
votdev added a commit to votdev/vm-import-controller that referenced this issue Oct 7, 2024
@votdev
VM Import Controller uses too much memory during QCOW2 conversion phase
3ee763d 
- Create the volume image using RAW disk format instead of QCOW2, so no conversion is required after downloading. This will reduce memory consumption.
- Download and write the image file in chunks (32KiB by default), so the whole file doesn't need to be downloaded completely and stored in memory before it is written to disk.
- Fix a variable name shadowing issue.
- Improve logging.

Related to: harvester/harvester#6674

Signed-off-by: Volker Theile <[email protected]>
starbops pushed a commit to harvester/vm-import-controller that referenced this issue Oct 9, 2024
@votdev @starbops
VM Import Controller uses too much memory during QCOW2 conversion phase
ebf60d5 
- Create the volume image using RAW disk format instead of QCOW2, so no conversion is required after downloading. This will reduce memory consumption.
- Download and write the image file in chunks (32KiB by default), so the whole file doesn't need to be downloaded completely and stored in memory before it is written to disk.
- Fix a variable name shadowing issue.
- Improve logging.

Related to: harvester/harvester#6674

Signed-off-by: Volker Theile <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@votdev votdev

@irishgordo irishgordo

Labels
area/vm-importer issues related to vm-import-controller addon kind/bug Issues that are defects reported by users or that we know have reached a real release reproduce/always Reproducible 100% of the time severity/2 Function working but has a major issue w/o workaround (a major incident with significant impact)
Projects
None yet
Milestone
v1.5.0
Development

No branches or pull requests
4 participants
@votdev @irishgordo @belgaied2 @harvesterhci-io-github-bot