Upgrade the OVA base OS and the Build OVA workflow to use AL2023

[teddytpc1]

Description

The OVA base OS must be upgraded to resolve vulnerabilities and EOL issues.
Some research and PoC were made here. We must use it as a guideline.

Related

• Upgrade the base OS for the Wazuh AMI and OVA to AL2023 #146

[CarlosALgit]

Update Report

This issue will require several changes to both the Github Actions workflows and the Ansible playbook so I will make a check list to keep track of the needed changes. (This list can be modified depending if more or less changes are required).
The tasks here are more or less the ones described in the implementation plan for the OVA here.

• https://github.com/wazuh/wazuh-automation/issues/2029
• Modify the Github Actions workflow to create and destroy the AWS instance using the Allocator.
• Modify the Ansible playbook to add generate_base_box.sh and setup.sh scripts to create the base VM in the AWS instance.
• Modify the Ansible playbook to install VirtualBox and Vagrant in the AWS instance.
• Create a folder inside the ova directory in which we are going to allocate the generate_base_box.sh, setup.sh, Vagrantfile files and also the Python script.
• Modify the Ansible playbbok to connect to the VM via ssh and execute the Python script.
• Create the Python script that will clone the virtual machines repo and execute the provision.sh script.
• Modify the Python script to create the network file needed inside the AL2023 VM which is inside the AWS instance.
• Modify the Ansible playbook to execute all tasks inside the AL2023 VM which is inside the AWS instance.
• Modify the Ansible playbook to execute the vboxmanage network commands and export the VM as an OVA file.
• Modify the Github Actions workflow to copy the OVA file from the AWS instance and upload it to S3.
• Review the provision.sh script and it's dependencies as these scripts run inside the VM.
• Review the Standarizing OVA step in the workflow as it executes the setOVADefault.sh script and I don't clearly know if it's necessary from now on. Probably will ask for more information to the team.

[CarlosALgit]

Update Report

Rewriting the workflow

I have been rewriting the workflow to add the creation of the base instance with the Allocator and in it install Python to be able to run the Ansible playbook.

Executing commands in the VM

Also, I have been thinking about the possible solutions to, from the workflow, execute commands inside the VM that will be inside the AWS instance.
Since what needs to be executed inside the VM are two git clones, launch a script and create a file, the most reasonable option is to bind this inside a Python script and via ssh from the AWS instance, execute the mentioned Python script.

Getting the IP of the VM

Finally, to get the IP of the VM inside the instance, you can use VirtualBox commands but I have been testing and none of the workarounds I have tried have worked.
What I have come up with is to include in the Vagrantfile the IP address I want the VM to have so I can run ssh with that specified IP.

[CarlosALgit]

Update Report

I've been rewriting the workflow to delete the created instance with the Allocator.

Also, I have added the step to copy the OVA file from the AWS instance.

Finally, I have deleted the steps of the workflow that aren't necessary anymore.

[CarlosALgit]

Update Report

Today I have been working in the Ansible playbook to execute the generate_base_box.sh and setup.sh scripts.

Also, I have moved and verified the tasks of the playbook to a newly created Python script that will be the one executed inside the VM. I have also been working in adding the new necessary steps inside this script.

[CarlosALgit]

Update Report

Today I've been changing the Ansible playbook to install the necessary tools to deploy a VM inside the AWS metal instance.
I also have renamed and reconfigured some variables that were in the workflow and then were used in the playbook that we do not need from now on and implementing other ones that we are going to need.
I also changed this comment in order to keep track of the things that are being needed as I develop the issue.

[CarlosALgit]

Update Report

Today I had a little time to continue developing the workflow due to other urgent tasks.

I implemented the wazuh-virtual-machines repository clonation, the creation of the base box using the custom scripts, the adding of the box to the instance Vagrant context and the deploy of the VM using the Vagrantfile.

Also, I created a folder inside the ova directory called workflow_assets to include all the necessary files for the development.

[CarlosALgit]

Update Report

Today I have modified the Ansible playbook to connect to the VM via SSH and run the Python script. I have also modified the Vagrantfile accordingly.
I also have added the necessary commands to stop the VM, custom it with the VirtualBox commands and export it as OVA.

Finally, I have also added two more task on this comment as I need to check if the execution of some steps or scripts is still necessary.

Tomorrow, I will develop the Python script as I have to modify the current commands as the paths that we clean have changed. I will also add the necessary customization commands to have the base VM configured as will be the future OVA.

[CarlosALgit]

Update Report

Today I have been working on developing the Python script itself. It remains to add the task to customize the network of the VM and migrate the cleaning tasks that were in the Ansible playbook.

[CarlosALgit]

Update Report

Today I have finished the code development.
The tests have started and I am now fixing them. There are some errors with the Ansible modules as I'm not familiar with them.
Tomorrow I will be working in fixing the bugs of the workflow.

[CarlosALgit]

Update Report

I have been doing tests with the workflow and solving some issues that have appeared while testing.

The last change I made is adding retries to the vagrant up command in the AWS instance so we have successfully created the al2023.box. This is the last run of the workflow in which it fails because of the vagrant up.

The next step if the vagrant up works will be to test the new Python script. I'll be doing that tomorrow.

[CarlosALgit]

Update Report

Today I have been testing the workflow and fixing the issues that were coming out. Now, it reaches the task of executing the Python script so we are close to have the OVA created.

Tomorrow I will be fixing those errors and finishing the workflow development.