From 746823c3473afe0942054836c6b8e3510e13a070 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Thu, 7 Dec 2023 15:10:09 +0000 Subject: [PATCH 01/71] feat(doc): add guest configuration introduction Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 38 +++++++++++++++++++++++++++++ source/bao_hyp/img/guest-config.svg | 4 +++ 2 files changed, 42 insertions(+) create mode 100644 source/bao_hyp/img/guest-config.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 8206198..e1332b2 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -68,6 +68,44 @@ relative to the config source file. Guests Configuration -------------------- +Within this configuration file, it is possible to define various aspects of +your virtual environment, including hardware resources, devices, and storage +options. One of the most important features of the configuration file is the +ability to define VMs, or guests (VM 1, VM 2, ... VM N). This allows to create +and manage multiple VMs, each with its own operating system and hardware +resources. + +.. figure:: img/guest-config.svg + :align: center + :name: guest-config-fig + + +The configuration of different guests is done by populating a struct called +*vmconfig*, as follows: + +.. code-block:: c + + struct vm_config { + struct image; + vaddr_t entry; + cpumap_t cpu_affinity; + colormap_t colors; + struct vm_platform platform; + }; + +For each VM, the following parameters must be specified: + +- **image** [mandatory] - corresponds to the guest image (see details in \ + `Guest Image`_) +- **entry** [mandatory] - defines the entry point address in VM's address \ + space; +- **platform description** [mandatory] - corresponds to the guest platform \ + (see details in `Virtual Machine Configuration`_); +- **cpu_affinity** [optional] - corresponds to the selection of physical CPUs \ + assigned to the virtual platform (see details in `CPU Affinity`_); +- **colors** [optional] - enables the cache coloring feature (see details in \ + `Coloring`_). + Guest Image *********** diff --git a/source/bao_hyp/img/guest-config.svg b/source/bao_hyp/img/guest-config.svg new file mode 100644 index 0000000..759d2c9 --- /dev/null +++ b/source/bao_hyp/img/guest-config.svg @@ -0,0 +1,4 @@ + + + +
Hypervisor
Hypervisor
Hardware
Hardware
vm_list
vm_list
...
...
VM N
VM N
VM 2
VM 2
VM 1
VM 1Text is not SVG - cannot display \ No newline at end of file From 407c8ddc6e79e8515393151c00bdaeb56be0835a Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 7 Dec 2023 18:20:35 +0000 Subject: [PATCH 02/71] feat(doc): add guest image configuration Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 46 +++++++++++++++++++++++++++++++++ source/bao_hyp/img/vm-image.svg | 4 +++ source/spelling_wordlist.txt | 2 ++ 3 files changed, 52 insertions(+) create mode 100644 source/bao_hyp/img/vm-image.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index e1332b2..701ff4f 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -108,6 +108,52 @@ For each VM, the following parameters must be specified: Guest Image *********** +.. _Guest Image: + +The guest ``image`` comprises a structure that describes the image \ +configuration running on the guest side. It encompasses the following options: + +- **image** [mandatory] - definition of the ``image`` to run on a given VM. + The ``image`` corresponds to the following structure: + +.. code-block:: c + + struct image { + vaddr_t base_addr; + paddr_t load_addr; + size_t size; + bool separately_loaded; + bool inplace; + }; + +where: + +- **base_addr** [mandatory] - corresponds to the ``image`` load address in \ + the VM's address space; +- **load_addr** [mandatory] - corresponds to the ``image`` load address in \ + the hypervisor address space. This value can be defined using the macro \ + VM_IMAGE_OFFSET(img_name); +- **size** [mandatory] - corresponds to the image size. This value can be \ + defined using the macro VM_IMAGE_SIZE(img_name); +- **separately_loaded** [optional] - informs the hypervisor if the VM image is + to be loaded separately by a bootloader; By default, separately_loaded is \ + set as false; +- **inplace** [optional]- use the image inplace and don’t copy the image. By \ + default, inplace is set as false; + +.. figure:: img/vm-image.svg + :align: center + :name: vm-image-fig + +To ease the process of configuring the image running on each VM, the configuration of Bao allows the +use of two different macros: + +1. **VM_IMAGE_BUILTIN** - This macro simplifies image configuration by requiring only the + ``img_name`` and the image ``base_addr``. This macro specifies both the base address and image + size. + +2. **VM_IMAGE_LOADED** - This macro requires additional configurations. It requires the definition + of image ``base_addr``, the image ``load_addr``, and the image ``size``. Virtual Machine Configuration ***************************** diff --git a/source/bao_hyp/img/vm-image.svg b/source/bao_hyp/img/vm-image.svg new file mode 100644 index 0000000..adccfbf --- /dev/null +++ b/source/bao_hyp/img/vm-image.svg @@ -0,0 +1,4 @@ + + + +
VM 0
Address
Space
VM 0...
...
...
0x01
0x01
0x00
0x00
0x02
0x02
base_addr
base_addr
VM N
Address
Space
VM N...
...
...
0x01
0x01
0x00
0x00
0x02
0x02
0xFF
0xFF
0xFE
0xFE
Hypervisor
Address
Space
Hypervisor...
0x00
0x00
0xFF
0xFF
...
...
VM_IMAGE_OFFSET()
VM_IMAGE_OFFSE...
0xFF
0xFF
0xFE
0xFE
base_addr
base_addr
load_addr
load_addr
load_addr
load_addr
VMN
image
VMN...
VM0
image
VM0...Text is not SVG - cannot display \ No newline at end of file diff --git a/source/spelling_wordlist.txt b/source/spelling_wordlist.txt index b5cb0f2..e477dbe 100644 --- a/source/spelling_wordlist.txt +++ b/source/spelling_wordlist.txt @@ -71,3 +71,5 @@ config vCPU vCPUs yaml +bootloader +inplace From c92f56b943a023e9dec45c5ae39a22fff3d10320 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 19:44:38 +0100 Subject: [PATCH 03/71] feat(doc): add VM configuration - introduction Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 23 +++++++++++++++++++++++ source/spelling_wordlist.txt | 1 + 2 files changed, 24 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 701ff4f..27cdc5d 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -158,23 +158,46 @@ use of two different macros: Virtual Machine Configuration ***************************** +The VM configuration enables users to define the characteristics of each the \ +virtualized platform. It encompasses critical details that define the VM's \ +run-time environment, performance, and overall capabilities. By customizing \ +this configuration, users can tailor the virtual platform to suit specific \ +workload requirements and application needs for their virtual machines. The \ +configuration encompasses the definition of: + +- **Number of virtual CPUs** - see details in `Number of vCPUs`_; +- **Virtual memory regions** - see details in `Memory Mapping`_;` +- **Inter-Process Comunication (IPC)** - see details in \ + `Inter-Process Communication (IPC)`_; +- **Devices** - see details in `Devices`_; +- **Memory Management** - see details in `Memory Management`_; +- **Architectural-Specific Configurations** - see details in \ + `Architectural-Specific Configurations`_; + 1. Number of vCPUs ################## +.. _Number of vCPUs: + 2. Memory Mapping ################# +.. _Memory Mapping: 3. Inter-Process Communication (IPC) #################################### +.. _Inter-Process Communication (IPC): 4. Devices ########## +.. _Devices: 5. Memory Management #################### +.. _Memory Management: 6. Architectural-Specific Configurations ######################################## +.. _Architectural-Specific Configurations: CPU Affinity ************ diff --git a/source/spelling_wordlist.txt b/source/spelling_wordlist.txt index e477dbe..c6bdfe9 100644 --- a/source/spelling_wordlist.txt +++ b/source/spelling_wordlist.txt @@ -73,3 +73,4 @@ vCPUs yaml bootloader inplace +virtualized From 413b37b98df6d3315985d2ca0112c736f3c0754d Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 19:45:53 +0100 Subject: [PATCH 04/71] feat(doc): add VM configuration - number of vCPUs Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 27cdc5d..f5bed46 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -178,6 +178,8 @@ configuration encompasses the definition of: ################## .. _Number of vCPUs: +- **cpu_num** [mandatory] - defines the number of CPUs assigned to the VM; + 2. Memory Mapping ################# From b6db67a40b469b4d3a01c60b8477ac57198e85ab Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 19:51:31 +0100 Subject: [PATCH 05/71] feat(doc): add VM configuration - memory mapping Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index f5bed46..97d680c 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -185,6 +185,32 @@ configuration encompasses the definition of: ################# .. _Memory Mapping: +- **region_num** [mandatory] - defines the number of memory regions mapped to \ + the VM. This structure contemplates the following parameters: + +.. code-block:: c + + struct vm_mem_region { + paddr_t base; + size_t size; + bool place_phys; + paddr_t phys; + }; + +where: + +- **base** [mandatory] - corresponds to the base virtual address of the \ + memory region; +- **size** [mandatory] - corresponds to the size of the memory region; +- **place_phys** [optional] - the memory region is mapped into the virtual \ + memory, and it's important to note that the virtual address (VA) might not \ + necessarily be the same as the physical address (PA). When "place_phys" is + set to true, the virtual address corresponds to the physical address. \ + If ``place_phys`` equals to true, it allows to specify the physical address \ + of the memory region. By default, ``place_phys`` equals to false; +- **phys** [mandatory if ``place_phys`` is true] - it corresponds to the \ + physical address where the memory region should be mapped; + 3. Inter-Process Communication (IPC) #################################### .. _Inter-Process Communication (IPC): From da5b673c6078320c4931db1ae5931a966b42d4df Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 19:54:32 +0100 Subject: [PATCH 06/71] feat(doc): add VM configuration - IPC Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 97d680c..48be1d1 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -215,6 +215,38 @@ where: #################################### .. _Inter-Process Communication (IPC): +- **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By \ + default, ``ipc_num`` equals to zero; +- **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification \ + of the IPC and is configured through the following structure: + +.. code-block:: c + + struct ipc { + paddr_t base; + size_t size; + size_t shmem_id; + size_t interrupt_num; + irqid_t *interrupts; + }; + + +where: + +- **base** [mandatory] - corresponds to the virtual base address of the IPC \ + memory region; +- **size** [mandatory] - corresponds to the size of the IPC memory region; +- **shmem_id** [mandatory] - corresponds to the ID of the shared memory \ + associated with the IPC; +- **interrupt_num** [mandatory] - defines the number of interrupts assigned \ + to the IPC; +- **interrupts** [mandatory if *interrupt_num* > 0] - defines a list of \ + interrupt IDs assigned to the IPC - ``(irqid_t[]) {irq_1, ..., irq_n}``; + +.. warning:: + Specifying a number of interrupts in the ``interrupts`` buffer that \ + differs from the ``interrupt_num`` may result in unforeseen behavior. + 4. Devices ########## .. _Devices: From b78ee89a6b939667fda0584e0e1b43bb905220d6 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 19:57:52 +0100 Subject: [PATCH 07/71] feat(doc): add VM configuration - devices Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 31 +++++++++++++++++++++++++++++++ source/spelling_wordlist.txt | 1 + 2 files changed, 32 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 48be1d1..68d8576 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -251,6 +251,37 @@ where: ########## .. _Devices: +- **dev_num** [mandatory] - corresponds to the number of devices assigned to \ + the VM; +- **devs** [mandatory if *dev_num* > 0] - corresponds to the specification of \ + the VM's devices and is configured through the following structure: + +.. code-block:: c + + struct vm_dev_region { + paddr_t pa; + vaddr_t va; + size_t size; + size_t interrupt_num; + irqid_t *interrupts; + streamid_t id; /* bus master id for iommu effects */ + }; + +where: + +- **pa** [mandatory] - corresponds to the base physical address of the device; +- **va** [mandatory] - corresponds to the base virtual address of the device; +- **size** [mandatory] - corresponds to the size of the device memory region; +- **interrupt_num** [optional] - corresponds to the number of interrupts \ + generated by the device to the VM. By default, ``interrupt_num`` equals to 0; +- **interrupts** [mandatory if *interrupt_num*>0] - defines a list of \ + interrupt IDs generated by the device - ``(irqid_t[]) {irq_1, ..., irq_n};`` +- **id** [optional] - corresponds to the bus master id for iommu effects: + +.. warning:: + Specifying a number of interrupts in the ``interrupts`` buffer that \ + differs from the ``interrupt_num`` may result in unforeseen behavior. + 5. Memory Management #################### .. _Memory Management: diff --git a/source/spelling_wordlist.txt b/source/spelling_wordlist.txt index c6bdfe9..0835633 100644 --- a/source/spelling_wordlist.txt +++ b/source/spelling_wordlist.txt @@ -74,3 +74,4 @@ yaml bootloader inplace virtualized +iommu From c22ba3958192a35fa9939ebd4c029348a23c0686 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 7 Dec 2023 18:28:01 +0000 Subject: [PATCH 08/71] feat(doc): add VM configuration - memory management Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 6 ++++++ source/spelling_wordlist.txt | 3 +++ 2 files changed, 9 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 68d8576..fd75e9c 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -286,6 +286,12 @@ where: #################### .. _Memory Management: + **mmu** [optional] - In MPU-based platforms which might also support virtual \ + memory (i.e. aarch64 cortex-r) the hypervisor sets up the VM using an MPU \ + by default. If the user wants this VM to use the MMU they must set the \ + config ``mmu`` parameter to true; + + 6. Architectural-Specific Configurations ######################################## .. _Architectural-Specific Configurations: diff --git a/source/spelling_wordlist.txt b/source/spelling_wordlist.txt index 0835633..4e6d3ec 100644 --- a/source/spelling_wordlist.txt +++ b/source/spelling_wordlist.txt @@ -75,3 +75,6 @@ bootloader inplace virtualized iommu +MPU +aarch64 +aarch From 3d43ffb97fefc7df04196258c387eef48fd28149 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 20:05:49 +0100 Subject: [PATCH 09/71] feat(doc): add VM configuration - architectural-specific configurations Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index fd75e9c..52da778 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -296,6 +296,41 @@ where: ######################################## .. _Architectural-Specific Configurations: +- **arch** [mandatory] - allows the definition of architecture dependent \ + configurations and is configured through the following structure: + +.. code-block:: c + + struct arch_vm_platform { + + // Configuration of the Generic Interrupt Controller (GIC) + struct vgic_dscrp { + paddr_t gicd_addr; + paddr_t gicc_addr; + paddr_t gicr_addr; + size_t interrupt_num; + } gic; + + // Configuration of the System Memory Management Unit (SMMU) + struct { + streamid_t global_mask; + size_t group_num; + struct smmu_group { + streamid_t mask; + streamid_t id; + } *groups; + } smmu; + }; + + +where: + +- **vgic_dscrp** [mandatory] - corresponds to the configuration of the \ + Generic Interrupt Controller (GIC); +- **vgic_dscrp** [optional] - corresponds to the configuration of the System \ + Memory Management Unit (SMMU); + + CPU Affinity ************ From 1300f5aa9867f46066470190293972044b951692 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 20:08:27 +0100 Subject: [PATCH 10/71] feat(doc): add cpu affinity Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 16 ++++++++++++++++ source/bao_hyp/img/cpu-affinity.svg | 4 ++++ 2 files changed, 20 insertions(+) create mode 100644 source/bao_hyp/img/cpu-affinity.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 52da778..e43830d 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -334,6 +334,22 @@ where: CPU Affinity ************ +The configuration file of the Bao hypervisor also enables the definition of +core affinity, which involves selecting the physical core where the guest +should run. + +.. figure:: img/cpu-affinity.svg + :align: center + :name: cpu-affinity-fig + +This functionality is achieved through the following configuration parameter: + +- **cpu_affinity** [optional] - corresponds to a bitmap signaling the \ + preferred physical CPUs assigned to the VM. If this value is mutually \ + exclusive for all the VMs, the physical CPUs assigned to each VM follow the \ + bitmap. Otherwise (in case of bit overlap or lack of affinity definition), \ + the CPU assignment is defined by the hypervisor; + Coloring ******** diff --git a/source/bao_hyp/img/cpu-affinity.svg b/source/bao_hyp/img/cpu-affinity.svg new file mode 100644 index 0000000..42ab6c2 --- /dev/null +++ b/source/bao_hyp/img/cpu-affinity.svg @@ -0,0 +1,4 @@ + + + +
VM 1
cpu_affinity: 0x0100
VM 1...
Hypervisor
Hypervisor
VM 0
cpu_affinity: 0x1000
VM 0...
VM 2
cpu_affinity: 0x0011
VM 2...
Memory
Memory
CPU0
VM 0
CPU0...
CPU1
VM 1
CPU1...
CPU2
VM 2
CPU2...
CPU3
VM 2
CPU3...Text is not SVG - cannot display \ No newline at end of file From 7415b8d26fb478601a5a82a4a5297b9296b261f9 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 20:10:42 +0100 Subject: [PATCH 11/71] feat(doc): add cache coloring Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 10 ++++++++++ source/bao_hyp/img/llc-colors.svg | 4 ++++ 2 files changed, 14 insertions(+) create mode 100644 source/bao_hyp/img/llc-colors.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index e43830d..d7a5633 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -353,6 +353,16 @@ This functionality is achieved through the following configuration parameter: Coloring ******** +- **colors** [optional] - corresponds to a bitmap for the assigned cache \ + colors of the VM. This value is truncated depending on the number of \ + available colors calculated at run-time, i.e., its platform-dependent. By \ + default, the coloring mechanism is not active. For instance, the following \ + picture depicts a hypothetical setup with a 50/50 coloring scheme; + +.. figure:: img/llc-colors.svg + :align: center + :name: llc-colors-fig + Shared Memory Configuration --------------------------- diff --git a/source/bao_hyp/img/llc-colors.svg b/source/bao_hyp/img/llc-colors.svg new file mode 100644 index 0000000..dd9b14f --- /dev/null +++ b/source/bao_hyp/img/llc-colors.svg @@ -0,0 +1,4 @@ + + + +









Hypervisor
Hypervisor
VM 0

VM 0...
VM 1

VM 1
LLC
LLC
CPU0
CPU0
CPU1
CPU1
CPU2
CPU2
CPU3
CPU3

colors: 0xF0F0F0
colors: 0xF0F0F0
colors: 0x0F0F0F
colors: 0x0F0F0FText is not SVG - cannot display \ No newline at end of file From 77c44296ea4aaa1180b569c0e28312a9dd2805b2 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 7 Dec 2023 16:17:27 +0000 Subject: [PATCH 12/71] fix(doc): include format tweaks to use the new column limit of 100 Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 185 +++++++++++++++++--------------------- 1 file changed, 82 insertions(+), 103 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index d7a5633..f9ec42a 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -68,11 +68,10 @@ relative to the config source file. Guests Configuration -------------------- -Within this configuration file, it is possible to define various aspects of -your virtual environment, including hardware resources, devices, and storage -options. One of the most important features of the configuration file is the -ability to define VMs, or guests (VM 1, VM 2, ... VM N). This allows to create -and manage multiple VMs, each with its own operating system and hardware +Within this configuration file, it is possible to define various aspects of your virtual +environment, including hardware resources, devices, and storage options. One of the most important +features of the configuration file is the ability to define VMs, or guests (VM 1, VM 2, ... VM N). +This allows to create and manage multiple VMs, each with its own operating system and hardware resources. .. figure:: img/guest-config.svg @@ -80,8 +79,7 @@ resources. :name: guest-config-fig -The configuration of different guests is done by populating a struct called -*vmconfig*, as follows: +The configuration of different guests is done by populating a struct called *vmconfig*, as follows: .. code-block:: c @@ -95,26 +93,23 @@ The configuration of different guests is done by populating a struct called For each VM, the following parameters must be specified: -- **image** [mandatory] - corresponds to the guest image (see details in \ - `Guest Image`_) -- **entry** [mandatory] - defines the entry point address in VM's address \ - space; -- **platform description** [mandatory] - corresponds to the guest platform \ - (see details in `Virtual Machine Configuration`_); -- **cpu_affinity** [optional] - corresponds to the selection of physical CPUs \ - assigned to the virtual platform (see details in `CPU Affinity`_); -- **colors** [optional] - enables the cache coloring feature (see details in \ - `Coloring`_). +- **image** [mandatory] - corresponds to the guest image (see details in `Guest Image`_) +- **entry** [mandatory] - defines the entry point address in VM's address space; +- **platform description** [mandatory] - corresponds to the guest platform (see details in `Virtual + Machine Configuration`_); +- **cpu_affinity** [optional] - corresponds to the selection of physical CPUs assigned to the + virtual platform (see details in `CPU Affinity`_); +- **colors** [optional] - enables the cache coloring feature (see details in `Coloring`_). Guest Image *********** .. _Guest Image: -The guest ``image`` comprises a structure that describes the image \ -configuration running on the guest side. It encompasses the following options: +The guest ``image`` comprises a structure that describes the image configuration running on the +guest side. It encompasses the following options: -- **image** [mandatory] - definition of the ``image`` to run on a given VM. - The ``image`` corresponds to the following structure: +- **image** [mandatory] - definition of the ``image`` to run on a given VM. The ``image`` + corresponds to the following structure: .. code-block:: c @@ -128,25 +123,22 @@ configuration running on the guest side. It encompasses the following options: where: -- **base_addr** [mandatory] - corresponds to the ``image`` load address in \ - the VM's address space; -- **load_addr** [mandatory] - corresponds to the ``image`` load address in \ - the hypervisor address space. This value can be defined using the macro \ - VM_IMAGE_OFFSET(img_name); -- **size** [mandatory] - corresponds to the image size. This value can be \ - defined using the macro VM_IMAGE_SIZE(img_name); -- **separately_loaded** [optional] - informs the hypervisor if the VM image is - to be loaded separately by a bootloader; By default, separately_loaded is \ +- **base_addr** [mandatory] - corresponds to the ``image`` load address in the VM's address space; +- **load_addr** [mandatory] - corresponds to the ``image`` load address in the hypervisor address + space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name); +- **size** [mandatory] - corresponds to the image size. This value can be defined using the macro + VM_IMAGE_SIZE(img_name); +- **separately_loaded** [optional] - informs the hypervisor if the VM image is to be loaded + separately by a bootloader; By default, separately_loaded is set as false; +- **inplace** [optional]- use the image inplace and don’t copy the image. By default, inplace is set as false; -- **inplace** [optional]- use the image inplace and don’t copy the image. By \ - default, inplace is set as false; .. figure:: img/vm-image.svg :align: center :name: vm-image-fig -To ease the process of configuring the image running on each VM, the configuration of Bao allows the -use of two different macros: +To ease the process of configuring the image running on each VM, the configuration of Bao allows +the use of two different macros: 1. **VM_IMAGE_BUILTIN** - This macro simplifies image configuration by requiring only the ``img_name`` and the image ``base_addr``. This macro specifies both the base address and image @@ -158,21 +150,19 @@ use of two different macros: Virtual Machine Configuration ***************************** -The VM configuration enables users to define the characteristics of each the \ -virtualized platform. It encompasses critical details that define the VM's \ -run-time environment, performance, and overall capabilities. By customizing \ -this configuration, users can tailor the virtual platform to suit specific \ -workload requirements and application needs for their virtual machines. The \ -configuration encompasses the definition of: +The VM configuration enables users to define the characteristics of each the virtualized platform. +It encompasses critical details that define the VM's run-time environment, performance, and overall +capabilities. By customizing this configuration, users can tailor the virtual platform to suit +specific workload requirements and application needs for their virtual machines. The configuration +encompasses the definition of: - **Number of virtual CPUs** - see details in `Number of vCPUs`_; - **Virtual memory regions** - see details in `Memory Mapping`_;` -- **Inter-Process Comunication (IPC)** - see details in \ - `Inter-Process Communication (IPC)`_; +- **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_; - **Devices** - see details in `Devices`_; - **Memory Management** - see details in `Memory Management`_; -- **Architectural-Specific Configurations** - see details in \ - `Architectural-Specific Configurations`_; +- **Architectural-Specific Configurations** - see details in `Architectural-Specific + Configurations`_; 1. Number of vCPUs ################## @@ -185,8 +175,8 @@ configuration encompasses the definition of: ################# .. _Memory Mapping: -- **region_num** [mandatory] - defines the number of memory regions mapped to \ - the VM. This structure contemplates the following parameters: +- **region_num** [mandatory] - defines the number of memory regions mapped to the VM. This + structure contemplates the following parameters: .. code-block:: c @@ -199,26 +189,24 @@ configuration encompasses the definition of: where: -- **base** [mandatory] - corresponds to the base virtual address of the \ - memory region; +- **base** [mandatory] - corresponds to the base virtual address of the memory region; - **size** [mandatory] - corresponds to the size of the memory region; -- **place_phys** [optional] - the memory region is mapped into the virtual \ - memory, and it's important to note that the virtual address (VA) might not \ - necessarily be the same as the physical address (PA). When "place_phys" is - set to true, the virtual address corresponds to the physical address. \ - If ``place_phys`` equals to true, it allows to specify the physical address \ - of the memory region. By default, ``place_phys`` equals to false; -- **phys** [mandatory if ``place_phys`` is true] - it corresponds to the \ - physical address where the memory region should be mapped; +- **place_phys** [optional] - the memory region is mapped into the virtual memory, and it's + important to note that the virtual address (VA) might not necessarily be the same as the physical + address (PA). When "place_phys" is set to true, the virtual address corresponds to the physical + address. If ``place_phys`` equals to true, it allows to specify the physical address of the + memory region. By default, ``place_phys`` equals to false; +- **phys** [mandatory if ``place_phys`` is true] - it corresponds to the physical address where the + memory region should be mapped; 3. Inter-Process Communication (IPC) #################################### .. _Inter-Process Communication (IPC): -- **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By \ - default, ``ipc_num`` equals to zero; -- **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification \ - of the IPC and is configured through the following structure: +- **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num`` + equals to zero; +- **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification of the IPC and is + configured through the following structure: .. code-block:: c @@ -233,28 +221,24 @@ where: where: -- **base** [mandatory] - corresponds to the virtual base address of the IPC \ - memory region; +- **base** [mandatory] - corresponds to the virtual base address of the IPC memory region; - **size** [mandatory] - corresponds to the size of the IPC memory region; -- **shmem_id** [mandatory] - corresponds to the ID of the shared memory \ - associated with the IPC; -- **interrupt_num** [mandatory] - defines the number of interrupts assigned \ - to the IPC; -- **interrupts** [mandatory if *interrupt_num* > 0] - defines a list of \ - interrupt IDs assigned to the IPC - ``(irqid_t[]) {irq_1, ..., irq_n}``; +- **shmem_id** [mandatory] - corresponds to the ID of the shared memory associated with the IPC; +- **interrupt_num** [mandatory] - defines the number of interrupts assigned to the IPC; +- **interrupts** [mandatory if *interrupt_num* > 0] - defines a list of interrupt IDs assigned to + the IPC - ``(irqid_t[]) {irq_1, ..., irq_n}``; .. warning:: - Specifying a number of interrupts in the ``interrupts`` buffer that \ - differs from the ``interrupt_num`` may result in unforeseen behavior. + Specifying a number of interrupts in the ``interrupts`` buffer that differs from the + ``interrupt_num`` may result in unforeseen behavior. 4. Devices ########## .. _Devices: -- **dev_num** [mandatory] - corresponds to the number of devices assigned to \ - the VM; -- **devs** [mandatory if *dev_num* > 0] - corresponds to the specification of \ - the VM's devices and is configured through the following structure: +- **dev_num** [mandatory] - corresponds to the number of devices assigned to the VM; +- **devs** [mandatory if *dev_num* > 0] - corresponds to the specification of the VM's devices and + is configured through the following structure: .. code-block:: c @@ -272,32 +256,30 @@ where: - **pa** [mandatory] - corresponds to the base physical address of the device; - **va** [mandatory] - corresponds to the base virtual address of the device; - **size** [mandatory] - corresponds to the size of the device memory region; -- **interrupt_num** [optional] - corresponds to the number of interrupts \ - generated by the device to the VM. By default, ``interrupt_num`` equals to 0; -- **interrupts** [mandatory if *interrupt_num*>0] - defines a list of \ - interrupt IDs generated by the device - ``(irqid_t[]) {irq_1, ..., irq_n};`` +- **interrupt_num** [optional] - corresponds to the number of interrupts generated by the device to + the VM. By default, ``interrupt_num`` equals to 0; +- **interrupts** [mandatory if *interrupt_num*>0] - defines a list of interrupt IDs generated by + the device - ``(irqid_t[]) {irq_1, ..., irq_n};`` - **id** [optional] - corresponds to the bus master id for iommu effects: .. warning:: - Specifying a number of interrupts in the ``interrupts`` buffer that \ - differs from the ``interrupt_num`` may result in unforeseen behavior. + Specifying a number of interrupts in the ``interrupts`` buffer that differs from the + ``interrupt_num`` may result in unforeseen behavior. 5. Memory Management #################### .. _Memory Management: - **mmu** [optional] - In MPU-based platforms which might also support virtual \ - memory (i.e. aarch64 cortex-r) the hypervisor sets up the VM using an MPU \ - by default. If the user wants this VM to use the MMU they must set the \ - config ``mmu`` parameter to true; - + **mmu** [optional] - In MPU-based platforms which might also support virtual memory (i.e. aarch64 + cortex-r) the hypervisor sets up the VM using an MPU by default. If the user wants this VM to use + the MMU they must set the config ``mmu`` parameter to true; 6. Architectural-Specific Configurations ######################################## .. _Architectural-Specific Configurations: -- **arch** [mandatory] - allows the definition of architecture dependent \ - configurations and is configured through the following structure: +- **arch** [mandatory] - allows the definition of architecture dependent configurations and is + configured through the following structure: .. code-block:: c @@ -325,18 +307,17 @@ where: where: -- **vgic_dscrp** [mandatory] - corresponds to the configuration of the \ - Generic Interrupt Controller (GIC); -- **vgic_dscrp** [optional] - corresponds to the configuration of the System \ - Memory Management Unit (SMMU); +- **vgic_dscrp** [mandatory] - corresponds to the configuration of the Generic Interrupt Controller + (GIC); +- **vgic_dscrp** [optional] - corresponds to the configuration of the System Memory Management Unit + (SMMU); CPU Affinity ************ -The configuration file of the Bao hypervisor also enables the definition of -core affinity, which involves selecting the physical core where the guest -should run. +The configuration file of the Bao hypervisor also enables the definition of core affinity, which +involves selecting the physical core where the guest should run. .. figure:: img/cpu-affinity.svg :align: center @@ -344,19 +325,17 @@ should run. This functionality is achieved through the following configuration parameter: -- **cpu_affinity** [optional] - corresponds to a bitmap signaling the \ - preferred physical CPUs assigned to the VM. If this value is mutually \ - exclusive for all the VMs, the physical CPUs assigned to each VM follow the \ - bitmap. Otherwise (in case of bit overlap or lack of affinity definition), \ - the CPU assignment is defined by the hypervisor; +- **cpu_affinity** [optional] - corresponds to a bitmap signaling the preferred physical CPUs + assigned to the VM. If this value is mutually exclusive for all the VMs, the physical CPUs + assigned to each VM follow the bitmap. Otherwise (in case of bit overlap or lack of affinity + definition), the CPU assignment is defined by the hypervisor; Coloring ******** -- **colors** [optional] - corresponds to a bitmap for the assigned cache \ - colors of the VM. This value is truncated depending on the number of \ - available colors calculated at run-time, i.e., its platform-dependent. By \ - default, the coloring mechanism is not active. For instance, the following \ +- **colors** [optional] - corresponds to a bitmap for the assigned cache colors of the VM. This + value is truncated depending on the number of available colors calculated at run-time, i.e., its + platform-dependent. By default, the coloring mechanism is not active. For instance, the following picture depicts a hypothetical setup with a 50/50 coloring scheme; .. figure:: img/llc-colors.svg From 223e053537fd31fa09d5c0361dc03c00369200e3 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 10:50:00 +0000 Subject: [PATCH 13/71] fix(img): update cpu-affinity Signed-off-by: Diogo21Costa --- source/bao_hyp/img/cpu-affinity.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/img/cpu-affinity.svg b/source/bao_hyp/img/cpu-affinity.svg index 42ab6c2..cd886f3 100644 --- a/source/bao_hyp/img/cpu-affinity.svg +++ b/source/bao_hyp/img/cpu-affinity.svg @@ -1,4 +1,4 @@ -
VM 1
cpu_affinity: 0x0100
VM 1...
Hypervisor
Hypervisor
VM 0
cpu_affinity: 0x1000
VM 0...
VM 2
cpu_affinity: 0x0011
VM 2...
Memory
Memory
CPU0
VM 0
CPU0...
CPU1
VM 1
CPU1...
CPU2
VM 2
CPU2...
CPU3
VM 2
CPU3...Text is not SVG - cannot display \ No newline at end of file +
VM 2
cpu_affinity: 0b0010
VM 2...
Hypervisor
Hypervisor
VM 1
cpu_affinity: 0b0001
VM 1...
VM 3
cpu_affinity: 0b1100
VM 3...
Memory
Memory
CPU1
VM 1
CPU1...
CPU2
VM 2
CPU2...
CPU3
VM 3
CPU3...
CPU4
VM 3
CPU4...Text is not SVG - cannot display \ No newline at end of file From b5096f066240ab0ae3aeb42a4522629b5de1082c Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 23 Dec 2023 11:19:08 +0000 Subject: [PATCH 14/71] fix(heading): update section naming Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index f9ec42a..6920757 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -65,7 +65,7 @@ describe the image running on the VM (see `Guest Image`_); 2. A C string with the guest image's binary file path. It can be either an absolute path or a path relative to the config source file. -Guests Configuration +VM Configuration -------------------- Within this configuration file, it is possible to define various aspects of your virtual From 88b3e28a7b6047369c6d3d96ae9101978d5faecd Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 10:48:30 +0000 Subject: [PATCH 15/71] fix(vm_config): fix introduction text Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 6920757..97dc863 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -68,11 +68,10 @@ relative to the config source file. VM Configuration -------------------- -Within this configuration file, it is possible to define various aspects of your virtual -environment, including hardware resources, devices, and storage options. One of the most important -features of the configuration file is the ability to define VMs, or guests (VM 1, VM 2, ... VM N). -This allows to create and manage multiple VMs, each with its own operating system and hardware -resources. +Bao's configuration file allows you to partition the platforms' hardware resources, such as CPU +cores, memory, or devices, by assigning them one or more VMs. It also allows you to configure the +guest image to run on that VM. In Bao, resources are exclusively assigned to each VM, including +communication channels between two or more VMs, which may utilize shared memory or dedicated links. .. figure:: img/guest-config.svg :align: center From fedaf738e68d7f2e46b9b6e4805f59b905b6d82a Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 23 Dec 2023 11:56:48 +0000 Subject: [PATCH 16/71] fix(vm_config): update vm entry description Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 97dc863..7d96e79 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -78,7 +78,8 @@ communication channels between two or more VMs, which may utilize shared memory :name: guest-config-fig -The configuration of different guests is done by populating a struct called *vmconfig*, as follows: +Each entry in the ``vmlist`` mentioned earlier is a ``vm_config`` struct, defining the +configuration for different guests by populating the following struct called *vmconfig*: .. code-block:: c @@ -90,7 +91,9 @@ The configuration of different guests is done by populating a struct called *vmc struct vm_platform platform; }; -For each VM, the following parameters must be specified: +Each entry in this list represents a unique VM configuration, defining its image, memory address, +CPU affinity, color mapping, and platform details. For each VM, the following parameters must be +specified: - **image** [mandatory] - corresponds to the guest image (see details in `Guest Image`_) - **entry** [mandatory] - defines the entry point address in VM's address space; From 9d82a525c686ec909a7ce7ed37303dda3cc973ae Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 10:53:18 +0000 Subject: [PATCH 17/71] fix(vm_config): fix struct vm_image declaration Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 7d96e79..af93bf3 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -84,7 +84,7 @@ configuration for different guests by populating the following struct called *vm .. code-block:: c struct vm_config { - struct image; + struct vm_image image; vaddr_t entry; cpumap_t cpu_affinity; colormap_t colors; @@ -115,13 +115,13 @@ guest side. It encompasses the following options: .. code-block:: c - struct image { + struct vm_image { vaddr_t base_addr; paddr_t load_addr; size_t size; bool separately_loaded; bool inplace; - }; + } image; where: From 8e218ad39833bff3581b40f15186c15bce74f875 Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Fri, 5 Jan 2024 10:10:09 +0000 Subject: [PATCH 18/71] fix(guest_image): update size field description Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index af93bf3..a7bcf4c 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -128,8 +128,8 @@ where: - **base_addr** [mandatory] - corresponds to the ``image`` load address in the VM's address space; - **load_addr** [mandatory] - corresponds to the ``image`` load address in the hypervisor address space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name); -- **size** [mandatory] - corresponds to the image size. This value can be defined using the macro - VM_IMAGE_SIZE(img_name); +- **size** [mandatory] - corresponds to the image size. For builtin images declared using + `VM_IMAGE`, this value can be defined using the macro VM_IMAGE_SIZE(img_name); - **separately_loaded** [optional] - informs the hypervisor if the VM image is to be loaded separately by a bootloader; By default, separately_loaded is set as false; - **inplace** [optional]- use the image inplace and don’t copy the image. By default, inplace is From 4af39fd99f8ee31aeb562091d5c9112ef40fb072 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 10:56:39 +0000 Subject: [PATCH 19/71] fix(typo): improve Virtual Machine Configuration section Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index a7bcf4c..79b277c 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -152,8 +152,8 @@ the use of two different macros: Virtual Machine Configuration ***************************** -The VM configuration enables users to define the characteristics of each the virtualized platform. -It encompasses critical details that define the VM's run-time environment, performance, and overall +The VM configuration enables users to define the characteristics of each virtualized platform. It +encompasses critical details that define the VM's run-time environment, performance, and overall capabilities. By customizing this configuration, users can tailor the virtual platform to suit specific workload requirements and application needs for their virtual machines. The configuration encompasses the definition of: From 31f18acbd6ba42cb2862f91496075a673f2ff24f Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 11:09:06 +0000 Subject: [PATCH 20/71] update(doc): refactor Virtual Machine Configuration description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 79b277c..51e416a 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -154,10 +154,27 @@ Virtual Machine Configuration The VM configuration enables users to define the characteristics of each virtualized platform. It encompasses critical details that define the VM's run-time environment, performance, and overall -capabilities. By customizing this configuration, users can tailor the virtual platform to suit -specific workload requirements and application needs for their virtual machines. The configuration -encompasses the definition of: +capabilities. The virtual machine configuration is performed by populating the structure `struct +vm_platform`, outlined below: +.. code-block:: c + + struct vm_platform { + size_t cpu_num; + size_t region_num; + struct vm_mem_region* regions; + size_t ipc_num; + struct ipc* ipcs; + size_t dev_num; + struct vm_dev_region* devs; + bool mmu; + struct arch_vm_platform arch; + } + +By customizing this configuration, users can tailor the virtual platform to suit specific workload +requirements and application needs for their virtual machines. The configuration encompasses the +definition of: + - **Number of virtual CPUs** - see details in `Number of vCPUs`_; - **Virtual memory regions** - see details in `Memory Mapping`_;` - **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_; From 01557573a7211bf9021ac26fb1d90e1012a1f830 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 11:11:37 +0000 Subject: [PATCH 21/71] update(VM Config): improve struct description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 51e416a..0b2bc82 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -95,13 +95,15 @@ Each entry in this list represents a unique VM configuration, defining its image CPU affinity, color mapping, and platform details. For each VM, the following parameters must be specified: -- **image** [mandatory] - corresponds to the guest image (see details in `Guest Image`_) -- **entry** [mandatory] - defines the entry point address in VM's address space; -- **platform description** [mandatory] - corresponds to the guest platform (see details in `Virtual - Machine Configuration`_); -- **cpu_affinity** [optional] - corresponds to the selection of physical CPUs assigned to the - virtual platform (see details in `CPU Affinity`_); -- **colors** [optional] - enables the cache coloring feature (see details in `Coloring`_). +- **image** [mandatory] - a structure containing information about guest image loading (see details + in `Guest Image`_) +- **entry** [mandatory] - defines the entry point address in VM's address space; +- **platform description** [mandatory] - a description of the VM platform, defining its resource + assignments and requirements (see details in `Virtual Machine Configuration`_); +- **cpu_affinity** [optional] - defines the affinity of the VM's vCPUs to the physical CPUs + assigned to the virtual platform (see details in `CPU Affinity`_); +- **colors** [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see + details in `Coloring`_). Guest Image *********** From 4aa86ece9603ea5a3e4eac48d28fbb9c5dc64e60 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 11:21:48 +0000 Subject: [PATCH 22/71] update(doc): improve Virtual Machine Configuration description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 0b2bc82..74fda58 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -177,8 +177,8 @@ By customizing this configuration, users can tailor the virtual platform to suit requirements and application needs for their virtual machines. The configuration encompasses the definition of: -- **Number of virtual CPUs** - see details in `Number of vCPUs`_; -- **Virtual memory regions** - see details in `Memory Mapping`_;` +- **Number of CPUs** - see details in `Number of vCPUs`_; +- **Memory regions** - see details in `Memory Mapping`_;` - **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_; - **Devices** - see details in `Devices`_; - **Memory Management** - see details in `Memory Management`_; From d6c2e0a374af93d2c91877365d1e84b0211c2f29 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 11:32:36 +0000 Subject: [PATCH 23/71] update(n_cpus): add warning to number of vCPUs description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 74fda58..dd5fc1a 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -191,6 +191,10 @@ definition of: - **cpu_num** [mandatory] - defines the number of CPUs assigned to the VM; +.. warning:: + Ensure that the cumulative count of CPUs allocated across all VMs listed in the `vmlist` does not + exceed the total number of available CPUs on the platform. Failing to adhere to this requirement + might result in the guest failing to boot without any warning. 2. Memory Mapping ################# From 580d62645aa02c4b599ab43676ee78359a8d9622 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 11:37:23 +0000 Subject: [PATCH 24/71] fix(vm_config): rename memory regions subsection Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index dd5fc1a..6138448 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -178,7 +178,7 @@ requirements and application needs for their virtual machines. The configuration definition of: - **Number of CPUs** - see details in `Number of vCPUs`_; -- **Memory regions** - see details in `Memory Mapping`_;` +- **Memory regions** - see details in `Memory Regions`_;` - **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_; - **Devices** - see details in `Devices`_; - **Memory Management** - see details in `Memory Management`_; @@ -196,9 +196,9 @@ definition of: exceed the total number of available CPUs on the platform. Failing to adhere to this requirement might result in the guest failing to boot without any warning. -2. Memory Mapping +2. Memory Regions ################# -.. _Memory Mapping: +.. _Memory Regions: - **region_num** [mandatory] - defines the number of memory regions mapped to the VM. This structure contemplates the following parameters: From 5f9b9ea89e868fa245b2b6774b942bb295594d6d Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 11:40:57 +0000 Subject: [PATCH 25/71] fix(vm_config): improve region_num description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 6138448..530b9e7 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -200,8 +200,10 @@ definition of: ################# .. _Memory Regions: -- **region_num** [mandatory] - defines the number of memory regions mapped to the VM. This - structure contemplates the following parameters: +- **region_num** [mandatory] - defines the number of memory regions in the VM, specifically, the + number of `vm_mem_region` entries in the `vm_platform`'s `regions` list. + +The `vm_mem_region` structure contemplates the following parameters: .. code-block:: c From 9bf7c0ccdebf35701f89b15a9d8a52beeb2257ef Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 14:05:55 +0000 Subject: [PATCH 26/71] update(mem_reg): include small entry Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 530b9e7..b7d3df7 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -200,10 +200,18 @@ definition of: ################# .. _Memory Regions: +Bao employs a two-stage translation mechanism to manage VM memory. Each VM memory region is +delineated by its virtual address in the second stage. Generally, Bao manages the allocation of +physical memory for these regions. However, there is an option to define the physical memory region +where the specific region will be mapped. + +For each VM, users can define multiple memory regions. To facilitate this, users first define the +total number of memory regions via the `region_num` parameter: + - **region_num** [mandatory] - defines the number of memory regions in the VM, specifically, the number of `vm_mem_region` entries in the `vm_platform`'s `regions` list. -The `vm_mem_region` structure contemplates the following parameters: +Then, each memory region is described by populating the `struct vm_mem_region`: .. code-block:: c From 1ca6f2e1c80531ea26c68b92669af78222a9e960 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 14:06:29 +0000 Subject: [PATCH 27/71] update(mem_reg): include important notes Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index b7d3df7..fed36a0 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -226,14 +226,33 @@ where: - **base** [mandatory] - corresponds to the base virtual address of the memory region; - **size** [mandatory] - corresponds to the size of the memory region; + +.. note:: + It is mandatory for `base` and `size` to align with the smallest page size of the architecture. + For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to 64 bytes. + - **place_phys** [optional] - the memory region is mapped into the virtual memory, and it's important to note that the virtual address (VA) might not necessarily be the same as the physical address (PA). When "place_phys" is set to true, the virtual address corresponds to the physical address. If ``place_phys`` equals to true, it allows to specify the physical address of the memory region. By default, ``place_phys`` equals to false; + +.. note:: + In MPU systems, place_phys and phys are ignored. + - **phys** [mandatory if ``place_phys`` is true] - it corresponds to the physical address where the memory region should be mapped; +.. note:: + For enhanced performance, especially in MMU-based targets, it's recommended to align `base` and + `size` to the architecture's huge pages (e.g., 2 MiB for Arm and RISC-V). Similarly, if + `place_phys` is enabled, aligning `phys` to the architecture's huge pages can also improve + performance. + +.. note:: + In MPU systems, place_phys and phys are ignored. + + 3. Inter-Process Communication (IPC) #################################### .. _Inter-Process Communication (IPC): From 1c373fe856ae5500898773a8e8a40f08393dfdf1 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 5 Jan 2024 14:07:48 +0000 Subject: [PATCH 28/71] update(mem_reg): include details on the use of place_phys Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index fed36a0..e6a42bb 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -252,6 +252,8 @@ where: .. note:: In MPU systems, place_phys and phys are ignored. +The usage of `place_phys` and `phys` allows users to manually allocate memory and obtain physical +mappings. This feature provides a means to define the physical memory region explicitly. 3. Inter-Process Communication (IPC) #################################### From 4b057cd11f289d87d6c70c3ef205118ce1610361 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:28:51 +0000 Subject: [PATCH 29/71] update(guest_img): update separately_loaded description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 24 ++++++++++++++++-------- source/bao_hyp/img/guest-image.svg | 4 ++++ 2 files changed, 20 insertions(+), 8 deletions(-) create mode 100644 source/bao_hyp/img/guest-image.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index e6a42bb..db80f0e 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -128,8 +128,8 @@ guest side. It encompasses the following options: where: - **base_addr** [mandatory] - corresponds to the ``image`` load address in the VM's address space; -- **load_addr** [mandatory] - corresponds to the ``image`` load address in the hypervisor address - space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name); +- **load_addr** [mandatory] - corresponds to the ``image`` load address in physical memory/physical + address space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name); - **size** [mandatory] - corresponds to the image size. For builtin images declared using `VM_IMAGE`, this value can be defined using the macro VM_IMAGE_SIZE(img_name); - **separately_loaded** [optional] - informs the hypervisor if the VM image is to be loaded @@ -137,12 +137,7 @@ where: - **inplace** [optional]- use the image inplace and don’t copy the image. By default, inplace is set as false; -.. figure:: img/vm-image.svg - :align: center - :name: vm-image-fig - -To ease the process of configuring the image running on each VM, the configuration of Bao allows -the use of two different macros: +To simplify the image configuration process for each VM, Bao allows the use of two macros: 1. **VM_IMAGE_BUILTIN** - This macro simplifies image configuration by requiring only the ``img_name`` and the image ``base_addr``. This macro specifies both the base address and image @@ -151,6 +146,19 @@ the use of two different macros: 2. **VM_IMAGE_LOADED** - This macro requires additional configurations. It requires the definition of image ``base_addr``, the image ``load_addr``, and the image ``size``. +If the ``separately_loaded`` parameter is configured as false, the hypervisor interprets this +setting as the offset of the built-in guest image within its own image, denoted as +``VM_IMAGE_OFFSET``. During run-time, the hypervisor adjusts this value to be subsequently +interpreted as a physical address. This adjustment involves adding the address at which the +hypervisor itself was loaded. However, if the ``separately_loaded`` parameter is configured as true, +the guest image is not embedded in the hypervisor image; instead, it is loaded independently. For +more details, refer to the figure below. + +.. figure:: img/guest-image.svg + :align: center + :name: vm-image-fig + + Virtual Machine Configuration ***************************** diff --git a/source/bao_hyp/img/guest-image.svg b/source/bao_hyp/img/guest-image.svg new file mode 100644 index 0000000..86e82c2 --- /dev/null +++ b/source/bao_hyp/img/guest-image.svg @@ -0,0 +1,4 @@ + + + +
Guest config
Guest config
struct vm_image image {
        .base_addr = img_base_addr,
.load_addr = VM_IMAGE_OFFSET(img_name),
.size = VM_IMAGE_SIZE(img_name), 
.separately_loaded = false,
};
struct vm_image image {...
struct vm_image image = VM_IMAGE_BUILTIN(img_name, img_base_addr)
struct vm_image image = VM_IMAGE_BUILTIN(img_name, img_base_addr)
0x0000
0x0000
0x4000
0x4000
Guest
img
Guest...
0x2000
0x2000
0x6000
0x6000
Bao img
Bao img
 VM_IMAGE_OFFSET 
 VM_IMAGE_OFFSET 
Guest config
Guest config
struct vm_image image {
        .base_addr = img_base_addr,
.load_addr = img_load_addr,
.size = img_size, 
.separately_loaded = true,
};
struct vm_image image {...
struct vm_image image = VM_IMAGE_LOADED(img_base_addr,       img_load_addr, img_size) 
struct vm_image image = VM_IMAGE_LOADED(img_base_addr,...
0x0000
0x0000
0x4000
0x4000
Guest
img
Guest...
0x2000
0x2000
0x6000
0x6000
Bao img
Bao imgText is not SVG - cannot display \ No newline at end of file From 3264cd3e8fb08e697f194a081afaa31d30e692de Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:30:18 +0000 Subject: [PATCH 30/71] fix(vm_config): fix format Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index db80f0e..aeac84f 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -96,14 +96,14 @@ CPU affinity, color mapping, and platform details. For each VM, the following pa specified: - **image** [mandatory] - a structure containing information about guest image loading (see details - in `Guest Image`_) -- **entry** [mandatory] - defines the entry point address in VM's address space; + in `Guest Image`_) +- **entry** [mandatory] - defines the entry point address in VM's address space; - **platform description** [mandatory] - a description of the VM platform, defining its resource - assignments and requirements (see details in `Virtual Machine Configuration`_); + assignments and requirements (see details in `Virtual Machine Configuration`_); - **cpu_affinity** [optional] - defines the affinity of the VM's vCPUs to the physical CPUs - assigned to the virtual platform (see details in `CPU Affinity`_); + assigned to the virtual platform (see details in `CPU Affinity`_); - **colors** [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see - details in `Coloring`_). + details in `Coloring`_). Guest Image *********** From af0601c91ccfb3189b2c209b152c5f9dbf825d4e Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:31:07 +0000 Subject: [PATCH 31/71] fix(vm_config): fix format Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index aeac84f..b554b44 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -184,7 +184,7 @@ vm_platform`, outlined below: By customizing this configuration, users can tailor the virtual platform to suit specific workload requirements and application needs for their virtual machines. The configuration encompasses the definition of: - + - **Number of CPUs** - see details in `Number of vCPUs`_; - **Memory regions** - see details in `Memory Regions`_;` - **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_; From ba5a87f740c6b693ae9e29ad52f74fc8f7b4b502 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:32:04 +0000 Subject: [PATCH 32/71] fix(guest_img): fix format Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index b554b44..6ff488c 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -150,9 +150,9 @@ If the ``separately_loaded`` parameter is configured as false, the hypervisor in setting as the offset of the built-in guest image within its own image, denoted as ``VM_IMAGE_OFFSET``. During run-time, the hypervisor adjusts this value to be subsequently interpreted as a physical address. This adjustment involves adding the address at which the -hypervisor itself was loaded. However, if the ``separately_loaded`` parameter is configured as true, -the guest image is not embedded in the hypervisor image; instead, it is loaded independently. For -more details, refer to the figure below. +hypervisor itself was loaded. However, if the ``separately_loaded`` parameter is configured as +true, the guest image is not embedded in the hypervisor image; instead, it is loaded independently. +For more details, refer to the figure below. .. figure:: img/guest-image.svg :align: center From 42c87cefcf5f152d954dd77127fe934ce8e1b7ce Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:33:50 +0000 Subject: [PATCH 33/71] fix(guest_img): remove outdated image Signed-off-by: Diogo Costa --- source/bao_hyp/img/vm-image.svg | 4 ---- 1 file changed, 4 deletions(-) delete mode 100644 source/bao_hyp/img/vm-image.svg diff --git a/source/bao_hyp/img/vm-image.svg b/source/bao_hyp/img/vm-image.svg deleted file mode 100644 index adccfbf..0000000 --- a/source/bao_hyp/img/vm-image.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -
VM 0
Address
Space
VM 0...
...
...
0x01
0x01
0x00
0x00
0x02
0x02
base_addr
base_addr
VM N
Address
Space
VM N...
...
...
0x01
0x01
0x00
0x00
0x02
0x02
0xFF
0xFF
0xFE
0xFE
Hypervisor
Address
Space
Hypervisor...
0x00
0x00
0xFF
0xFF
...
...
VM_IMAGE_OFFSET()
VM_IMAGE_OFFSE...
0xFF
0xFF
0xFE
0xFE
base_addr
base_addr
load_addr
load_addr
load_addr
load_addr
VMN
image
VMN...
VM0
image
VM0...Text is not SVG - cannot display \ No newline at end of file From 987878a6aa4e515a104be67c7cb9152d3c0fe7d5 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:41:35 +0000 Subject: [PATCH 34/71] update(guest_img): empashise the importance of using macros to configure Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 6ff488c..64d4f6b 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -137,7 +137,11 @@ where: - **inplace** [optional]- use the image inplace and don’t copy the image. By default, inplace is set as false; -To simplify the image configuration process for each VM, Bao allows the use of two macros: +To ensure accurate and efficient configuration of VM images, it is strongly recommended to leverage +the designated macros provided by Bao. These macros, namely ``VM_IMAGE_BUILTIN`` and +``VM_IMAGE_LOADED``, are specifically designed to simplify the image configuration process and +enhance compatibility with the hypervisor. + 1. **VM_IMAGE_BUILTIN** - This macro simplifies image configuration by requiring only the ``img_name`` and the image ``base_addr``. This macro specifies both the base address and image @@ -146,8 +150,12 @@ To simplify the image configuration process for each VM, Bao allows the use of t 2. **VM_IMAGE_LOADED** - This macro requires additional configurations. It requires the definition of image ``base_addr``, the image ``load_addr``, and the image ``size``. -If the ``separately_loaded`` parameter is configured as false, the hypervisor interprets this -setting as the offset of the built-in guest image within its own image, denoted as +Using these macros not only streamlines the configuration steps but also ensures adherence to the +correct syntax and parameters. Attempting to manually configure image details without utilizing +these macros may result in errors or unintended behavior. + +Moreover, if the ``separately_loaded`` parameter is configured as false, the hypervisor interprets +this setting as the offset of the built-in guest image within its own image, denoted as ``VM_IMAGE_OFFSET``. During run-time, the hypervisor adjusts this value to be subsequently interpreted as a physical address. This adjustment involves adding the address at which the hypervisor itself was loaded. However, if the ``separately_loaded`` parameter is configured as From dd49512744034522cd1abd99b0207ef2c5b862bf Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:45:29 +0000 Subject: [PATCH 35/71] update(guest_img): include macros use-cases examples Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 64d4f6b..2de82f1 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -154,6 +154,18 @@ Using these macros not only streamlines the configuration steps but also ensures correct syntax and parameters. Attempting to manually configure image details without utilizing these macros may result in errors or unintended behavior. +The integration of the appropriate macro tailored to your specific use case, encompassing essential +parameters such as img_name, base_addr, load_addr, and size. This practice not only fosters +consistency but also enhances reliability in your VM setup. For instance: + +- **IMAGE_BUILTIN**: Simplifies system configuration by leveraging Bao's default knowledge of the + image location. No separate configuration or loading of guest images through a bootloader is + required, and adjustments to the size of guest images are unnecessary. + +- **IMAGE_LOADED**: Highly recommended, especially for MPU systems, where manual allocation of space + for the guest image can be challenging if embedded in Bao's binary. Without utilizing LOADED, Bao + may need to copy the image, potentially resulting in space wastage + Moreover, if the ``separately_loaded`` parameter is configured as false, the hypervisor interprets this setting as the offset of the built-in guest image within its own image, denoted as ``VM_IMAGE_OFFSET``. During run-time, the hypervisor adjusts this value to be subsequently From e9e1c43697985f49f1c3c7b0400b42b10e280401 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:50:28 +0000 Subject: [PATCH 36/71] update(mem_manag): fix struct declaration Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 2de82f1..31a3723 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -378,7 +378,7 @@ where: } gic; // Configuration of the System Memory Management Unit (SMMU) - struct { + struct smmu_config{ streamid_t global_mask; size_t group_num; struct smmu_group { From 431be7fe3361d849639a1aa0ccacdae07fc7eb24 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 16:53:43 +0000 Subject: [PATCH 37/71] update(arch): fix typo Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 31a3723..2ff0f56 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -393,8 +393,8 @@ where: - **vgic_dscrp** [mandatory] - corresponds to the configuration of the Generic Interrupt Controller (GIC); -- **vgic_dscrp** [optional] - corresponds to the configuration of the System Memory Management Unit - (SMMU); +- **smmu_config** [optional] - corresponds to the configuration of the System Memory Management + Unit (SMMU); CPU Affinity From e501eb4f6d92ed2ecac512ce6b2356c3b49caccf Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 17 Jan 2024 17:25:01 +0000 Subject: [PATCH 38/71] update(cache_col): update image Signed-off-by: Diogo Costa --- source/bao_hyp/img/llc-colors.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/img/llc-colors.svg b/source/bao_hyp/img/llc-colors.svg index dd9b14f..5b5eff0 100644 --- a/source/bao_hyp/img/llc-colors.svg +++ b/source/bao_hyp/img/llc-colors.svg @@ -1,4 +1,4 @@ -









Hypervisor
Hypervisor
VM 0

VM 0...
VM 1

VM 1
LLC
LLC
CPU0
CPU0
CPU1
CPU1
CPU2
CPU2
CPU3
CPU3

colors: 0xF0F0F0
colors: 0xF0F0F0
colors: 0x0F0F0F
colors: 0x0F0F0FText is not SVG - cannot display \ No newline at end of file +
Hypervisor
Hypervisor
VM 1

VM 1...
VM 2

VM 2
CPU1
CPU1
CPU2
CPU2
CPU3
CPU3
CPU4
CPU4
colors: 0xF0F0F0
colors: 0xF0F0F0
colors: 0xF0F0F0
colors: 0xF0F0F0
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache










Last-Level Cache
Last-Level Cache
Main Memory
Main Memory







































































































































Text is not SVG - cannot display \ No newline at end of file From 2bbd78f5573c827b7df785a6c983c26983be633c Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 19 Jan 2024 09:46:02 +0000 Subject: [PATCH 39/71] update(cache_col): improve coloring description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 2ff0f56..ac479f2 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -417,6 +417,12 @@ This functionality is achieved through the following configuration parameter: Coloring ******** +Cache coloring is a technique used to partition shared Last Level Cache (LLC) sets among different +guests (i.e., VMs). The main goal is to minimize cache conflicts and enhance overall system +performance by carefully assigning specific colors, representing cache sets, to different entities. +Bao supports cache coloring which can be configured using the``colors`` field within the +``vm_config`` struct. + - **colors** [optional] - corresponds to a bitmap for the assigned cache colors of the VM. This value is truncated depending on the number of available colors calculated at run-time, i.e., its platform-dependent. By default, the coloring mechanism is not active. For instance, the following @@ -426,6 +432,11 @@ Coloring :align: center :name: llc-colors-fig +It's important to note that cache coloring relies on the careful assignment of colors to each VM. +However, this mechanism may not take effect if the physical mapping feature is enabled for a +specific memory region. Cache coloring exclusively operates in virtual memory systems, i.e., +systems featuring Memory Management Units (MMUs) for address translation. + Shared Memory Configuration --------------------------- From 404de5a52494eed38516059edb32e4e7031331a3 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 19 Jan 2024 09:58:35 +0000 Subject: [PATCH 40/71] update(ipc): add note regarding to shmem size and base Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index ac479f2..7088ef5 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -307,6 +307,14 @@ where: - **base** [mandatory] - corresponds to the virtual base address of the IPC memory region; - **size** [mandatory] - corresponds to the size of the IPC memory region; + +.. note:: + The ``size`` field must be less than or equal to the size of the shared memory. Additionally, for + MPU systems, the ``base`` field is ignored, as the region address is the same as the shared + memory object address. Also, it is mandatory for both ``base`` and ``size`` to be aligned with + the architecture's smallest page size. For MMU systems, this corresponds to 4K in all + architectures, while for MPU systems, the alignment corresponds to 64 bytes. + - **shmem_id** [mandatory] - corresponds to the ID of the shared memory associated with the IPC; - **interrupt_num** [mandatory] - defines the number of interrupts assigned to the IPC; - **interrupts** [mandatory if *interrupt_num* > 0] - defines a list of interrupt IDs assigned to From 854fe14d5edc08d3e958ff0fa3f458baa11507d1 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 19 Jan 2024 11:22:22 +0000 Subject: [PATCH 41/71] update(ipc): add overview of IPC workflow Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 13 +++++++++++++ source/bao_hyp/img/ipc.svg | 4 ++++ 2 files changed, 17 insertions(+) create mode 100644 source/bao_hyp/img/ipc.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 7088ef5..f1d6622 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -287,6 +287,19 @@ mappings. This feature provides a means to define the physical memory region exp #################################### .. _Inter-Process Communication (IPC): +Inter-Process Communication (IPC) enables seamless communication between distinct processes in a +computing system. This mechanism facilitates data exchange, synchronization, and coordination, +fostering enhanced functionality and system efficiency. + +Bao provides support for IPC, allowing VMs to establish communication channels. The IPC +configuration involves defining the number of IPCs using the ``ipc_num`` field within the +``vm_config`` struct. The specifics of each IPC are then outlined through the ``ipcs`` structure, +including fields such as ``base``, ``size``, ``shmem_id``, ``interrupt_num``, and ``interrupts``. + +.. figure:: img/ipc.svg + :align: center + :name: ipc-fig + - **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num`` equals to zero; - **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification of the IPC and is diff --git a/source/bao_hyp/img/ipc.svg b/source/bao_hyp/img/ipc.svg new file mode 100644 index 0000000..2528fb8 --- /dev/null +++ b/source/bao_hyp/img/ipc.svg @@ -0,0 +1,4 @@ + + + +
Hypervisor
Hypervisor
VM 1
VM 1
VM 2
VM 2
Main Memory
Main Memory







































































































































VM1 Memory
VM1 Memory
VM2 Memory
VM2 Memory
Shared Memory
Shared Memory
Write
Write
1
1
IRQ
IRQ
2
2
Read
Read
3
3Text is not SVG - cannot display \ No newline at end of file From ae5a5bdb1904aeea420d94356fc72ceb0cc801f3 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Fri, 19 Jan 2024 11:32:39 +0000 Subject: [PATCH 42/71] update(ipc): include note regarding to base and size Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index f1d6622..50028ff 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -291,15 +291,15 @@ Inter-Process Communication (IPC) enables seamless communication between distinc computing system. This mechanism facilitates data exchange, synchronization, and coordination, fostering enhanced functionality and system efficiency. +.. figure:: img/ipc.svg + :align: center + :name: ipc-fig + Bao provides support for IPC, allowing VMs to establish communication channels. The IPC configuration involves defining the number of IPCs using the ``ipc_num`` field within the ``vm_config`` struct. The specifics of each IPC are then outlined through the ``ipcs`` structure, including fields such as ``base``, ``size``, ``shmem_id``, ``interrupt_num``, and ``interrupts``. -.. figure:: img/ipc.svg - :align: center - :name: ipc-fig - - **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num`` equals to zero; - **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification of the IPC and is @@ -361,6 +361,11 @@ where: - **pa** [mandatory] - corresponds to the base physical address of the device; - **va** [mandatory] - corresponds to the base virtual address of the device; - **size** [mandatory] - corresponds to the size of the device memory region; + +.. note:: + It is mandatory for `base` and `size` to align with the smallest page size of the architecture. + For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to 64 bytes. + - **interrupt_num** [optional] - corresponds to the number of interrupts generated by the device to the VM. By default, ``interrupt_num`` equals to 0; - **interrupts** [mandatory if *interrupt_num*>0] - defines a list of interrupt IDs generated by From c77efcb018437041330f894e724c94c8d63404ee Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 12 Mar 2024 11:53:27 +0000 Subject: [PATCH 43/71] fix(vm_config): improve readability Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 50028ff..47c5204 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -69,7 +69,7 @@ VM Configuration -------------------- Bao's configuration file allows you to partition the platforms' hardware resources, such as CPU -cores, memory, or devices, by assigning them one or more VMs. It also allows you to configure the +cores, memory, or devices, by assigning them to one or more VMs. It also allows you to configure the guest image to run on that VM. In Bao, resources are exclusively assigned to each VM, including communication channels between two or more VMs, which may utilize shared memory or dedicated links. @@ -78,8 +78,8 @@ communication channels between two or more VMs, which may utilize shared memory :name: guest-config-fig -Each entry in the ``vmlist`` mentioned earlier is a ``vm_config`` struct, defining the -configuration for different guests by populating the following struct called *vmconfig*: +Each entry in the ``vmlist`` mentioned earlier is a ``vm_config`` struct, which defines the +configuration of each individual guest: .. code-block:: c @@ -95,14 +95,14 @@ Each entry in this list represents a unique VM configuration, defining its image CPU affinity, color mapping, and platform details. For each VM, the following parameters must be specified: -- **image** [mandatory] - a structure containing information about guest image loading (see details +- `image` [mandatory] - a structure containing information about guest image loading (see details in `Guest Image`_) -- **entry** [mandatory] - defines the entry point address in VM's address space; -- **platform description** [mandatory] - a description of the VM platform, defining its resource +- `entry` [mandatory] - defines the entry point address in VM's address space; +- `platform description` [mandatory] - a description of the VM platform, defining its resource assignments and requirements (see details in `Virtual Machine Configuration`_); -- **cpu_affinity** [optional] - defines the affinity of the VM's vCPUs to the physical CPUs +- `cpu_affinity` [optional] - defines the affinity of the VM's vCPUs to the physical CPUs assigned to the virtual platform (see details in `CPU Affinity`_); -- **colors** [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see +- `colors` [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see details in `Coloring`_). Guest Image From 7ea8acb3dbb996b7b73d288505f97f0d96aca1ca Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 12 Mar 2024 11:59:03 +0000 Subject: [PATCH 44/71] fix(guest_image): improve readability Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 47c5204..70915bd 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -154,9 +154,9 @@ Using these macros not only streamlines the configuration steps but also ensures correct syntax and parameters. Attempting to manually configure image details without utilizing these macros may result in errors or unintended behavior. -The integration of the appropriate macro tailored to your specific use case, encompassing essential -parameters such as img_name, base_addr, load_addr, and size. This practice not only fosters -consistency but also enhances reliability in your VM setup. For instance: +The integration of the appropriate macro, tailored to your specific use case, is crucial for +ensuring consistency and reliability in your VM setup. This macro should encompass essential +parameters such as img_name, base_addr, load_addr, and size. For instance: - **IMAGE_BUILTIN**: Simplifies system configuration by leveraging Bao's default knowledge of the image location. No separate configuration or loading of guest images through a bootloader is @@ -176,6 +176,7 @@ For more details, refer to the figure below. .. figure:: img/guest-image.svg :align: center + :width: 100% :name: vm-image-fig From a89962699dcd7ea12c5d85d03db631d1f06e9306 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 12 Mar 2024 12:53:54 +0000 Subject: [PATCH 45/71] fix(vm_config): improve readability Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 142 +++++++++++++++++++------------------- 1 file changed, 71 insertions(+), 71 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 70915bd..d59cb26 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -95,14 +95,14 @@ Each entry in this list represents a unique VM configuration, defining its image CPU affinity, color mapping, and platform details. For each VM, the following parameters must be specified: -- `image` [mandatory] - a structure containing information about guest image loading (see details +- ``image`` [mandatory] - a structure containing information about guest image loading (see details in `Guest Image`_) -- `entry` [mandatory] - defines the entry point address in VM's address space; -- `platform description` [mandatory] - a description of the VM platform, defining its resource +- ``entry`` [mandatory] - defines the entry point address in VM's address space; +- ``platform description`` [mandatory] - a description of the VM platform, defining its resource assignments and requirements (see details in `Virtual Machine Configuration`_); -- `cpu_affinity` [optional] - defines the affinity of the VM's vCPUs to the physical CPUs +- ``cpu_affinity`` [optional] - defines the affinity of the VM's vCPUs to the physical CPUs assigned to the virtual platform (see details in `CPU Affinity`_); -- `colors` [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see +- ``colors`` [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see details in `Coloring`_). Guest Image @@ -127,14 +127,14 @@ guest side. It encompasses the following options: where: -- **base_addr** [mandatory] - corresponds to the ``image`` load address in the VM's address space; -- **load_addr** [mandatory] - corresponds to the ``image`` load address in physical memory/physical - address space. This value can be defined using the macro VM_IMAGE_OFFSET(img_name); -- **size** [mandatory] - corresponds to the image size. For builtin images declared using - `VM_IMAGE`, this value can be defined using the macro VM_IMAGE_SIZE(img_name); -- **separately_loaded** [optional] - informs the hypervisor if the VM image is to be loaded +- ``base_addr`` [mandatory] - corresponds to the ``image`` load address in the VM's address space; +- ``load_addr`` [mandatory] - corresponds to the ``image`` load address in physical memory/physical + address space. This value can be defined using the macro ``VM_IMAGE_OFFSET(img_name)``; +- ``size`` [mandatory] - corresponds to the image size. For builtin images declared using + `VM_IMAGE`, this value can be defined using the macro ``VM_IMAGE_SIZE(img_name)``; +- ``separately_loaded`` [optional] - informs the hypervisor if the VM image is to be loaded separately by a bootloader; By default, separately_loaded is set as false; -- **inplace** [optional]- use the image inplace and don’t copy the image. By default, inplace is +- ``inplace`` [optional]- use the image inplace and don’t copy the image. By default, inplace is set as false; To ensure accurate and efficient configuration of VM images, it is strongly recommended to leverage @@ -143,11 +143,11 @@ the designated macros provided by Bao. These macros, namely ``VM_IMAGE_BUILTIN`` enhance compatibility with the hypervisor. -1. **VM_IMAGE_BUILTIN** - This macro simplifies image configuration by requiring only the +1. ``VM_IMAGE_BUILTIN`` - This macro simplifies image configuration by requiring only the ``img_name`` and the image ``base_addr``. This macro specifies both the base address and image size. -2. **VM_IMAGE_LOADED** - This macro requires additional configurations. It requires the definition +2. ``VM_IMAGE_LOADED`` - This macro requires additional configurations. It requires the definition of image ``base_addr``, the image ``load_addr``, and the image ``size``. Using these macros not only streamlines the configuration steps but also ensures adherence to the @@ -158,11 +158,11 @@ The integration of the appropriate macro, tailored to your specific use case, is ensuring consistency and reliability in your VM setup. This macro should encompass essential parameters such as img_name, base_addr, load_addr, and size. For instance: -- **IMAGE_BUILTIN**: Simplifies system configuration by leveraging Bao's default knowledge of the +- ``IMAGE_BUILTIN``: Simplifies system configuration by leveraging Bao's default knowledge of the image location. No separate configuration or loading of guest images through a bootloader is required, and adjustments to the size of guest images are unnecessary. -- **IMAGE_LOADED**: Highly recommended, especially for MPU systems, where manual allocation of space +- ``IMAGE_LOADED``: Highly recommended, especially for MPU systems, where manual allocation of space for the guest image can be challenging if embedded in Bao's binary. Without utilizing LOADED, Bao may need to copy the image, potentially resulting in space wastage @@ -184,9 +184,8 @@ Virtual Machine Configuration ***************************** The VM configuration enables users to define the characteristics of each virtualized platform. It -encompasses critical details that define the VM's run-time environment, performance, and overall -capabilities. The virtual machine configuration is performed by populating the structure `struct -vm_platform`, outlined below: +capabilities. The virtual machine configuration is performed by populating the structure ``struct +vm_platform``, outlined below: .. code-block:: c @@ -203,25 +202,25 @@ vm_platform`, outlined below: } By customizing this configuration, users can tailor the virtual platform to suit specific workload -requirements and application needs for their virtual machines. The configuration encompasses the +requirements and application needs for their virtual machines. The configuration includes the definition of: -- **Number of CPUs** - see details in `Number of vCPUs`_; -- **Memory regions** - see details in `Memory Regions`_;` -- **Inter-Process Comunication (IPC)** - see details in `Inter-Process Communication (IPC)`_; -- **Devices** - see details in `Devices`_; -- **Memory Management** - see details in `Memory Management`_; -- **Architectural-Specific Configurations** - see details in `Architectural-Specific +- ``Number of CPUs`` - see details in `Number of vCPUs`_; +- ``Memory regions`` - see details in `Memory Regions`_;` +- ``Inter-Process Comunication (IPC)`` - see details in `Inter-Process Communication (IPC)`_; +- ``Devices`` - see details in `Devices`_; +- ``Memory Management`` - see details in `Memory Management`_; +- ``Architectural-Specific Configurations`` - see details in `Architectural-Specific Configurations`_; 1. Number of vCPUs ################## .. _Number of vCPUs: -- **cpu_num** [mandatory] - defines the number of CPUs assigned to the VM; +- ``cpu_num`` [mandatory] - defines the number of CPUs assigned to the VM; .. warning:: - Ensure that the cumulative count of CPUs allocated across all VMs listed in the `vmlist` does not + Ensure that the cumulative count of CPUs allocated across all VMs listed in the ``vmlist`` does not exceed the total number of available CPUs on the platform. Failing to adhere to this requirement might result in the guest failing to boot without any warning. @@ -235,12 +234,12 @@ physical memory for these regions. However, there is an option to define the phy where the specific region will be mapped. For each VM, users can define multiple memory regions. To facilitate this, users first define the -total number of memory regions via the `region_num` parameter: +total number of memory regions via the ``region_num`` parameter: -- **region_num** [mandatory] - defines the number of memory regions in the VM, specifically, the - number of `vm_mem_region` entries in the `vm_platform`'s `regions` list. +- ``region_num`` [mandatory] - defines the number of memory regions in the VM, specifically, the + number of ``vm_mem_region`` entries in the ``vm_platform``'s ``regions`` list. -Then, each memory region is described by populating the `struct vm_mem_region`: +Then, each memory region is described by populating the ``struct vm_mem_region``: .. code-block:: c @@ -253,35 +252,30 @@ Then, each memory region is described by populating the `struct vm_mem_region`: where: -- **base** [mandatory] - corresponds to the base virtual address of the memory region; -- **size** [mandatory] - corresponds to the size of the memory region; +- ``base`` [mandatory] - corresponds to the base virtual address of the memory region; +- ``size`` [mandatory] - corresponds to the size of the memory region; .. note:: - It is mandatory for `base` and `size` to align with the smallest page size of the architecture. + It is mandatory for ``base`` and ``size`` to align with the smallest page size of the architecture. For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to 64 bytes. -- **place_phys** [optional] - the memory region is mapped into the virtual memory, and it's +- ``place_phys`` [optional] - the memory region is mapped into the virtual memory, and it's important to note that the virtual address (VA) might not necessarily be the same as the physical address (PA). When "place_phys" is set to true, the virtual address corresponds to the physical address. If ``place_phys`` equals to true, it allows to specify the physical address of the memory region. By default, ``place_phys`` equals to false; -.. note:: - In MPU systems, place_phys and phys are ignored. - -- **phys** [mandatory if ``place_phys`` is true] - it corresponds to the physical address where the +- ``phys`` [mandatory if ``place_phys`` is true] - it corresponds to the physical address where the memory region should be mapped; .. note:: - For enhanced performance, especially in MMU-based targets, it's recommended to align `base` and - `size` to the architecture's huge pages (e.g., 2 MiB for Arm and RISC-V). Similarly, if - `place_phys` is enabled, aligning `phys` to the architecture's huge pages can also improve - performance. + - For enhanced performance, especially in MMU-based targets, it's recommended to align ``base`` + and ``size`` to the architecture's huge pages (e.g., 2MiB for Arm and RISC-V). Similarly, if + ``place_phys`` is enabled, aligning ``phys`` to the architecture's huge pages can also + improve performance. + - In MPU systems, place_phys and phys are ignored. -.. note:: - In MPU systems, place_phys and phys are ignored. - -The usage of `place_phys` and `phys` allows users to manually allocate memory and obtain physical +The usage of ``place_phys`` and ``phys`` allows users to manually allocate memory and obtain physical mappings. This feature provides a means to define the physical memory region explicitly. 3. Inter-Process Communication (IPC) @@ -294,16 +288,18 @@ fostering enhanced functionality and system efficiency. .. figure:: img/ipc.svg :align: center + :width: 100% :name: ipc-fig + Bao provides support for IPC, allowing VMs to establish communication channels. The IPC configuration involves defining the number of IPCs using the ``ipc_num`` field within the ``vm_config`` struct. The specifics of each IPC are then outlined through the ``ipcs`` structure, including fields such as ``base``, ``size``, ``shmem_id``, ``interrupt_num``, and ``interrupts``. -- **ipc_num** [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num`` +- ``ipc_num`` [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num`` equals to zero; -- **ipcs** [mandatory if ``ipc_num`` > 0] - corresponds to the specification of the IPC and is +- ``ipcs`` [mandatory if ``ipc_num`` > 0] - corresponds to the specification of the IPC and is configured through the following structure: .. code-block:: c @@ -319,8 +315,8 @@ including fields such as ``base``, ``size``, ``shmem_id``, ``interrupt_num``, an where: -- **base** [mandatory] - corresponds to the virtual base address of the IPC memory region; -- **size** [mandatory] - corresponds to the size of the IPC memory region; +- ``base`` [mandatory] - corresponds to the virtual base address of the IPC memory region; +- ``size`` [mandatory] - corresponds to the size of the IPC memory region; .. note:: The ``size`` field must be less than or equal to the size of the shared memory. Additionally, for @@ -329,9 +325,9 @@ where: the architecture's smallest page size. For MMU systems, this corresponds to 4K in all architectures, while for MPU systems, the alignment corresponds to 64 bytes. -- **shmem_id** [mandatory] - corresponds to the ID of the shared memory associated with the IPC; -- **interrupt_num** [mandatory] - defines the number of interrupts assigned to the IPC; -- **interrupts** [mandatory if *interrupt_num* > 0] - defines a list of interrupt IDs assigned to +- ``shmem_id`` [mandatory] - corresponds to the ID of the shared memory associated with the IPC; +- ``interrupt_num`` [mandatory] - defines the number of interrupts assigned to the IPC; +- ``interrupts`` [mandatory if *interrupt_num* > 0] - defines a list of interrupt IDs assigned to the IPC - ``(irqid_t[]) {irq_1, ..., irq_n}``; .. warning:: @@ -342,8 +338,8 @@ where: ########## .. _Devices: -- **dev_num** [mandatory] - corresponds to the number of devices assigned to the VM; -- **devs** [mandatory if *dev_num* > 0] - corresponds to the specification of the VM's devices and +- ``dev_num`` [mandatory] - corresponds to the number of devices assigned to the VM; +- ``devs`` [mandatory if *dev_num* > 0] - corresponds to the specification of the VM's devices and is configured through the following structure: .. code-block:: c @@ -359,19 +355,19 @@ where: where: -- **pa** [mandatory] - corresponds to the base physical address of the device; -- **va** [mandatory] - corresponds to the base virtual address of the device; -- **size** [mandatory] - corresponds to the size of the device memory region; +- ``pa`` [mandatory] - corresponds to the base physical address of the device; +- ``va`` [mandatory] - corresponds to the base virtual address of the device; +- ``size`` [mandatory] - corresponds to the size of the device memory region; .. note:: - It is mandatory for `base` and `size` to align with the smallest page size of the architecture. + It is mandatory for ``base`` and ``size`` to align with the smallest page size of the architecture. For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to 64 bytes. -- **interrupt_num** [optional] - corresponds to the number of interrupts generated by the device to +- ``interrupt_num`` [optional] - corresponds to the number of interrupts generated by the device to the VM. By default, ``interrupt_num`` equals to 0; -- **interrupts** [mandatory if *interrupt_num*>0] - defines a list of interrupt IDs generated by +- ``interrupts`` [mandatory if *interrupt_num*>0] - defines a list of interrupt IDs generated by the device - ``(irqid_t[]) {irq_1, ..., irq_n};`` -- **id** [optional] - corresponds to the bus master id for iommu effects: +- ``id`` [optional] - corresponds to the bus master id for iommu effects: .. warning:: Specifying a number of interrupts in the ``interrupts`` buffer that differs from the @@ -381,7 +377,7 @@ where: #################### .. _Memory Management: - **mmu** [optional] - In MPU-based platforms which might also support virtual memory (i.e. aarch64 + ``mmu`` [optional] - In MPU-based platforms which might also support virtual memory (i.e. aarch64 cortex-r) the hypervisor sets up the VM using an MPU by default. If the user wants this VM to use the MMU they must set the config ``mmu`` parameter to true; @@ -389,7 +385,7 @@ where: ######################################## .. _Architectural-Specific Configurations: -- **arch** [mandatory] - allows the definition of architecture dependent configurations and is +- ``arch`` [mandatory] - allows the definition of architecture dependent configurations and is configured through the following structure: .. code-block:: c @@ -418,9 +414,9 @@ where: where: -- **vgic_dscrp** [mandatory] - corresponds to the configuration of the Generic Interrupt Controller +- ``vgic_dscrp`` [mandatory] - corresponds to the configuration of the Generic Interrupt Controller (GIC); -- **smmu_config** [optional] - corresponds to the configuration of the System Memory Management +- ``smmu_config`` [optional] - corresponds to the configuration of the System Memory Management Unit (SMMU); @@ -432,11 +428,13 @@ involves selecting the physical core where the guest should run. .. figure:: img/cpu-affinity.svg :align: center + :width: 100% :name: cpu-affinity-fig + This functionality is achieved through the following configuration parameter: -- **cpu_affinity** [optional] - corresponds to a bitmap signaling the preferred physical CPUs +- ``cpu_affinity`` [optional] - corresponds to a bitmap signaling the preferred physical CPUs assigned to the VM. If this value is mutually exclusive for all the VMs, the physical CPUs assigned to each VM follow the bitmap. Otherwise (in case of bit overlap or lack of affinity definition), the CPU assignment is defined by the hypervisor; @@ -447,18 +445,20 @@ Coloring Cache coloring is a technique used to partition shared Last Level Cache (LLC) sets among different guests (i.e., VMs). The main goal is to minimize cache conflicts and enhance overall system performance by carefully assigning specific colors, representing cache sets, to different entities. -Bao supports cache coloring which can be configured using the``colors`` field within the +Bao supports cache coloring which can be configured using the ``colors`` field within the ``vm_config`` struct. -- **colors** [optional] - corresponds to a bitmap for the assigned cache colors of the VM. This +- ``colors`` [optional] - corresponds to a bitmap for the assigned cache colors of the VM. This value is truncated depending on the number of available colors calculated at run-time, i.e., its platform-dependent. By default, the coloring mechanism is not active. For instance, the following picture depicts a hypothetical setup with a 50/50 coloring scheme; .. figure:: img/llc-colors.svg :align: center + :width: 100% :name: llc-colors-fig + It's important to note that cache coloring relies on the careful assignment of colors to each VM. However, this mechanism may not take effect if the physical mapping feature is enabled for a specific memory region. Cache coloring exclusively operates in virtual memory systems, i.e., From e671c5b5c256bfde087aaefd017f54e8d6ffbde4 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 12 Mar 2024 14:13:25 +0000 Subject: [PATCH 46/71] ref(arch): remove mem. management section and split arch specific config Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 127 +++++++++++++++++++++++++++----------- 1 file changed, 92 insertions(+), 35 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index d59cb26..2d83d95 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -209,7 +209,6 @@ definition of: - ``Memory regions`` - see details in `Memory Regions`_;` - ``Inter-Process Comunication (IPC)`` - see details in `Inter-Process Communication (IPC)`_; - ``Devices`` - see details in `Devices`_; -- ``Memory Management`` - see details in `Memory Management`_; - ``Architectural-Specific Configurations`` - see details in `Architectural-Specific Configurations`_; @@ -373,52 +372,110 @@ where: Specifying a number of interrupts in the ``interrupts`` buffer that differs from the ``interrupt_num`` may result in unforeseen behavior. -5. Memory Management -#################### -.. _Memory Management: - ``mmu`` [optional] - In MPU-based platforms which might also support virtual memory (i.e. aarch64 - cortex-r) the hypervisor sets up the VM using an MPU by default. If the user wants this VM to use - the MMU they must set the config ``mmu`` parameter to true; - -6. Architectural-Specific Configurations +5. Architectural-Specific Configurations ######################################## .. _Architectural-Specific Configurations: - ``arch`` [mandatory] - allows the definition of architecture dependent configurations and is configured through the following structure: -.. code-block:: c +.. tabs:: + .. tab:: Arm - struct arch_vm_platform { - - // Configuration of the Generic Interrupt Controller (GIC) - struct vgic_dscrp { - paddr_t gicd_addr; - paddr_t gicc_addr; - paddr_t gicr_addr; - size_t interrupt_num; - } gic; - - // Configuration of the System Memory Management Unit (SMMU) - struct smmu_config{ - streamid_t global_mask; - size_t group_num; - struct smmu_group { - streamid_t mask; - streamid_t id; - } *groups; - } smmu; - }; + For the Arm architecure: + .. code-block:: c -where: + struct arch_platform { + struct gic_dscrp { + paddr_t gicc_addr; + paddr_t gich_addr; + paddr_t gicv_addr; + paddr_t gicd_addr; + paddr_t gicr_addr; + + irqid_t maintenance_id; + } gic; + + struct smmu_dscrp { + paddr_t base; + streamid_t global_mask; + } smmu; + + struct clusters { + size_t num; + size_t* core_num; + } clusters; + }; + + Where, for the GIC interrupt controller ``struct gic_dscrp`` description: + + - ``gic.gicc_addr`` [mandatory for GICv2 platforms] - base address for the GIC's CPU Interface; + - ``gic.gich_addr`` [mandatory for GICv2 platforms] - base address for the GIC's Virtual + Interface Control Registers; + - ``gic.gicv_addr`` [mandatory for GICv2 platforms] - base address for the GIC's Virtual CPU + Interface; + - ``gic.gicd_addr`` [mandatory] - base address for the GIC's Distributor; + - ``gic.gicr_addr`` [mandatory for GICv3/4 platforms] - base address for the GIC's + Redistributor; + - ``gic.maintenance_id`` [mandatory] - The interrupt ID for the GIC's maintenance interrupt; + + For the SMMU `struct smmu_dscrp`: + + - ``smmu.base`` [mandatory] - is the base address for the SMMU; + - ``smmu.global_mask`` [optional; only valid for SMMUv2] - a mask to be applied to all SMMUv2's + Stream Match Registers; + + + Finally, when CPUs are organized in clusters, in the Arm architecture their IDs are assigned + using an hierarchical schema. To be able to calculate the linearized ID for each core, we + require the port to provide the number of CPUs of cluster in ascending order of AFF1. + + .. tab:: RISC-V + + For the RISC-V architecture: + + .. code-block:: c + + struct arch_platform { + union irqc_dscrp { + + struct { + paddr_t base; + } plic; + + struct { + struct { + paddr_t base; + } aplic; + } aia; + + } irqc; + + struct { + paddr_t base; // Base address of the IOMMU mmapped IF + irqid_t fq_irq_id; + } iommu; + + struct { + paddr_t base; + } aclint_sswi; + }; + + In case the available interrupt controller is the legacy PLIC: + + - ``irqc.plic.base`` [mandatory if PLIC is available] - is the base address for the PLIC; + + In case the available interrupt controller is an AIA contaning an APLIC: + + - ``irqc.aia.aplic.base`` [mandatory if APLIC is available] - is the base address for the APLIC; -- ``vgic_dscrp`` [mandatory] - corresponds to the configuration of the Generic Interrupt Controller - (GIC); -- ``smmu_config`` [optional] - corresponds to the configuration of the System Memory Management - Unit (SMMU); + When an IOMMU is available: + - ``iommu.base`` [mandatory if IOMMU is available] - is the base address for the IOMMU; + - ``iommu.fq_irq_id`` [mandatory if IOMMU is available] - the Fault Queue interrupt ID (the + current implementatio assumes this is a wired interrupt); CPU Affinity ************ From 3594c1332ec47b112507320600b30c835c41b6ac Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 12 Mar 2024 14:20:27 +0000 Subject: [PATCH 47/71] fix(format): include format tweaks Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 24 ++++++++++++------------ source/spelling_wordlist.txt | 3 +++ 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 2d83d95..e511199 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -69,8 +69,8 @@ VM Configuration -------------------- Bao's configuration file allows you to partition the platforms' hardware resources, such as CPU -cores, memory, or devices, by assigning them to one or more VMs. It also allows you to configure the -guest image to run on that VM. In Bao, resources are exclusively assigned to each VM, including +cores, memory, or devices, by assigning them to one or more VMs. It also allows you to configure +the guest image to run on that VM. In Bao, resources are exclusively assigned to each VM, including communication channels between two or more VMs, which may utilize shared memory or dedicated links. .. figure:: img/guest-config.svg @@ -156,15 +156,15 @@ these macros may result in errors or unintended behavior. The integration of the appropriate macro, tailored to your specific use case, is crucial for ensuring consistency and reliability in your VM setup. This macro should encompass essential -parameters such as img_name, base_addr, load_addr, and size. For instance: +parameters such as ``img_name``, ``base_addr``, ``load_addr``, and size. For instance: - ``IMAGE_BUILTIN``: Simplifies system configuration by leveraging Bao's default knowledge of the image location. No separate configuration or loading of guest images through a bootloader is required, and adjustments to the size of guest images are unnecessary. -- ``IMAGE_LOADED``: Highly recommended, especially for MPU systems, where manual allocation of space - for the guest image can be challenging if embedded in Bao's binary. Without utilizing LOADED, Bao - may need to copy the image, potentially resulting in space wastage +- ``IMAGE_LOADED``: Highly recommended, especially for MPU systems, where manual allocation of + space for the guest image can be challenging if embedded in Bao's binary. Without utilizing + LOADED, Bao may need to copy the image, potentially resulting in space wastage. Moreover, if the ``separately_loaded`` parameter is configured as false, the hypervisor interprets this setting as the offset of the built-in guest image within its own image, denoted as @@ -274,8 +274,8 @@ where: improve performance. - In MPU systems, place_phys and phys are ignored. -The usage of ``place_phys`` and ``phys`` allows users to manually allocate memory and obtain physical -mappings. This feature provides a means to define the physical memory region explicitly. +The usage of ``place_phys`` and ``phys`` allows users to manually allocate memory and obtain +physical mappings. This feature provides a means to define the physical memory region explicitly. 3. Inter-Process Communication (IPC) #################################### @@ -290,7 +290,7 @@ fostering enhanced functionality and system efficiency. :width: 100% :name: ipc-fig - + Bao provides support for IPC, allowing VMs to establish communication channels. The IPC configuration involves defining the number of IPCs using the ``ipc_num`` field within the ``vm_config`` struct. The specifics of each IPC are then outlined through the ``ipcs`` structure, @@ -383,7 +383,7 @@ where: .. tabs:: .. tab:: Arm - For the Arm architecure: + For the Arm architecture: .. code-block:: c @@ -467,7 +467,7 @@ where: - ``irqc.plic.base`` [mandatory if PLIC is available] - is the base address for the PLIC; - In case the available interrupt controller is an AIA contaning an APLIC: + In case the available interrupt controller is an AIA containing an APLIC: - ``irqc.aia.aplic.base`` [mandatory if APLIC is available] - is the base address for the APLIC; @@ -475,7 +475,7 @@ where: - ``iommu.base`` [mandatory if IOMMU is available] - is the base address for the IOMMU; - ``iommu.fq_irq_id`` [mandatory if IOMMU is available] - the Fault Queue interrupt ID (the - current implementatio assumes this is a wired interrupt); + current implementation assumes this is a wired interrupt); CPU Affinity ************ diff --git a/source/spelling_wordlist.txt b/source/spelling_wordlist.txt index 4e6d3ec..cc81406 100644 --- a/source/spelling_wordlist.txt +++ b/source/spelling_wordlist.txt @@ -75,6 +75,9 @@ bootloader inplace virtualized iommu +MiB +MMU MPU aarch64 aarch +linearized From df438311391a84f05683af57c6a402a016537804 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 12 Mar 2024 14:23:26 +0000 Subject: [PATCH 48/71] fix(cpu_affinity): include improved description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index e511199..66ea7de 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -101,7 +101,8 @@ specified: - ``platform description`` [mandatory] - a description of the VM platform, defining its resource assignments and requirements (see details in `Virtual Machine Configuration`_); - ``cpu_affinity`` [optional] - defines the affinity of the VM's vCPUs to the physical CPUs - assigned to the virtual platform (see details in `CPU Affinity`_); + assigned to the virtual platform. The affinity will be followed as best as possible, but it is + not guaranteed (see details in `CPU Affinity`_).; - ``colors`` [optional] - assignment of shared LLC cache colors (or partitions) to this VM (see details in `Coloring`_). From 7bfac37f5444434e623341767dc631d076f2a6ce Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 22 May 2024 13:52:09 +0100 Subject: [PATCH 49/71] fix(doc): improve readability Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 66ea7de..3f4cd3a 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -77,6 +77,7 @@ communication channels between two or more VMs, which may utilize shared memory :align: center :name: guest-config-fig +| Each entry in the ``vmlist`` mentioned earlier is a ``vm_config`` struct, which defines the configuration of each individual guest: @@ -180,6 +181,7 @@ For more details, refer to the figure below. :width: 100% :name: vm-image-fig +| Virtual Machine Configuration ***************************** @@ -291,6 +293,7 @@ fostering enhanced functionality and system efficiency. :width: 100% :name: ipc-fig +| Bao provides support for IPC, allowing VMs to establish communication channels. The IPC configuration involves defining the number of IPCs using the ``ipc_num`` field within the @@ -489,6 +492,7 @@ involves selecting the physical core where the guest should run. :width: 100% :name: cpu-affinity-fig +| This functionality is achieved through the following configuration parameter: @@ -516,6 +520,7 @@ Bao supports cache coloring which can be configured using the ``colors`` field w :width: 100% :name: llc-colors-fig +| It's important to note that cache coloring relies on the careful assignment of colors to each VM. However, this mechanism may not take effect if the physical mapping feature is enabled for a From 8a6069e2433f9c4946b4b5b41c74bb117b1f01fa Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 13:48:30 +0100 Subject: [PATCH 50/71] fix(vm_config): update image struct definition Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 3f4cd3a..c4bca29 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -85,7 +85,13 @@ configuration of each individual guest: .. code-block:: c struct vm_config { - struct vm_image image; + struct { + vaddr_t base_addr; + paddr_t load_addr; + size_t size; + bool separately_loaded; + bool inplace; + } image; vaddr_t entry; cpumap_t cpu_affinity; colormap_t colors; From 00a28ffd02c8edeac156fa2d0859626846edc820 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:01:11 +0100 Subject: [PATCH 51/71] fix(doc): improve readability Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index c4bca29..2952449 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -172,7 +172,7 @@ parameters such as ``img_name``, ``base_addr``, ``load_addr``, and size. For ins - ``IMAGE_LOADED``: Highly recommended, especially for MPU systems, where manual allocation of space for the guest image can be challenging if embedded in Bao's binary. Without utilizing - LOADED, Bao may need to copy the image, potentially resulting in space wastage. + LOADED, Bao may need to copy the image, potentially resulting in wasted space. Moreover, if the ``separately_loaded`` parameter is configured as false, the hypervisor interprets this setting as the offset of the built-in guest image within its own image, denoted as From 69735d900ae91470b9b688cd750092215ff595f4 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:03:48 +0100 Subject: [PATCH 52/71] fix(doc): remove content from memory regions section Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 5 ----- 1 file changed, 5 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 2952449..803650e 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -236,11 +236,6 @@ definition of: ################# .. _Memory Regions: -Bao employs a two-stage translation mechanism to manage VM memory. Each VM memory region is -delineated by its virtual address in the second stage. Generally, Bao manages the allocation of -physical memory for these regions. However, there is an option to define the physical memory region -where the specific region will be mapped. - For each VM, users can define multiple memory regions. To facilitate this, users first define the total number of memory regions via the ``region_num`` parameter: From e564d6e6f90ab61d7657c4118f1b55f674690370 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:06:33 +0100 Subject: [PATCH 53/71] fix(doc): redefine IPC nomenclature Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 803650e..682e176 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -216,7 +216,7 @@ definition of: - ``Number of CPUs`` - see details in `Number of vCPUs`_; - ``Memory regions`` - see details in `Memory Regions`_;` -- ``Inter-Process Comunication (IPC)`` - see details in `Inter-Process Communication (IPC)`_; +- ``Inter-Partition Comunication (IPC)`` - see details in `Inter-Partition Communication (IPC)`_; - ``Devices`` - see details in `Devices`_; - ``Architectural-Specific Configurations`` - see details in `Architectural-Specific Configurations`_; @@ -281,11 +281,11 @@ where: The usage of ``place_phys`` and ``phys`` allows users to manually allocate memory and obtain physical mappings. This feature provides a means to define the physical memory region explicitly. -3. Inter-Process Communication (IPC) -#################################### -.. _Inter-Process Communication (IPC): +3. Inter-Partition Communication (IPC) +###################################### +.. _Inter-Partition Communication (IPC): -Inter-Process Communication (IPC) enables seamless communication between distinct processes in a +Inter-Partition Communication (IPC) enables seamless communication between distinct partitions in a computing system. This mechanism facilitates data exchange, synchronization, and coordination, fostering enhanced functionality and system efficiency. From 696ae378ab4327d6c353941cfb76d0c8d5c05d10 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:10:00 +0100 Subject: [PATCH 54/71] fix(doc): improve IPC section readability Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 682e176..ba5d301 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -285,9 +285,8 @@ physical mappings. This feature provides a means to define the physical memory r ###################################### .. _Inter-Partition Communication (IPC): -Inter-Partition Communication (IPC) enables seamless communication between distinct partitions in a -computing system. This mechanism facilitates data exchange, synchronization, and coordination, -fostering enhanced functionality and system efficiency. +Inter-Partition Communication (IPC) enables communication between distinct partitions in a +computing system, facilitating data exchange, synchronization, and coordination between partitions. .. figure:: img/ipc.svg :align: center From a83bb96520b8955efd336b16238d573fb23e92fb Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:12:43 +0100 Subject: [PATCH 55/71] fix(doc): fix typo Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index ba5d301..4381461 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -297,7 +297,7 @@ computing system, facilitating data exchange, synchronization, and coordination Bao provides support for IPC, allowing VMs to establish communication channels. The IPC configuration involves defining the number of IPCs using the ``ipc_num`` field within the -``vm_config`` struct. The specifics of each IPC are then outlined through the ``ipcs`` structure, +``vm_platform`` struct. The specifics of each IPC are then outlined through the ``ipcs`` structure, including fields such as ``base``, ``size``, ``shmem_id``, ``interrupt_num``, and ``interrupts``. - ``ipc_num`` [optional] - defines the number of IPCs assigned to the VM. By default, ``ipc_num`` From 831e0663c86cd0e53ff5411f8c67aa12de02777e Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:49:29 +0100 Subject: [PATCH 56/71] fix(doc): improve readability by introducing GA and PA terms Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 4381461..91aaa0e 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -104,7 +104,7 @@ specified: - ``image`` [mandatory] - a structure containing information about guest image loading (see details in `Guest Image`_) -- ``entry`` [mandatory] - defines the entry point address in VM's address space; +- ``entry`` [mandatory] - defines the entry point address in guest's address space; - ``platform description`` [mandatory] - a description of the VM platform, defining its resource assignments and requirements (see details in `Virtual Machine Configuration`_); - ``cpu_affinity`` [optional] - defines the affinity of the VM's vCPUs to the physical CPUs @@ -135,9 +135,9 @@ guest side. It encompasses the following options: where: -- ``base_addr`` [mandatory] - corresponds to the ``image`` load address in the VM's address space; -- ``load_addr`` [mandatory] - corresponds to the ``image`` load address in physical memory/physical - address space. This value can be defined using the macro ``VM_IMAGE_OFFSET(img_name)``; +- ``base_addr`` [mandatory] - corresponds to the ``image`` load ``guest address``; +- ``load_addr`` [mandatory] - corresponds to the ``image`` load ``physical address``. This value + can be defined using the macro ``VM_IMAGE_OFFSET(img_name)``; - ``size`` [mandatory] - corresponds to the image size. For builtin images declared using `VM_IMAGE`, this value can be defined using the macro ``VM_IMAGE_SIZE(img_name)``; - ``separately_loaded`` [optional] - informs the hypervisor if the VM image is to be loaded @@ -255,7 +255,7 @@ Then, each memory region is described by populating the ``struct vm_mem_region`` where: -- ``base`` [mandatory] - corresponds to the base virtual address of the memory region; +- ``base`` [mandatory] - corresponds to the base ``guest address`` of the memory region; - ``size`` [mandatory] - corresponds to the size of the memory region; .. note:: @@ -264,7 +264,7 @@ where: - ``place_phys`` [optional] - the memory region is mapped into the virtual memory, and it's important to note that the virtual address (VA) might not necessarily be the same as the physical - address (PA). When "place_phys" is set to true, the virtual address corresponds to the physical + address (PA). When "place_phys" is set to true, the guest address corresponds to the physical address. If ``place_phys`` equals to true, it allows to specify the physical address of the memory region. By default, ``place_phys`` equals to false; @@ -318,7 +318,7 @@ including fields such as ``base``, ``size``, ``shmem_id``, ``interrupt_num``, an where: -- ``base`` [mandatory] - corresponds to the virtual base address of the IPC memory region; +- ``base`` [mandatory] - corresponds to the base ``guest address`` of the IPC memory region; - ``size`` [mandatory] - corresponds to the size of the IPC memory region; .. note:: @@ -358,8 +358,8 @@ where: where: -- ``pa`` [mandatory] - corresponds to the base physical address of the device; -- ``va`` [mandatory] - corresponds to the base virtual address of the device; +- ``pa`` [mandatory] - corresponds to the base ``physical address`` of the device; +- ``va`` [mandatory] - corresponds to the base guest ``physical address`` of the device; - ``size`` [mandatory] - corresponds to the size of the device memory region; .. note:: From 3558ec91451e0f20fb210ad2178de2abc561258f Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:56:44 +0100 Subject: [PATCH 57/71] fix(doc): update device configuration note Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 91aaa0e..17d1837 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -363,8 +363,9 @@ where: - ``size`` [mandatory] - corresponds to the size of the device memory region; .. note:: - It is mandatory for ``base`` and ``size`` to align with the smallest page size of the architecture. - For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to 64 bytes. + It is mandatory for ``base`` and ``size`` to align with the smallest page size of the + architecture. For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to + 64 bytes. Please note that for MPU systems there is no virtual address (VA). - ``interrupt_num`` [optional] - corresponds to the number of interrupts generated by the device to the VM. By default, ``interrupt_num`` equals to 0; From d2efdcf6c8ba853a19100408e1b1620feef7fa85 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 14:58:26 +0100 Subject: [PATCH 58/71] fix(doc): remove empty subsection Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 17d1837..31b6920 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -528,9 +528,6 @@ However, this mechanism may not take effect if the physical mapping feature is e specific memory region. Cache coloring exclusively operates in virtual memory systems, i.e., systems featuring Memory Management Units (MMUs) for address translation. -Shared Memory Configuration ---------------------------- - Configuration File Location --------------------------- From a139c992247e67ff35c787efe7afd279a15a7ff7 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 4 Sep 2024 15:12:36 +0100 Subject: [PATCH 59/71] fix(doc): update MPU VA and PA configuration description Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 31b6920..1c79156 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -365,7 +365,8 @@ where: .. note:: It is mandatory for ``base`` and ``size`` to align with the smallest page size of the architecture. For MMU systems, this typically aligns to 4K, while for MPU systems, it aligns to - 64 bytes. Please note that for MPU systems there is no virtual address (VA). + 64 bytes. Please note that for MPU systems, the virtual address (``va``) must match the physical + address (``pa``). - ``interrupt_num`` [optional] - corresponds to the number of interrupts generated by the device to the VM. By default, ``interrupt_num`` equals to 0; From 8b6eb3ba3dcf8555777c25390f619c1295cd4bfc Mon Sep 17 00:00:00 2001 From: Diogo21Costa Date: Sat, 29 Jul 2023 20:17:23 +0100 Subject: [PATCH 60/71] feat(doc): add shared memory configuration Signed-off-by: Diogo21Costa --- source/bao_hyp/config.rst | 40 ++++++++++++++++++++++++++++++++++++ source/bao_hyp/img/shmem.svg | 4 ++++ 2 files changed, 44 insertions(+) create mode 100644 source/bao_hyp/img/shmem.svg diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 1c79156..c173e11 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -529,6 +529,46 @@ However, this mechanism may not take effect if the physical mapping feature is e specific memory region. Cache coloring exclusively operates in virtual memory systems, i.e., systems featuring Memory Management Units (MMUs) for address translation. +Shared Memory Configuration +--------------------------- +.. _Shared Memory Configuration: + +In Bao's configuration multiple (``shmem0``, ``shmem1``, ...) or no shared memory can be set. If +at least one shared memory is defined, an ID is assigned to which the shared memory is +associated. Later, this ID can be associated with an IPC in the multi-guest configuration. + +.. figure:: img/shmem.svg + :align: center + :name: shmem-fig + +| + +The configuration of shared memories is done by populating a struct called +``shmem``, as follows: + +.. code-block:: c + + struct shmem { + size_t size; + bool place_phys; + union { + paddr_t base; + paddr_t phys; + }; + }; + + +This struct contains the following parameters: + +- **size** [mandatory] - defines the total size of the shared memory. The size of the shared \ + memory must always be multiples of 4Kb (0x1000); +- **place_phys** [optional] - shared memory is mapped into the virtual memory and it's \ + important to note that the virtual address (VA) might not necessarily be the same as the \ + physical address (PA). When ``place_phys`` is set to true, the virtual address corresponds to \ + the physical address. By default, ``place_phy`` equals to false; +- **base / phys** [optional] - defines the physical address (base address) of the shared memory. + + Configuration File Location --------------------------- diff --git a/source/bao_hyp/img/shmem.svg b/source/bao_hyp/img/shmem.svg new file mode 100644 index 0000000..6824deb --- /dev/null +++ b/source/bao_hyp/img/shmem.svg @@ -0,0 +1,4 @@ + + + +
Hypervisor
Hypervisor
...
...
VM N
VM N
VM 1
VM 1
VM 0
VM 0

shmem0

shmem0
VM 0
VM 0
VM 2
VM 2

shmem1

shmem1
VM 0
VM 0
VM 1
VM 1
VM N
VM N
shmem_list
shmem_listText is not SVG - cannot display \ No newline at end of file From 33652e5f3d7a0867ce9cda4be518f5496cf1078b Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 17 Sep 2024 14:16:33 +0100 Subject: [PATCH 61/71] fix(doc): improve shared memory documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index c173e11..767f083 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -533,9 +533,9 @@ Shared Memory Configuration --------------------------- .. _Shared Memory Configuration: -In Bao's configuration multiple (``shmem0``, ``shmem1``, ...) or no shared memory can be set. If -at least one shared memory is defined, an ID is assigned to which the shared memory is -associated. Later, this ID can be associated with an IPC in the multi-guest configuration. +In Bao's configuration multiple (``shmem0``, ``shmem1``, ...) or no shared memory can be set. An ID +is assigned to each shared memory object. Later, this ID can be linked to an IPC in the multi-guest +configuration. .. figure:: img/shmem.svg :align: center From b4d14d8ed94fcbddc9aa54c21ecef8d0eb53111b Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 17 Sep 2024 14:17:48 +0100 Subject: [PATCH 62/71] fix(doc): improve shared memory documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 767f083..3ca55f1 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -543,8 +543,8 @@ configuration. | -The configuration of shared memories is done by populating a struct called -``shmem``, as follows: +The configuration of shared memory is done by populating a struct called ``shmem``. The struct +definition is presented bellow: .. code-block:: c From f6b8cded246f578fef61aaf115ffd77e7988f732 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 17 Sep 2024 14:20:10 +0100 Subject: [PATCH 63/71] fix(doc): improve shared memory documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 3ca55f1..6d933e0 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -562,10 +562,10 @@ This struct contains the following parameters: - **size** [mandatory] - defines the total size of the shared memory. The size of the shared \ memory must always be multiples of 4Kb (0x1000); -- **place_phys** [optional] - shared memory is mapped into the virtual memory and it's \ - important to note that the virtual address (VA) might not necessarily be the same as the \ - physical address (PA). When ``place_phys`` is set to true, the virtual address corresponds to \ - the physical address. By default, ``place_phy`` equals to false; +- **place_phys** [optional] - shared memory is mapped into the virtual memory and it's important \ + to note that the guest address might not necessarily be the same as the physical address (PA). \ + When ``place_phys`` is set to true, the guest address corresponds to the physical address. By \ + default, ``place_phy`` equals to false; - **base / phys** [optional] - defines the physical address (base address) of the shared memory. From 07452d7d6faf6f7bd6e5205d03c04c05c84938f6 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 17 Sep 2024 14:27:37 +0100 Subject: [PATCH 64/71] fix(doc): improve shared memory documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 6d933e0..fde4b8c 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -562,11 +562,14 @@ This struct contains the following parameters: - **size** [mandatory] - defines the total size of the shared memory. The size of the shared \ memory must always be multiples of 4Kb (0x1000); -- **place_phys** [optional] - shared memory is mapped into the virtual memory and it's important \ - to note that the guest address might not necessarily be the same as the physical address (PA). \ - When ``place_phys`` is set to true, the guest address corresponds to the physical address. By \ - default, ``place_phy`` equals to false; -- **base / phys** [optional] - defines the physical address (base address) of the shared memory. +- **place_phys** [optional] - This flag determines the mapping of shared memory into virtual \ + memory. When ``place_phys`` is set to ``false`` (default value), the base address is a guest \ + (virtual) address. When set to ``true``, the guest address directly corresponds to the physical \ + address (PA) of the shared memory. +- **base / phys** [optional] - When ``place_phys`` is set to ``false``, the ``base`` parameter \ + should be used to specify the guest virtual address of the shared memory. When place_phys is \ + ``true``, the ``phys`` parameter should be used to specify the physical address of the shared \ + memory. Configuration File Location From 3f2a6f174933062acb335c27d1a70adca084d5bc Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Tue, 17 Sep 2024 14:47:16 +0100 Subject: [PATCH 65/71] fix(doc): improve shared memory documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index fde4b8c..ec558d9 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -533,9 +533,9 @@ Shared Memory Configuration --------------------------- .. _Shared Memory Configuration: -In Bao's configuration multiple (``shmem0``, ``shmem1``, ...) or no shared memory can be set. An ID -is assigned to each shared memory object. Later, this ID can be linked to an IPC in the multi-guest -configuration. +In Bao's configuration, you can set multiple shared memory regions (e.g., ``shmem0``, ``shmem1``, +...) or none at all. An ID is assigned to each shared memory object. Later, this ID can be linked +to an IPC in the multi-guest configuration. .. figure:: img/shmem.svg :align: center From 12a303c36d4dc3d6310e65291c125de5729336a5 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 19 Sep 2024 09:57:52 +0100 Subject: [PATCH 66/71] fix(doc): fix guest image documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index ec558d9..1ef55bd 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -120,7 +120,7 @@ Guest Image The guest ``image`` comprises a structure that describes the image configuration running on the guest side. It encompasses the following options: -- **image** [mandatory] - definition of the ``image`` to run on a given VM. The ``image`` +- ``image`` [mandatory] - definition of the ``image`` to run on a given VM. The ``image`` corresponds to the following structure: .. code-block:: c From 9ffd91e472802b0f7d18b6ea36be3344f52e904f Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 19 Sep 2024 09:59:06 +0100 Subject: [PATCH 67/71] fix(doc): fix VM config documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index 1ef55bd..e041205 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -214,11 +214,11 @@ By customizing this configuration, users can tailor the virtual platform to suit requirements and application needs for their virtual machines. The configuration includes the definition of: -- ``Number of CPUs`` - see details in `Number of vCPUs`_; -- ``Memory regions`` - see details in `Memory Regions`_;` -- ``Inter-Partition Comunication (IPC)`` - see details in `Inter-Partition Communication (IPC)`_; -- ``Devices`` - see details in `Devices`_; -- ``Architectural-Specific Configurations`` - see details in `Architectural-Specific +- **Number of CPUs** - see details in `Number of vCPUs`_; +- **Memory regions** - see details in `Memory Regions`_;` +- **Inter-Partition Comunication (IPC)** - see details in `Inter-Partition Communication (IPC)`_; +- **Devices** - see details in `Devices`_; +- **Architectural-Specific Configurations** - see details in `Architectural-Specific Configurations`_; 1. Number of vCPUs From 9744a25a97ea9dbb532bb1011ae33889d826cd17 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 19 Sep 2024 10:05:23 +0100 Subject: [PATCH 68/71] fix(doc): fix shared memory documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index e041205..ade4c5d 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -560,13 +560,13 @@ definition is presented bellow: This struct contains the following parameters: -- **size** [mandatory] - defines the total size of the shared memory. The size of the shared \ +- ``size`` [mandatory] - defines the total size of the shared memory. The size of the shared \ memory must always be multiples of 4Kb (0x1000); -- **place_phys** [optional] - This flag determines the mapping of shared memory into virtual \ +- ``place_phys`` [optional] - This flag determines the mapping of shared memory into virtual \ memory. When ``place_phys`` is set to ``false`` (default value), the base address is a guest \ (virtual) address. When set to ``true``, the guest address directly corresponds to the physical \ address (PA) of the shared memory. -- **base / phys** [optional] - When ``place_phys`` is set to ``false``, the ``base`` parameter \ +- ``base / phys`` [optional] - When ``place_phys`` is set to ``false``, the ``base`` parameter \ should be used to specify the guest virtual address of the shared memory. When place_phys is \ ``true``, the ``phys`` parameter should be used to specify the physical address of the shared \ memory. From b7787266d1b74ec1e430a6d98ad1429d0b1e16aa Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Thu, 19 Sep 2024 10:48:59 +0100 Subject: [PATCH 69/71] fix(doc): fix devices documentation Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index ade4c5d..ad7a6d8 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -342,8 +342,8 @@ where: .. _Devices: - ``dev_num`` [mandatory] - corresponds to the number of devices assigned to the VM; -- ``devs`` [mandatory if *dev_num* > 0] - corresponds to the specification of the VM's devices and - is configured through the following structure: +- ``devs`` [mandatory if ``dev_num`` > 0] - corresponds to the specification of the VM's devices + and is configured through the following structure: .. code-block:: c From 0c360b5e516e0cb6233d261d7a7877bcb5ec1676 Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Mon, 2 Dec 2024 14:23:19 +0000 Subject: [PATCH 70/71] fix(doc): add note regarding to MMIO mapping Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index ad7a6d8..dbba7a4 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -483,6 +483,16 @@ where: - ``iommu.fq_irq_id`` [mandatory if IOMMU is available] - the Fault Queue interrupt ID (the current implementation assumes this is a wired interrupt); +.. note:: + + When mapping MMIO regions for guests, the memory regions associated with the GIC (Generic + Interrupt Controller) must be excluded. Mapping these regions can lead to conflicts or incorrect + behavior, as they are typically managed by Bao through trap-and-emulate mechanisms. + + For instance, if a large MMIO range includes the GIC, the range should be split to create a + "hole" for the GIC. This ensures that GIC memory regions (or their equivalents in other + architectures, such as RISC-V) are not directly mapped into the guest's virtual address space. + CPU Affinity ************ From 7cc5013e494154afea5728bfca456de3de34749b Mon Sep 17 00:00:00 2001 From: Diogo Costa Date: Wed, 29 Jan 2025 11:39:02 +0700 Subject: [PATCH 71/71] fix(doc): update note regarding to MMIO mapping Signed-off-by: Diogo Costa --- source/bao_hyp/config.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/bao_hyp/config.rst b/source/bao_hyp/config.rst index dbba7a4..f4cce45 100644 --- a/source/bao_hyp/config.rst +++ b/source/bao_hyp/config.rst @@ -485,9 +485,10 @@ where: .. note:: - When mapping MMIO regions for guests, the memory regions associated with the GIC (Generic - Interrupt Controller) must be excluded. Mapping these regions can lead to conflicts or incorrect - behavior, as they are typically managed by Bao through trap-and-emulate mechanisms. + When mapping MMIO regions for guests, the memory regions associated with the interrupt + controller (only interrupt controllers are target of MMIO trap and emulate in Bao) must be + excluded. Mapping these regions can lead to conflicts or incorrect behavior, as they are + typically managed by Bao through trap-and-emulate mechanisms. For instance, if a large MMIO range includes the GIC, the range should be split to create a "hole" for the GIC. This ensures that GIC memory regions (or their equivalents in other