diff --git a/docs/getting-started/download.md b/docs/getting-started/download.md index 54f6223..c34b187 100644 --- a/docs/getting-started/download.md +++ b/docs/getting-started/download.md @@ -14,23 +14,30 @@ docker login ghcr.io ## Downloading the software -The main entry point for the software is the Hedgehog Fabricator CLI named `hhfab`. All software is published into the -OCI registry [GitHub Package](https://ghcr.io) including binaries, container images, or Helm charts. -Download the latest stable `hhfab` binary from the [GitHub Package](https://ghcr.io) using the following command: +The main entry point for the software is the Hedgehog Fabricator CLI named `hhfab`. + +Currently `hhfab` is supported on Linux x86/arm64 (tested on Ubuntu 22.04) and MacOS x86/arm64 for building +installers/upgraders. It may work on Windows WSL2 (with Ubuntu), but it's not tested. For running VLAB only Linux x86 +is currently supported. + +All software is published into the OCI registry [GitHub Package](https://ghcr.io) including binaries, container images, or Helm charts. +Download the latest stable `hhfab` binary from the [GitHub Package](https://ghcr.io) using the following command, it requires ORAS to be installed (see below): ```bash curl -fsSL https://i.hhdev.io/hhfab | bash ``` -Or download a specific version using the following command: +Or download a specific version (e.g. beta-1) using the following command: ```bash -curl -fsSL https://i.hhdev.io/hhfab | VERSION=alpha-X bash +curl -fsSL https://i.hhdev.io/hhfab | VERSION=beta-1 bash ``` -Use the `VERSION` environment variable to specify the version of the software to download. By default, the latest +Use the `VERSION` environment variable to specify the version of the software to download. By default, the latest stable release is downloaded. You can pick a specific release series (e.g. `alpha-2`) or a specific release. +### Installing ORAS + The download script requires [ORAS](https://oras.land/) to be installed. ORAS is used to download the binary from the OCI registry and can be installed using following command: @@ -38,8 +45,6 @@ OCI registry and can be installed using following command: curl -fsSL https://i.hhdev.io/oras | bash ``` -Currently only Linux x86 is supported for running `hhfab`. - ## Next steps * [Concepts](../concepts/overview.md) diff --git a/docs/install-upgrade/build-wiring.md b/docs/install-upgrade/build-wiring.md index f0cb925..8b6b863 100644 --- a/docs/install-upgrade/build-wiring.md +++ b/docs/install-upgrade/build-wiring.md @@ -8,7 +8,7 @@ A wiring diagram is a YAML file that is a digital representation of your network. You can find more YAML level details in the User Guide section [switch features and port naming](../user-guide/profiles.md) and the [api](../reference/api.md). It's mandatory for all switches to reference a `SwitchProfile` in the `spec.profile` of the `Switch` object. Only port naming defined by switch profiles could be used in the wiring diagram, NOS (or any other) port names aren't supported. In the meantime, to have a look at working wiring diagram for Hedgehog Fabric, run the sample generator that produces -VLAB-compatible wiring diagrams: +working wiring diagrams: ```console ubuntu@sl-dev:~$ hhfab sample -h @@ -28,6 +28,33 @@ OPTIONS: --help, -h show help ``` +Or you can generate a wiring diagram for a VLAB environment with flags to customize number of switches, links, servers, etc.: + +```console +ubuntu@sl-dev:~$ hhfab vlab gen --help +NAME: + hhfab vlab generate - generate VLAB wiring diagram + +USAGE: + hhfab vlab generate [command options] + +OPTIONS: + --bundled-servers value number of bundled servers to generate for switches (only for one of the second switch in the redundancy group or orphan switch) (default: 1) + --eslag-leaf-groups value eslag leaf groups (comma separated list of number of ESLAG switches in each group, should be 2-4 per group, e.g. 2,4,2 for 3 groups with 2, 4 and 2 switches) + --eslag-servers value number of ESLAG servers to generate for ESLAG switches (default: 2) + --fabric-links-count value number of fabric links if fabric mode is spine-leaf (default: 0) + --help, -h show help + --mclag-leafs-count value number of mclag leafs (should be even) (default: 0) + --mclag-peer-links value number of mclag peer links for each mclag leaf (default: 0) + --mclag-servers value number of MCLAG servers to generate for MCLAG switches (default: 2) + --mclag-session-links value number of mclag session links for each mclag leaf (default: 0) + --no-switches do not generate any switches (default: false) + --orphan-leafs-count value number of orphan leafs (default: 0) + --spines-count value number of spines if fabric mode is spine-leaf (default: 0) + --unbundled-servers value number of unbundled servers to generate for switches (only for one of the first switch in the redundancy group or orphan switch) (default: 1) + --vpc-loopbacks value number of vpc loopbacks for each switch (default: 0) +``` + ### Sample Switch Configuration ```yaml apiVersion: wiring.githedgehog.com/v1beta1 @@ -64,7 +91,7 @@ A connection represents the physical wires in your data center. They connect swi #### Server Connections -A server connection is a connection used to connect servers to the fabric. The fabric will configure the server-facing port according to the type of the connection (MLAG, Bundle, etc).The configuration of the actual server needs to be done by the server administrator. The server name is not validated by the fabric and is used as metadata to identify the connection. A server connection can be one of: +A server connection is a connection used to connect servers to the fabric. The fabric will configure the server-facing port according to the type of the connection (MLAG, Bundle, etc). The configuration of the actual server needs to be done by the server administrator. The server port names are not validated by the fabric and used as metadata to identify the connection. A server connection can be one of: - *Unbundled* - A single cable connecting switch to server. - *Bundled* - Two or more cables going to a single switch, a LAG or similar. diff --git a/docs/install-upgrade/config.md b/docs/install-upgrade/config.md index a8c4da2..e83fb2a 100644 --- a/docs/install-upgrade/config.md +++ b/docs/install-upgrade/config.md @@ -11,12 +11,21 @@ For a VLAB user, the typical workflow with hhfab is: 1. `hhfab init --dev` 1. `hhfab vlab gen` -1. `hhfab vlab up --kill-stale` +1. `hhfab vlab up` -The above workflow will get a user up and running with a spine-leaf VLAB. The `--kill-stale` option is supplied as its harmless on the first run and stops a lot of problems from happening with an successive run. +The above workflow will get a user up and running with a spine-leaf VLAB. ### HHFAB for Physical Machines +It's possible to start from scratch: + +1. `hhfab init` (see different flags to cusomize initial configuration) +1. Adjust the `fab.yaml` file to your needs +1. `hhfab validate` +1. `hhfab build` + +Or import existing config and wiring files: + 1. `hhfab init -c fab.yaml -w wiring-file.yaml -w extra-wiring-file.yaml` 1. `hhfab validate` 1. `hhfab build` @@ -25,65 +34,9 @@ After the above workflow a user will have a .img file suitable for installing th ## Fab.yaml -The fabric YAML object has 4 objects: - -- `mode` - either `spine-leaf` or `collapsed-core` -- `includeONIE` - defaults to `true` -- `defaultSwitchUsers` - the admin and operator credentials for SONiC. -- `defaultAlloyConfig` - the configuration details for telemetry of switch information - -### Forward switch metrics and logs - -There is an option to enable Grafana Alloy on all switches to forward metrics and logs to the configured targets using -Prometheus Remote-Write API and Loki API. If those APIs are available from Control Node(s), but not from the switches, -it's possible to enable HTTP Proxy on Control Node(s) that will be used by Grafana Alloy running on the switches to -access the configured targets. It could be done by passing `--control-proxy=true` to `hhfab init`. - -Metrics includes port speeds, counters, errors, operational status, transceivers, fans, power supplies, temperature -sensors, BGP neighbors, LLDP neighbors, and more. Logs include agent logs. +### Configure control node and switch users -Configuring the exporters and targets is currently only possible by editing the `fab.yaml` configuration file. An example configuration is provided below: - -```yaml -spec: - config: - ... - defaultAlloyConfig: - agentScrapeIntervalSeconds: 120 - unixScrapeIntervalSeconds: 120 - unixExporterEnabled: true - lokiTargets: - grafana_cloud: # target name, multiple targets can be configured - basicAuth: # optional - password: "" - username: "" - labels: # labels to be added to all logs - env: env-1 - url: https://logs-prod-021.grafana.net/loki/api/v1/push - useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy - prometheusTargets: - grafana_cloud: # target name, multiple targets can be configured - basicAuth: # optional - password: "" - username: "" - labels: # labels to be added to all metrics - env: env-1 - sendIntervalSeconds: 120 - url: https://prometheus-prod-36-prod-us-west-0.grafana.net/api/prom/push - useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy - unixExporterCollectors: # list of node-exporter collectors to enable, https://grafana.com/docs/alloy/latest/reference/components/prometheus.exporter.unix/#collectors-list - - cpu - - filesystem - - loadavg - - meminfo - collectSyslogEnabled: true # collect /var/log/syslog on switches and forward to the lokiTargets -``` - -For additional options, see the `AlloyConfig` [struct in Fabric repo](https://github.com/githedgehog/fabric/blob/master/api/meta/alloy.go). - -### Configure switch users - -Configuring switch users is done either passing `--default-password-hash` to `hhfab init` or editing the resulting `fab.yaml` file emitted by `hhfab init`. You can specify users to be configured on the switches in the following format: +Configuring control node and switch users is done either passing `--default-password-hash` to `hhfab init` or editing the resulting `fab.yaml` file emitted by `hhfab init`. You can specify users to be configured on the control node(s) and switches in the following format: ```yaml spec: @@ -96,8 +49,8 @@ spec: fabric: mode: spine-leaf # "spine-leaf" or "collapsed-core" - - defaultSwitchUsers: + + defaultSwitchUsers: admin: # at least one user with name 'admin' and role 'admin' role: admin #password: "$5$8nAYPGcl4..." # password hash @@ -110,6 +63,9 @@ spec: # - "ssh-ed25519 AAAAC3Nza..." ``` + +Control node(s) user is always named `core`. + The role of the user,`operator` is read-only access to `sonic-cli` command on the switches. In order to avoid conflicts, do not use the following usernames: `operator`,`hhagent`,`netops`. ### NTP and DHCP @@ -136,7 +92,57 @@ spec: ``` The **management** interface is for the control node to manage the fabric switches, *not* end-user management of the control node. For end-user management of the control node specify the **external** interface name. +### Forward switch metrics and logs + +There is an option to enable Grafana Alloy on all switches to forward metrics and logs to the configured targets using +Prometheus Remote-Write API and Loki API. If those APIs are available from Control Node(s), but not from the switches, +it's possible to enable HTTP Proxy on Control Node(s) that will be used by Grafana Alloy running on the switches to +access the configured targets. It could be done by passing `--control-proxy=true` to `hhfab init`. + +Metrics includes port speeds, counters, errors, operational status, transceivers, fans, power supplies, temperature +sensors, BGP neighbors, LLDP neighbors, and more. Logs include agent logs. + +Configuring the exporters and targets is currently only possible by editing the `fab.yaml` configuration file. An example configuration is provided below: + +```yaml +spec: + config: + ... + defaultAlloyConfig: + agentScrapeIntervalSeconds: 120 + unixScrapeIntervalSeconds: 120 + unixExporterEnabled: true + lokiTargets: + grafana_cloud: # target name, multiple targets can be configured + basicAuth: # optional + password: "" + username: "" + labels: # labels to be added to all logs + env: env-1 + url: https://logs-prod-021.grafana.net/loki/api/v1/push + useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy + prometheusTargets: + grafana_cloud: # target name, multiple targets can be configured + basicAuth: # optional + password: "" + username: "" + labels: # labels to be added to all metrics + env: env-1 + sendIntervalSeconds: 120 + url: https://prometheus-prod-36-prod-us-west-0.grafana.net/api/prom/push + useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy + unixExporterCollectors: # list of node-exporter collectors to enable, https://grafana.com/docs/alloy/latest/reference/components/prometheus.exporter.unix/#collectors-list + - cpu + - filesystem + - loadavg + - meminfo + collectSyslogEnabled: true # collect /var/log/syslog on switches and forward to the lokiTargets +``` + +For additional options, see the `AlloyConfig` [struct in Fabric repo](https://github.com/githedgehog/fabric/blob/master/api/meta/alloy.go). + ## Complete Example File + ```yaml apiVersion: fabricator.githedgehog.com/v1beta1 kind: Fabricator diff --git a/docs/install-upgrade/overview.md b/docs/install-upgrade/overview.md index da4926a..f7443af 100644 --- a/docs/install-upgrade/overview.md +++ b/docs/install-upgrade/overview.md @@ -5,17 +5,17 @@ ## Prerequisites -* A machine with access to the Internet to use Fabricator and build installer -* An 8 GB USB flash drive, if you are not using virtual media +* A machine with access to the Internet to use Fabricator and build installer with at least 8 GB RAM and 25 GB of disk space +* An 16 GB USB flash drive, if you are not using virtual media * Have a machine to function as the Fabric Control Node. [System Requirements](./requirements.md) as well as IPMI access to it to install the OS. -* A management switch with at least 1 10GbE port +* A management switch with at least 1 10GbE port is recommended * Enough [Supported Switches](./supported-devices.md) for your Fabric ## Overview of Install Process This section is dedicated to the Hedgehog Fabric installation on bare-metal control node(s) and switches, their -preparation and configuration. To install the vlab see [Vlab Overview](../vlab/overview.md). +preparation and configuration. To install the VLAB see [VLAB Overview](../vlab/overview.md). Download and install `hhfab` following instructions from the [Download](../getting-started/download.md) section. @@ -32,7 +32,7 @@ The main steps to install Fabric are: 1. Connect management switch to Fabric control node 1. Connect 1GbE Management port of switches to management switch 1. Prepare supported switches - 1. Ensure switch serial numbers and / or management interface mac addresses are recorded in wiring diagram + 1. Ensure switch serial numbers and / or first management interface MAC addresses are recorded in wiring diagram 1. Boot them into ONIE Install Mode to have them automatically provisioned ## Build Control Node configuration and Installer @@ -63,7 +63,7 @@ There are utilities that assist this process such as [etcher](https://etcher.bal ## Install Control Node -This control node should be given a static IP address. Either a lease or statically assigned. +This control node should be given a static IP address. Either a lease or statically assigned. 1. Configure the server to use UEFI boot **without** secure boot @@ -93,7 +93,7 @@ The control node is dual-homed. It has a 10GbE interface that connects to the ma Now that the install has finished, you can start interacting with the Fabric using `kubectl`, `kubectl fabric` and `k9s`, all pre-installed as part of the Control Node installer. -At this stage, the fabric hands out DHCP addresses to the switches via the management network. Optionally, you can monitor this process by going through the following steps: +At this stage, the fabric hands out DHCP addresses to the switches via the management network. Optionally, you can monitor this process by going through the following steps: - enter `k9s` at the command prompt - use the arrow keys to select the pod named `fabric-boot` - the logs of the pod will be displayed showing the DHCP lease process diff --git a/docs/install-upgrade/requirements.md b/docs/install-upgrade/requirements.md index edc0b78..a85331f 100644 --- a/docs/install-upgrade/requirements.md +++ b/docs/install-upgrade/requirements.md @@ -1,31 +1,31 @@ # System Requirements +## Out of Band Management Network + +In order to provision and manage the switches that comprise the fabric, an out of band switch must also be installed. This network is to be used exclusively by the control node and the fabric switches, no other access is permitted. This switch (or switches) is not managed by the fabric. It is recommended that this switch have at least a 10GbE port and that port connect to the control node. + ## Control Node -- Fast SSDs for system/root as well as Kubernetes and container runtime folders are required for stable work -- SSDs are mandatory for Control Nodes -- 10 GbE port for connection to management network + +- Fast SSDs for system/root is mandatory for Control Nodes + - NVMe SSDs are recommended + - DRAM-less NAND SSDs are not supported (e.g. Crucial BX500) +- 10 GbE port for connection to management network is recommended - Minimal (non-HA) setup is a single Control Node - (Future) Full (HA) setup is at least 3 Control Nodes - (Future) Extra nodes could be used for things like Logging, Monitoring, Alerting stack, and more - -## Out of Band Management Network - -In order to provision and manage the switches that comprise the fabric, an out of band switch must also be installed. This network is to be used exclusively by the control node and the fabric switches, no other access is permitted. This switch is not manged by the fabric. It is recommended that this switch have at least a 10GbE port and that port connect to the control node. - - -## Non-HA (minimal) setup - 1 Control Node +### Non-HA (minimal) setup - 1 Control Node - Control Node runs non-HA Kubernetes Control Plane installation with non-HA Hedgehog Fabric Control Plane on top of it - Not recommended for more then 10 devices participating in the Hedgehog Fabric or production deployments | | Minimal | Recommended | | ---- | ------- | ----------- | -| CPU | 4 | 8 | -| RAM | 12 GB | 16 GB | -| Disk | 100 GB | 250 GB | +| CPU | 6 | 8 | +| RAM | 16 GB | 32 GB | +| Disk | 150 GB | 250 GB | -## (Future) HA setup - 3+ Control Nodes (per node) +### (Future) HA setup - 3+ Control Nodes (per node) - Each Control Node runs part of the HA Kubernetes Control Plane installation with Hedgehog Fabric Control Plane on top of it in HA mode as well @@ -33,15 +33,23 @@ In order to provision and manage the switches that comprise the fabric, an out o | | Minimal | Recommended | | ---- | ------- | ----------- | -| CPU | 4 | 8 | -| RAM | 12 GB | 16 GB | -| Disk | 100 GB | 250 GB | +| CPU | 6 | 8 | +| RAM | 16 GB | 32 GB | +| Disk | 150 GB | 250 GB | + +### Reference Control Node Configuration + +- AMD EPYC 4344P (8C/16T, 3.8 GHz, 32 MB L3, 65W, single socket) +- 32 GB DDR5-4800 ECC UDIMM (2 x 16 GB) +- Micron 7450 MAX 400GB NVMe ## Device participating in the Hedgehog Fabric (e.g. switch) - (Future) Each participating device is part of the Kubernetes cluster, so it runs Kubernetes kubelet - Additionally, it runs the Hedgehog Fabric Agent that controls devices configuration +Following resources should be available on a device to run in the Hedgehog Fabric (after other software such as SONiC usage): + | | Minimal | Recommended | | ---- | ------- | ----------- | | CPU | 1 | 2 | diff --git a/docs/install-upgrade/supported-devices.md b/docs/install-upgrade/supported-devices.md index bf5abdc..f27eebd 100644 --- a/docs/install-upgrade/supported-devices.md +++ b/docs/install-upgrade/supported-devices.md @@ -1,10 +1,12 @@ # Supported Devices -You can find mode details in the User Guide including [switch features and port naming](../user-guide/profiles.md). +You can find detailed information about devices in the [Switch Profiles Catalog](../reference/profiles.md) and in the +User Guide [switch features and port naming](../user-guide/profiles.md). ## Spine * Celestica DS3000 +* Celestica DS4000 * Dell S5232F-ON * Edgecore DCS204 (AS7726-32X) * Edgecore DCS501 (AS7712-32X-EC) diff --git a/docs/reference/api.md b/docs/reference/api.md index fb84089..7a79cfe 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -38,6 +38,8 @@ _Appears in:_ #### Agent + + Agent is an internal API object used by the controller to pass all relevant information to the agent running on a specific switch in order to fully configure it and manage its lifecycle. It is not intended to be used directly by users. Spec of the object isn't user-editable, it is managed by the controller. Status of the object is updated by @@ -1868,6 +1870,7 @@ _Appears in:_ | `label` _string_ | Label defines the physical port label you can see on the actual switch | | | | `group` _string_ | If port isn't directly manageable, group defines the group it belongs to, exclusive with profile | | | | `profile` _string_ | If port is directly configurable, profile defines the profile it belongs to, exclusive with group | | | +| `management` _boolean_ | Management defines if port is a management port, it's a special case and it can't have a group or profile | | | | `oniePortName` _string_ | OniePortName defines the ONIE port name for management ports only | | | diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index f5c6ae6..29c3851 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -1,5 +1,29 @@ # Release notes +## Beta-1 + +### Device support + +- Celestica DS4000 as a spine + +### SONiC + +- Broadcom SONiC 4.4.0 support + +### Fabric provisioning, management + +- Out-of-band management network connectivity +- Deprecated support for in-band management network connectivity, chain boot, and front-panel boot until further notice +- Automatic zero touch switch provisioning [ ZTP ] is based on the serial number or the first management interface MAC address +- Full support for airgap installations and upgrades by default +- Self-contained USB image generation for control node installation +- Automated in-place upgrades for control node(s) moving forward + +## API + +- API version v1beta1 +- Guaranteed backward compatibility moving forward + ## Alpha-7 ### Device Support diff --git a/docs/vlab/running.md b/docs/vlab/running.md index e8b9a0b..c8f25aa 100644 --- a/docs/vlab/running.md +++ b/docs/vlab/running.md @@ -5,29 +5,29 @@ before running VLAB. ## Initialize VLAB -First, initialize Fabricator by running `hhfab init --dev`. This command supports several customization options that are listed in the output of `hhfab init --help`. +First, initialize Fabricator by running `hhfab init --dev`. This command supports several customization options that are listed in the output of `hhfab init --help`. ```console ubuntu@docs:~$ hhfab init --dev -11:26:52 INF Hedgehog Fabricator version=v0.30.0 -11:26:52 INF Generated initial config -11:26:52 INF Adjust configs (incl. credentials, modes, subnets, etc.) file=fab.yaml -11:26:52 INF Include wiring files (.yaml) or adjust imported ones dir=include +11:26:52 INF Hedgehog Fabricator version=v0.30.0 +11:26:52 INF Generated initial config +11:26:52 INF Adjust configs (incl. credentials, modes, subnets, etc.) file=fab.yaml +11:26:52 INF Include wiring files (.yaml) or adjust imported ones dir=include ``` ## VLAB Topology -By default, the command creates 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf with 2 fabric connections (between each spine and leaf), 2 MCLAG peer links and 2 MCLAG session links as well as 2 loopbacks per leaf for implementing VPC loopback workaround. To generate the preceding topology, `hhfab vlab gen`. You can also configure the number of spines, leafs, connections, and so on. For example, flags `--spines-count` and `--mclag-leafs-count` allow you to set the number of spines and MCLAG leaves, respectively. For complete options, `hhfab vlab gen -h`. +By default, the command creates 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf with 2 fabric connections (between each spine and leaf), 2 MCLAG peer links and 2 MCLAG session links as well as 2 loopbacks per leaf for implementing VPC loopback workaround. To generate the preceding topology, `hhfab vlab gen`. You can also configure the number of spines, leafs, connections, and so on. For example, flags `--spines-count` and `--mclag-leafs-count` allow you to set the number of spines and MCLAG leaves, respectively. For complete options, `hhfab vlab gen -h`. ```console ubuntu@docs:~$ hhfab vlab gen -21:27:16 INF Hedgehog Fabricator version=v0.30.0 -21:27:16 INF Building VLAB wiring diagram fabricMode=spine-leaf -21:27:16 INF >>> spinesCount=2 fabricLinksCount=2 -21:27:16 INF >>> eslagLeafGroups=2 -21:27:16 INF >>> mclagLeafsCount=2 mclagSessionLinks=2 mclagPeerLinks=2 -21:27:16 INF >>> orphanLeafsCount=1 vpcLoopbacks=2 -21:27:16 INF >>> mclagServers=2 eslagServers=2 unbundledServers=1 bundledServers=1 -21:27:16 INF Generated wiring file name=vlab.generated.yaml +21:27:16 INF Hedgehog Fabricator version=v0.30.0 +21:27:16 INF Building VLAB wiring diagram fabricMode=spine-leaf +21:27:16 INF >>> spinesCount=2 fabricLinksCount=2 +21:27:16 INF >>> eslagLeafGroups=2 +21:27:16 INF >>> mclagLeafsCount=2 mclagSessionLinks=2 mclagPeerLinks=2 +21:27:16 INF >>> orphanLeafsCount=1 vpcLoopbacks=2 +21:27:16 INF >>> mclagServers=2 eslagServers=2 unbundledServers=1 bundledServers=1 +21:27:16 INF Generated wiring file name=vlab.generated.yaml ``` ### Collapsed Core @@ -106,18 +106,18 @@ ubuntu@docs:~$ hhfab vlab up 11:49:06 INF Preparing new vm=spine-01 type=switch 11:49:06 INF Preparing new vm=spine-02 type=switch 11:49:06 INF Starting VMs count=18 cpu="54 vCPUs" ram="49664 MB" disk="550 GB" -11:49:59 INF Uploading control install vm=control-1 type=control -11:53:39 INF Running control install vm=control-1 type=control -11:53:40 INF control-install: 01:53:39 INF Hedgehog Fabricator Recipe version=v0.30.0 vm=control-1 -11:53:40 INF control-install: 01:53:39 INF Running control node installation vm=control-1 -12:00:32 INF control-install: 02:00:31 INF Control node installation complete vm=control-1 -12:00:32 INF Control node is ready vm=control-1 type=control -12:00:32 INF All VMs are ready +11:49:59 INF Uploading control install vm=control-1 type=control +11:53:39 INF Running control install vm=control-1 type=control +11:53:40 INF control-install: 01:53:39 INF Hedgehog Fabricator Recipe version=v0.30.0 vm=control-1 +11:53:40 INF control-install: 01:53:39 INF Running control node installation vm=control-1 +12:00:32 INF control-install: 02:00:31 INF Control node installation complete vm=control-1 +12:00:32 INF Control node is ready vm=control-1 type=control +12:00:32 INF All VMs are ready ``` When the message `INF Control node is ready vm=control-1 type=control` from the installer's output means that the installer has finished. After this line has been displayed, you can get into the control node and other VMs to watch the Fabric coming up and switches getting -provisioned. See [Accessing the Vlab](#accessing-the-vlab). +provisioned. See [Accessing the VLAB](#accessing-the-vlab). ## Configuring VLAB VMs