Bundle helm charts with CLI (loft-sh#812)

* feat(cli): embed helm charts

Signed-off-by: Rohan CJ <[email protected]>

* feat(docs): update CONTRIBUTING.md wtih CLI building steps

Signed-off-by: Rohan CJ <[email protected]>

Signed-off-by: Rohan CJ <[email protected]>

.dockerignore

@@ -2,11 +2,10 @@
 /.devspace
 /.git
 /.vscode
-/charts
 /api
 /kubeconfig.yaml
 /cmd/virtualclusterctl
 /test
 /conformance
 /docs
-devspace.yaml
+devspace.yaml

.github/workflows/e2e.yaml

@@ -57,7 +57,7 @@ jobs:
       - name: Build vcluster cli
         run: |
           set -x
-          GO111MODULE=on GOFLAGS=-mod=vendor go build -v -o ./vcluster ./cmd/vclusterctl/main.go
+          go generate ./... && GO111MODULE=on GOFLAGS=-mod=vendor go build -v -o ./vcluster ./cmd/vclusterctl/main.go
       - name: Upload vcluster cli to artifact
         uses: actions/upload-artifact@v2
         with:
@@ -100,6 +100,10 @@ jobs:
       with:
         version: "v0.14.0"
         image: kindest/node:v1.25.0
+
+    - name: Generate Embedded Helm Charts
+      run: |
+        go generate ./...
     
     - name: Testing kind cluster set-up
       run: |

.github/workflows/lint.yaml

@@ -22,6 +22,9 @@ jobs:
         with:
           go-version: 1.19
       - uses: actions/checkout@v3
+      - name: Generate Embedded Helm Charts
+        run: |
+          go generate ./...
       - name: Run golangci-lint
         uses: golangci/[email protected]
         with:

.gitignore

@@ -9,3 +9,5 @@ profile.out
 /vcluster
 /zzz
 /cmd/vclusterctl/__debug_bin
+/release
+/cmd/vclusterctl/cmd/charts/

CONTRIBUTING.md

@@ -1,8 +1,20 @@
-## Contributing to vcluster
+# Contributing to vcluster
 
-Thank you for contributing to vcluster! Here you can find common questions around developing vcluster.
+Thank you for contributing to vcluster! Here you can find common questions around developing vcluster. [See the docs](https://www.vcluster.com/docs/architecture/basics) for architecture details.
 
-### How can I get involved?
+## Table of Contents
+
+- [How can I get involved?](#how-can-i-get-involved)
+- [Developing the Syncer Container using devspace](#developing-the-syncer-container-using-devspace)
+  - [Setup vcluster for Development](#setup-vcluster-for-development)
+- [Debug vcluster with Delve](#debug-vcluster-with-delve)
+  - [Running vcluster Tests](#running-vcluster-tests)
+- [Build vcluster CLI tool](#build-vcluster-cli-tool)
+- [Developing the hostpath-mapper component instead of syncer](#developing-the-hostpath-mapper-component-instead-of-syncer)
+  - [License](#license)
+  - [Copyright notice](#copyright-notice)
+
+## How can I get involved?
 
 There are a number of areas where contributions can be accepted:
 
@@ -18,11 +30,17 @@ There are a number of areas where contributions can be accepted:
 
 This is just a short list of ideas, if you have other ideas for contributing please make a suggestion.
 
+## Developing the Syncer Container using devspace
+
+See [docs](https://www.vcluster.com/docs/architecture/basics#vcluster-syncer) for explanation of the syncer.
+Devspace will set up a vcluster and sync your code changes from the repository to a running container.
+
 ### Setup vcluster for Development
 
 We recommend to develop vcluster directly in a Kubernetes cluster as it makes feedback a lot quicker. For the quick setup, you'll need to install [devspace](https://github.com/loft-sh/devspace#1-install-devspace), docker, kubectl, helm and make sure you have a local Kubernetes cluster (such as Docker Desktop, minikube, KinD or similar) installed.
 
 Fork and clone the repo:
+
 - Click Fork button (top right) to establish a cloud-based fork.
 - git clone your-fork-url
 
@@ -37,7 +55,7 @@ Which should produce an output similar to:
 ```
 [info]   Using namespace 'vcluster'
 [info]   Using kube context 'docker-desktop'
-[done] √ Created image pull secret vcluster/devspace-auth-ghcr-io   
+[done] √ Created image pull secret vcluster/devspace-auth-ghcr-io
 [info]   Building image 'ghcr.io/loft-sh/loft-enterprise/dev-vcluster:szFLbkA' with engine 'docker'
 Sending build context to Docker daemon  52.71MB
 Step 1/14 : FROM golang:1.15 as builder
@@ -49,12 +67,12 @@ Step 14/14 : ENTRYPOINT ["go", "run", "-mod", "vendor", "cmd/vcluster/main.go"]
 Successfully built b14eacaa5e29
 Successfully tagged ghcr.io/loft-sh/loft-enterprise/dev-vcluster:szFLbkA
 [info]   Skip image push for ghcr.io/loft-sh/loft-enterprise/dev-vcluster
-[done] √ Done processing image 'ghcr.io/loft-sh/loft-enterprise/dev-vcluster'                         
+[done] √ Done processing image 'ghcr.io/loft-sh/loft-enterprise/dev-vcluster'
 [info]   Execute 'helm upgrade vcluster ./chart --namespace vcluster --values /var/folders/bc/qxzrp6f93zncnj1xyz25kyp80000gn/T/079539791 --install --kube-context docker-desktop'
 [info]   Execute 'helm list --namespace vcluster --output json --kube-context docker-desktop'
-[done] √ Deployed helm chart (Release revision: 1)              
-[done] √ Successfully deployed vcluster with helm               
-                                             
+[done] √ Deployed helm chart (Release revision: 1)
+[done] √ Successfully deployed vcluster with helm
+
 #########################################################
 [info]   DevSpace UI available at: http://localhost:8090
 #########################################################
@@ -68,6 +86,7 @@ root@vcluster-0:/vcluster#
 ```
 
 Then you can start vcluster with
+
 ```
 go run -mod vendor cmd/vcluster/main.go start
 ```
@@ -92,35 +111,53 @@ root@vcluster-0:/vcluster#
 
 To access the virtual cluster, you can use the `vcluster connect` command locally as with any other virtual cluster.
 
-##### Developing the hostpath-mapper instead of syncer
-In case you need to develop the hostpath-mapper daemonset instead of the syncer, you can use the `dev-hostpath-mapper` profile in `devspace.yaml`. You can do this by running the following command:
-```
-devspace dev -p dev-hostpath-mapper
-```
-This should deploy the hostpath-mapper as a deployment instead of a daemonset and take care of other modifications to be made and should allow you to develop the hostpath-mapper itself against the syncer.
+### Debug vcluster with Delve
 
-#### Debug vcluster with Delve
-If you wish to run vcluster in the debug mode with delve, run `devspace dev -n vcluster` and wait until you see the command prompt (`root@vcluster-0:/vcluster#`).  
-Run `dlv debug ./cmd/vcluster/main.go --listen=0.0.0.0:2345 --api-version=2 --output /tmp/__debug_bin --headless --build-flags="-mod=vendor" -- start`  
-Wait until the `API server listening at: [::]:2345` message appears.  
-Start the `Debug vcluster (localhost:2346)` configuration in VSCode to connect your debugger session.  
-If you are not using VSCode, configure your IDE to connect to `localhost:2346` for the "remote" delve debugging.  
-**Note:** vcluster won't start until you connect with the debugger.  
-**Note:** vcluster will be stopped once you detach your debugger session.  
+If you wish to run vcluster in the debug mode with delve, run `devspace dev -n vcluster` and wait until you see the command prompt (`root@vcluster-0:/vcluster#`).
+Run `dlv debug ./cmd/vcluster/main.go --listen=0.0.0.0:2345 --api-version=2 --output /tmp/__debug_bin --headless --build-flags="-mod=vendor" -- start`
+Wait until the `API server listening at: [::]:2345` message appears.
+Start the `Debug vcluster (localhost:2346)` configuration in VSCode to connect your debugger session.
+If you are not using VSCode, configure your IDE to connect to `localhost:2346` for the "remote" delve debugging.
+**Note:** vcluster won't start until you connect with the debugger.
+**Note:** vcluster will be stopped once you detach your debugger session.
 
 ### Running vcluster Tests
 
-You can run the unit test suite with `./hack/test.sh` which should execute all the vcluster tests.  
+You can run the unit test suite with `./hack/test.sh` which should execute all the vcluster tests.
 
-The e2e test suite can be run from the e2e folder(`cd e2e`) with this command - `go test -v -ginkgo.v`.  
+The e2e test suite can be run from the e2e folder(`cd e2e`) with this command - `go test -v -ginkgo.v`.
 Alternatively, if you [install ginkgo binary](https://github.com/onsi/ginkgo#global-installation), you can run it with `ginkgo -v`.
 
-For running conformance tests, please take a look at [conformance tests](https://github.com/loft-sh/vcluster/tree/main/conformance/v1.21)
+For running conformance tests, please take a look at [conformance tests](https://github.com/loft-sh/tree/vcluster/main/conformance/v1.21)
+
+## Build vcluster CLI tool
+
+Build:
+
+```
+go generate ./... && go build -o vcluster cmd/vclusterctl/main.go # build vcluster cli
+```
+
+Run:
+
+```
+./vcluster create v1 # create vcluster
+```
+
+## Developing the hostpath-mapper component instead of syncer
+
+In case you need to develop the hostpath-mapper daemonset instead of the syncer, you can use the `dev-hostpath-mapper` profile in `devspace.yaml`. You can do this by running the following command:
+
+```
+devspace dev -p dev-hostpath-mapper
+```
+
+This should deploy the hostpath-mapper as a deployment instead of a daemonset and take care of other modifications to be made and should allow you to develop the hostpath-mapper itself against the syncer.
 
-### License
+## License
 
 This project is licensed under the Apache 2.0 License.
 
-### Copyright notice
+## Copyright notice
 
 It is important to state that you retain copyright for your contributions, but agree to license them for usage by the project and author(s) under the Apache 2.0 license. Git retains history of authorship, but we use a catch-all statement rather than individual names.

Dockerfile

@@ -36,6 +36,11 @@ RUN mkdir -p /.cache /.config
 ENV GOCACHE=/.cache
 ENV GOENV=/.config
 
+# Copy and embed the helm charts
+COPY charts/ charts/
+COPY hack/ hack/
+RUN go generate ./...
+
 # Set home to "/" in order to for kubectl to automatically pick up vcluster kube config 
 ENV HOME /
 

Dockerfile.cli

@@ -13,13 +13,19 @@ COPY vendor/ vendor/
 COPY cmd/vclusterctl cmd/vclusterctl
 COPY pkg/ pkg/
 
+# Copy and embed the helm charts
+COPY charts/ charts/
+COPY hack/ hack/
+RUN go generate ./...
+
 ENV GO111MODULE on
 ENV DEBUG true
 
 # create and set GOCACHE now, this should slightly speed up the first build inside of the container
 RUN mkdir -p /.cache
 ENV GOCACHE=/.cache
 
+
 # Build cmd
 RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} GO111MODULE=on go build -mod vendor -o /vcluster cmd/vclusterctl/main.go
 
@@ -28,4 +34,4 @@ FROM alpine:3.16
 # Set home to "/" in order to for kubectl to automatically pick up vcluster kube config
 ENV KUBECONFIG=/root/.kube/config
 COPY --from=builder /vcluster /usr/local/bin/vcluster
-CMD /usr/local/bin/vcluster
+CMD /usr/local/bin/vcluster

cmd/vclusterctl/cmd/create.go

@@ -2,8 +2,10 @@ package cmd
 
 import (
 	"context"
+	"embed"
 	"encoding/base64"
 	"fmt"
+	"io/fs"
 	"os"
 	"os/exec"
 	"path/filepath"
@@ -38,6 +40,10 @@ import (
 	clientcmdapi "k8s.io/client-go/tools/clientcmd/api"
 )
 
+//go:generate ../../../hack/embed-charts.sh
+//go:embed charts/*.tgz
+var Charts embed.FS
+
 var (
 	AllowedDistros              = []string{"k3s", "k0s", "k8s", "eks"}
 	CreatedByVClusterAnnotation = "vcluster.loft.sh/created"
@@ -259,6 +265,8 @@ func getBase64DecodedString(values string) (string, error) {
 	return string(strDecoded), nil
 }
 
+// func
+
 func (cmd *CreateCmd) deployChart(vClusterName, chartValues, helmExecutablePath string) error {
 	// check if there is a vcluster directory already
 	workDir, err := os.Getwd()
@@ -269,16 +277,44 @@ func (cmd *CreateCmd) deployChart(vClusterName, chartValues, helmExecutablePath
 		return fmt.Errorf("aborting vcluster creation. Current working directory contains a file or a directory with the name equal to the vcluster chart name - \"%s\". Please execute vcluster create command from a directory that doesn't contain a file or directory named \"%s\"", cmd.ChartName, cmd.ChartName)
 	}
 
-	// rewrite chart location, this is an optimization to avoid
-	// downloading the whole index.yaml and parsing it
-	if cmd.LocalChartDir == "" && cmd.ChartVersion != "" && cmd.ChartRepo == LoftChartRepo {
-		if cmd.ChartVersion[0] == 'v' {
-			cmd.ChartVersion = cmd.ChartVersion[1:]
-		}
+	if cmd.LocalChartDir == "" {
+		chartEmbedded := false
+		if cmd.ChartVersion == upgrade.GetVersion() { // use embedded chart if default version
+			embeddedChartName := fmt.Sprintf("%s-%s.tgz", cmd.ChartName, upgrade.GetVersion())
+			// not using filepath.Join because the embed.FS separator is not OS specific
+			embeddedChartPath := fmt.Sprintf("charts/%s", embeddedChartName)
+
+			embeddedChartFile, err := Charts.ReadFile(embeddedChartPath)
+			if err != nil && errors.Is(err, fs.ErrNotExist) {
+				cmd.log.Infof("Chart not embedded: %q, pulling from helm repository.", err)
+			} else if err != nil {
+				cmd.log.Errorf("Unexpected error while accessing embedded file: %q", err)
+			} else {
+				temp, err := os.CreateTemp("", fmt.Sprintf("%s%s", embeddedChartName, "-"))
+				if err != nil {
+					cmd.log.Errorf("Error creating temp file: %v", err)
+				} else {
+					defer temp.Close()
+					defer os.Remove(temp.Name())
+					_, err = temp.Write(embeddedChartFile)
+					if err != nil {
+						cmd.log.Errorf("Error writing package file to temp: %v", err)
+					}
+					cmd.LocalChartDir = temp.Name()
+					chartEmbedded = true
+					cmd.log.Debugf("Using embedded chart: %q", embeddedChartName)
+				}
 
-		cmd.LocalChartDir = LoftChartRepo + "/charts/" + cmd.ChartName + "-" + cmd.ChartVersion + ".tgz"
-		cmd.ChartVersion = ""
-		cmd.ChartRepo = ""
+			}
+		}
+		// rewrite chart location, this is an optimization to avoid
+		// downloading the whole index.yaml and parsing it
+		if !chartEmbedded && cmd.ChartRepo == LoftChartRepo { // specify versioned path to repo url
+			cmd.ChartVersion = strings.TrimPrefix(cmd.ChartVersion, "v")
+			cmd.LocalChartDir = LoftChartRepo + "/charts/" + cmd.ChartName + "-" + cmd.ChartVersion + ".tgz"
+			cmd.ChartVersion = ""
+			cmd.ChartRepo = ""
+		}
 	}
 
 	if cmd.Upgrade {

cmd/vclusterctl/main.go

@@ -1,14 +1,15 @@
 package main
 
 import (
+	"os"
+
 	"github.com/loft-sh/vcluster/cmd/vclusterctl/cmd"
 	"github.com/loft-sh/vcluster/pkg/upgrade"
-	"os"
 
 	_ "k8s.io/client-go/plugin/pkg/client/auth"
 )
 
-var version string = ""
+var version string = "0.0.1"
 
 func main() {
 	upgrade.SetVersion(version)

hack/build-cli.sh

@@ -27,6 +27,8 @@ if [[ "$(pwd)" != "${VCLUSTER_ROOT}" ]]; then
   exit 1
 fi
 
+RELEASE_VERSION="${RELEASE_VERSION}" go generate ${VCLUSTER_ROOT}/...
+
 GO_BUILD_CMD="go build -a"
 GO_BUILD_LDFLAGS="-s -w -X main.commitHash=${COMMIT_HASH} -X main.buildDate=${DATE} -X main.version=${RELEASE_VERSION}"
 

hack/embed-charts.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+
+# This script generates packaged helm releases to embed in the vcluster binary
+
+set -eu
+
+
+VCLUSTER_ROOT="$(realpath $(dirname ${0})/..)"
+RELEASE_VERSION="${RELEASE_VERSION:-0.0.1}"
+EMBED_DIR="${VCLUSTER_ROOT}/cmd/vclusterctl/cmd/charts"
+
+rm -rfv "${EMBED_DIR}"
+mkdir "${EMBED_DIR}"
+for CHART in k3s k8s k0s eks;
+do
+    helm package --version "${RELEASE_VERSION}" "${VCLUSTER_ROOT}/charts/${CHART}" -d "${EMBED_DIR}";
+done
+

hack/test.sh

@@ -6,7 +6,7 @@ export GOFLAGS=-mod=vendor
 
 # Test if we can build the program
 echo "Building virtual cluster..."
-CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build cmd/vcluster/main.go || exit 1
+go generate ./... && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build cmd/vcluster/main.go || exit 1
 
 # List packages
 PKGS=$(go list ./... | grep -v -e /vendor/ -e /test)