Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Cluster Toolkit Dockerfile for backend integration with XPK #3237

Merged
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions tools/cloud-build/images/cluster-toolkit-dockerfile/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


# Use a google-cloud-cli image as the base
ARG BASE_IMAGE=gcr.io/google.com/cloudsdktool/google-cloud-cli:stable
FROM ${BASE_IMAGE}

# Install necessary tools and libraries
RUN apt-get update && \
apt-get install -y --no-install-recommends \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release \
git \
make \
unzip \
wget \
python3 \
python3-pip \
python3-venv

# Install Terraform
ARG TERRAFORM_VERSION=1.5.2
RUN wget -q "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" -O terraform.zip && \
unzip terraform.zip && \
mv terraform /usr/local/bin/ && \
rm terraform.zip

# Install Packer
ARG PACKER_VERSION=1.8.6
RUN wget -q "https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_amd64.zip" -O packer.zip && \
unzip packer.zip && \
mv packer /usr/local/bin/ && \
rm packer.zip

# Install Go
ARG GO_VERSION=1.21.0
RUN wget -q "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" -O go.tar.gz && \
tar -C /usr/local -xzf go.tar.gz && \
rm go.tar.gz

# Set GOPATH and add to PATH
ENV GOPATH /go
ENV PATH $PATH:/usr/local/go/bin:$GOPATH/bin

# Clone the Cluster Toolkit repository
ARG CLUSTER_TOOLKIT_VERSION=main
RUN git clone --branch ${CLUSTER_TOOLKIT_VERSION} https://github.com/GoogleCloudPlatform/cluster-toolkit.git /cluster-toolkit

# Create and activate the virtual environment within the gcluster folder
WORKDIR /cluster-toolkit
ENV VIRTUAL_ENV /cluster-toolkit/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH "$VIRTUAL_ENV/bin:$PATH"

# Build the Cluster Toolkit
RUN make install-dev-deps
RUN make

# Make gcluster available
RUN cp /cluster-toolkit/gcluster /usr/local/bin/gcluster
RachaelSTamakloe marked this conversation as resolved.
Show resolved Hide resolved
RUN echo $PATH

# Command to execute when running the container (placeholder)
ENTRYPOINT ["gcluster"]
CMD ["--help"]
59 changes: 59 additions & 0 deletions tools/cloud-build/images/cluster-toolkit-dockerfile/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Cluster Toolkit Dockerfile

This repository contains a Dockerfile for building a Docker image with Cluster Toolkit and its dependencies installed and the `gcluster` binary readily available for use.

## System Requirements

* [Docker Engine](https://docs.docker.com/engine/) needs to be installed to build a Docker image from this Dockerfile.

## Build Arguments
The following build arguments can be used to customize the build process:
* **`BASE_IMAGE`**: The base image to use for the build. Defaults to `gcr.io/google.com/cloudsdktool/google-cloud-cli:stable`.
* **`TERRAFORM_VERSION`**: The version of Terraform to install. Defaults to `1.5.2`.
* **`PACKER_VERSION`**: The version of Packer to install. Defaults to `1.8.6`.
* **`GO_VERSION`**: The version of Go to install. Defaults to `1.21.0`.
* **`CLUSTER_TOOLKIT_VERSION`**: The version (tag) of the Cluster Toolkit to install. Defaults to the current version associated with the `main` branch of the cluster toolkit repository.
RachaelSTamakloe marked this conversation as resolved.
Show resolved Hide resolved
RachaelSTamakloe marked this conversation as resolved.
Show resolved Hide resolved

## Build the Cluster Toolkit Docker Image
To build the Cluster Toolkit Docker image, navigate to the directory the Dockerfile is present in and run the following command:

```bash
docker build --build-arg BASE_IMAGE=<base_image> \
--build-arg TERRAFORM_VERSION=<terraform_version> \
--build-arg PACKER_VERSION=<packer_version> \
--build-arg GO_VERSION=<go_version> \
--build-arg CLUSTER_TOOLKIT_VERSION=<cluster_toolkit_version> \
-t <image_name> .
```

Example:

```bash
docker build --build-arg CLUSTER_TOOLKIT_VERSION=v1.40.0 -t my-cluster-toolkit-image .
```

The above example builds an image tagged `my-cluster-toolkit-image` and sets the CLUSTER_TOOLKIT_VERSION to v1.40.0 while using the default values for other arguments.

## Run the Cluster Toolkit Docker Image
To run the Cluster Toolkit Docker image, use the following command:

```bash
docker run -v ~/.config/gcloud/:/root/.config/gcloud -v $(pwd):/data <image_name> <gcluster_command>
```

This command runs the Cluster Toolkit Docker image and allows the `gcluster` binary to access your Google Cloud credentials and local files. Here's a breakdown:

* `-v ~/.config/gcloud/:/root/.config/gcloud`: This part mounts your local Google Cloud configuration directory `(~/.config/gcloud)` to the `/root/.config/gcloud` directory inside the container. This allows the `gcluster` binary to access your credentials and interact with Google Cloud resources.
RachaelSTamakloe marked this conversation as resolved.
Show resolved Hide resolved
* `-v $(pwd):/data`: This part mounts your current working directory `($(pwd))` to the `/data` directory inside the container. This allows you to access any files in your current directory (like blueprint files) from within the container. You can then reference these files in your `<gcluster_command>` using the `/data` path.
* `<image_name>`: Replace this with the name of your Docker image.
* `<gcluster_command>`: Replace this with the `gcluster` command you want to execute. The Cluster Toolkit Docker image has `ENTRYPOINT ["gcluster"]` in its Dockerfile. This means that the `gcluster` command is automatically executed when the container starts, and any arguments provided after `<image_name>` in the docker run command are passed as arguments to `gcluster`.

When using the docker container to run `gcluster` commands that either deploy or modify cloud resources, it's strongly recommended to save a local copy of the deployment folder that's generated inside of the docker container. This can be done by using the `--out` flag in combination with some `gcluster` commands to set the output directory where the `gcluster` deployment directory will be created. Ideally, the output directory should be a directory mounted from your local machine to the docker container. This ensures that the deployment folder persists even after the container exits, allowing you to access and manage the deployment artifacts even after the container is removed.
RachaelSTamakloe marked this conversation as resolved.
Show resolved Hide resolved
RachaelSTamakloe marked this conversation as resolved.
Show resolved Hide resolved

Example:

```bash
docker run -v ~/.config/gcloud/:/root/.config/gcloud -v $(pwd):/data my-cluster-toolkit-image deploy /data/my-blueprint.yaml --out /data --auto-approve
```

This example runs the `deploy` command with the blueprint file `my-blueprint.yaml` located in your current directory. The `--out /data` flag saves the deployment folder to your current directory, and `--auto-approve` automatically approves any prompts from `gcluster`, streamlining the deployment process.
Loading