Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(new) : v1.0.0 #4

Closed
wants to merge 12 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Deploy
on:
push:
tags:
- '*'
- "*"
jobs:
deploy:
name: Deploy
Expand Down
41 changes: 17 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
![GitHub License](https://img.shields.io/github/license/sguesdon/docker-gcp-private-mirror)
[![Built with Devbox](https://www.jetify.com/img/devbox/shield_galaxy.svg)](https://www.jetify.com/devbox/docs/contributor-quickstart/)
![Test Status](https://github.com/sguesdon/docker-gcp-private-mirror/actions/workflows/tests.yaml/badge.svg?branch=main)
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/docker-gcp-private-mirror)](https://artifacthub.io/packages/helm/docker-gcp-private-mirror/docker-gcp-private-mirror)
[![Docker hub](https://img.shields.io/docker/v/sguesdon/docker-gcp-private-mirror?logo=docker&label=Docker%20hub)](https://hub.docker.com/r/sguesdon/docker-gcp-private-mirror/builds)

# Docker GCP private mirror

This project was created with the aim of using an image mirror from Artifact Registry in a Kubernetes cluster (GKE) within Google Cloud Platform. It addresses several issues, the first being the ability to call a mirror that contains a URL with a URI, not just a hostname (it is possible that Containerd now supports this, but that has not always been the case). It also leverages Workload Identity to automatically add an authorization header to all requests sent to the Artifact Registry image mirror.

## Deployment using helm
## Helm deployment

Before proceeding with the installation, you must have deployed an Artifact Registry repository, have a cluster with Workload Identity enabled, and possess a GCP user with permissions to read from your repository. An (Opentofu example)[tests/tofu] is available in the tests folder without the GKE cluster deployment.
Before proceeding with the installation, you must have deployed an Artifact Registry repository, have a cluster with Workload Identity enabled, and possess a GCP user with permissions to read from your repository. An [Opentofu example](tests/tofu) is available in the tests folder without the GKE cluster deployment.

### Minimum Configuration
### Basic configuration

```yaml
# values.yaml
fullnameOverride: gcp-mirror
nginx:
proxy:
Expand All @@ -29,33 +33,38 @@ serviceAccount:
### Command line

```sh
helm install gcp-mirror oci://registry-1.docker.io/sguesdon/docker-gcp-private-mirror --version 0.0.1
helm install gcp-mirror oci://registry-1.docker.io/sguesdon/docker-gcp-private-mirror --version <version>
```

### Using the Helm chart as a Helm dependency
### Using Helm chart dependencies

```yaml
# Chart.yaml
# [...]
dependencies:
- name: docker-gcp-private-mirror
alias: gcp-mirror
version: 0.0.1
version: <version>
repository: oci://registry-1.docker.io/sguesdon
# [...]
```

## All values
## Advanced Configuration

Other configurations are available, including settings related to the NGINX cache. The behavior of the sidecar responsible for retrieving the Google token can also be modified. All the values are available [here](src/values.yaml).

## Requirements to run tests locally
## Running tests

To quickly run the project, you need to use [DevBox](https://www.jetify.com/docs/devbox/installing_devbox/) and [direnv](https://www.jetify.com/docs/devbox/ide_configuration/direnv/). I encourage you to install it.

You will need Kubernetes locally to run the tests. Currently, the tests have already been successfully executed on the Kubernetes provided by Docker Desktop and on Minikube.

> Before running your tests, you must ensure that `kubectl` is properly configured to connect to your local cluster.
> If you are using Minikube, you will need to set the following variable: `MINIKUBE=true`

```sh
devbox run test
```

## Quick Deployment for Testing

Expand All @@ -76,19 +85,3 @@ After installation, you can quickly test the solution using the following comman
kubectl run dind --rm -it --image=docker:dind --privileged -- --insecure-registry docker-mirror --registry-mirror http://docker-mirror
kubectl exec -it dind -- docker pull redis:latest
```

## Running tests

Before running the tests, ensure that you have a functional Kubernetes cluster in your development environment.

> If you are using Minikube, you will need to set the following variable: `MINIKUBE=true`

```sh
devbox run test
```

## Lint helm chart

```sh
devbox run lint
```
1 change: 1 addition & 0 deletions resources/scripts/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ package_mirror_helm_chart() {
VERSION=$1

echo "--- package helm chart"
cp README.md ./src/README.md
helm package ./src --version="${VERSION}"
}

Expand Down
6 changes: 5 additions & 1 deletion resources/scripts/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,12 @@ clean_and_deploy_test_helm_chart() {
helm uninstall "${RELEASE_NAME}" || echo "Skipping uninstall"
kubectl wait --timeout=300s --for=delete --all "pod"

echo "--- Upgrade local dependencies"
rm ./tests/helm-chart/charts/*.tgz
helm dependency update ./tests/helm-chart

echo "--- Deploy test helm chart"
helm upgrade --dependency-update --install "$RELEASE_NAME" ./tests/helm-chart
helm upgrade --install "$RELEASE_NAME" ./tests/helm-chart
kubectl wait --timeout=300s --for=condition=available --all "deployment"
}

Expand Down
14 changes: 8 additions & 6 deletions src/resources/nginx.conf.template
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ http {

init_by_lua_block {

-- Function to read an environment variable with a default fallback
local function getenv_with_default(env_var, default)
local value = os.getenv(env_var)
if not value or value == "" then
Expand All @@ -27,11 +26,9 @@ http {
return value
end

-- Load configurations from environment variables
local token_file_path = getenv_with_default("TOKEN_FILE_PATH", "/config/token")
local token_cache_expiration = tonumber(getenv_with_default("TOKEN_CACHE_EXPIRATION_SECONDS", "300"))

-- Function to read the token from the file
function read_token()
local file = io.open(token_file_path, "r")
if not file then
Expand All @@ -50,7 +47,6 @@ http {
return token
end

-- Function to get the Authorization header
function get_auth_header(clear_cache)
clear_cache = clear_cache or false
local auth_header = ngx.shared.cache:get("auth_header")
Expand All @@ -71,7 +67,6 @@ http {
return auth_header
end

-- Preload the token at startup
get_auth_header()
}

Expand Down Expand Up @@ -115,7 +110,7 @@ http {
local token = get_auth_header(true)
if token then
ngx.req.set_header("Authorization", token)
ngx.exec(ngx.var.request_uri) -- Relancer la requête
ngx.exec(ngx.var.request_uri)
else
ngx.log(ngx.ERR, "Failed to refresh token on retry attempt #" .. retry_attempt)
end
Expand All @@ -142,5 +137,12 @@ http {
rewrite ^/v2/(.+)$ /v2/$base_rewrite_path/$1 break;
}
}

# Health check
location = /health {
access_log off;
add_header 'Content-Type' 'application/json';
return 200 '{"status":"up"}';
}
}
}
5 changes: 5 additions & 0 deletions src/resources/wait-token-file.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh
while [ ! -f "$TOKEN_FILE_PATH" ]; do
echo "Waiting for token file..."
sleep 1
done
4 changes: 3 additions & 1 deletion src/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,12 @@ data:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "docker-gcp-private-mirror.fullname" . }}-cloud-sdk-script
name: {{ include "docker-gcp-private-mirror.fullname" . }}-scripts
data:
generate-token-file.sh: |-
{{- .Values.cloudSdk.script | default (.Files.Get "resources/generate-token-file.sh") | nindent 4 }}
wait-token-file.sh: |-
{{- .Files.Get "resources/wait-token-file.sh" | nindent 4 }}
---
apiVersion: v1
kind: ConfigMap
Expand Down
12 changes: 8 additions & 4 deletions src/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ spec:
- name: nginx-config
mountPath: /etc/nginx/templates/nginx.conf.template
subPath: nginx.conf
- name: scripts
mountPath: /docker-entrypoint.d/99-wait-token-file.sh
subPath: wait-token-file.sh
envFrom:
- configMapRef:
name: {{ include "docker-gcp-private-mirror.fullname" . }}-nginx-env
Expand All @@ -78,7 +81,7 @@ spec:
{{- end }}
- name: shared-config
mountPath: /config
- name: cloud-sdk-script
- name: scripts
mountPath: /tmp/generate-token-file.sh
subPath: generate-token-file.sh
envFrom:
Expand All @@ -88,12 +91,13 @@ spec:
{{- if .Values.volumes }}
{{- toYaml .Values.volumes | nindent 8 }}
{{- end }}
- name: scripts
configMap:
name: {{ include "docker-gcp-private-mirror.fullname" . }}-scripts
defaultMode: 0777
- name: nginx-config
configMap:
name: {{ include "docker-gcp-private-mirror.fullname" . }}-nginx-config
- name: cloud-sdk-script
configMap:
name: {{ include "docker-gcp-private-mirror.fullname" . }}-cloud-sdk-script
- name: shared-config
emptyDir: {}
{{- with .Values.nodeSelector }}
Expand Down
22 changes: 11 additions & 11 deletions src/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,22 @@ nginx:
probes:
readinessProbe:
httpGet:
path: /v2/
path: /health
port: http
livenessProbe:
httpGet:
path: /v2/
path: /health
port: http
proxy:
bufferSize: '256k'
buffers: '8 256k'
busyBuffersSize: '512k'
largeClientHeaderBuffers: '32 5120k'
upstreamHost: 'europe-docker.pkg.dev'
upstreamProtocol: 'https'
rewritePath: 'gcp_project/registry_name'
tokenCacheExpirationSeconds: '300'
maxAuthRetryAttempts: '1'
bufferSize: "256k"
buffers: "8 256k"
busyBuffersSize: "512k"
largeClientHeaderBuffers: "32 5120k"
upstreamHost: "europe-docker.pkg.dev"
upstreamProtocol: "https"
rewritePath: "gcp_project/registry_name"
tokenCacheExpirationSeconds: "300"
maxAuthRetryAttempts: "1"

cloudSdk:
image:
Expand Down
21 changes: 17 additions & 4 deletions tests/fake-registry/src/authz.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,27 @@ import {
ERROR_IMAGE_NAME_URI,
FORWARDED_ERROR_IMAGE_NAME_URI,
FORWARDED_IMAGE_NAME_URI,
FORWARDED_MISSING_IMAGE_NAME_URI,
IMAGE_NAME_URI,
} from "./config/routes";
import { Request } from "./types/request.type";
import { SuperTestResponse } from "./types/supertest-response.type";

describe("authz cache management", () => {
it("should success call after reload bearer cache token in mirror service", async () => {
const token = Date.now().toString();

const origTokenResponse = await mockServerRequest
.get("/token")
.set("Content-Type", "text/plain")
.buffer(true)
.parse((res, cb) => {
let data = Buffer.from("");
res.on("data", (chunk) => (data = Buffer.concat([data, chunk])));
res.on("end", () => cb(null, data.toString()));
});

const origToken = origTokenResponse.body;

// update token parsed by sidecar
await mockServerRequest
.put("/token")
Expand All @@ -21,7 +34,7 @@ describe("authz cache management", () => {
res.on("data", (chunk) => (data = Buffer.concat([data, chunk])));
res.on("end", () => cb(null, data.toString()));
})
.send("xyz");
.send(token);

// wait sidecar parse new token calling mock server
await new Promise((resolve) => setTimeout(resolve, 2000));
Expand All @@ -42,7 +55,7 @@ describe("authz cache management", () => {
uri: FORWARDED_IMAGE_NAME_URI,
statusCode: 401,
headers: expect.objectContaining({
authorization: "Bearer abcd",
authorization: `Bearer ${origToken}`,
}),
});

Expand All @@ -51,7 +64,7 @@ describe("authz cache management", () => {
uri: FORWARDED_IMAGE_NAME_URI,
statusCode: 200,
headers: expect.objectContaining({
authorization: "Bearer xyz",
authorization: `Bearer ${token}`,
}),
});
});
Expand Down
21 changes: 15 additions & 6 deletions tests/helm-chart/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,24 @@ private-gcp-mirror:
repository: quay.io/curl/curl
script: |-
#!/bin/sh

URL="http://fake-registry/token"

while true; do
curl http://fake-registry/token > /config/token;
[ $? -ne 0 ] && exit 1
sleep 1
STATUS=$(curl -o /dev/null -s -w "%{http_code}" "$URL")
if [ "$STATUS" -eq 200 ]; then
break
else
sleep 5
fi
done

while true; do
curl "$URL" > /config/token;
[ $? -ne 0 ] && exit 1
sleep 1
done
nginx:
probes:
readinessProbe:
initialDelaySeconds: 30
proxy:
upstreamHost: "fake-registry"
upstreamProtocol: http
Expand Down
Loading