Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ Integrate GitHub Actions, add Supercronic #1

Merged
merged 37 commits into from
Jul 17, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
e061414
[PIMINT-171] Build Docker images with GitHub action
astehlik Jul 16, 2024
c2f3bbb
[PIMINT-171] Fix Docker image directory
astehlik Jul 16, 2024
c6ff571
[PIMINT-171] Add Prettier config and make commands
astehlik Jul 16, 2024
219d038
[PIMINT-171] Rename PHP build job
astehlik Jul 16, 2024
b772f87
[PIMINT-171] Install supercronic in PHP CLI images
astehlik Jul 16, 2024
ae6b6bd
[PIMINT-171] Cancel running jobs on new push
astehlik Jul 16, 2024
5deccd3
[PIMINT-171] Schedule rebuild every other day
astehlik Jul 16, 2024
f7edf10
[PIMINT-171] Restructure multi arch build
astehlik Jul 16, 2024
d306fa1
[PIMINT-171] Re-enable build for gh-build branch
astehlik Jul 16, 2024
22c3f4b
[PIMINT-171] Add missing dependency
astehlik Jul 16, 2024
b2a5cff
[PIMINT-171] Fix build input name
astehlik Jul 16, 2024
03b4a99
[PIMINT-171] Lowercase image name
astehlik Jul 16, 2024
6931e4c
[PIMINT-171] Prevent duplicat artifact names
astehlik Jul 16, 2024
487c5e8
[PIMINT-171] Add scope for Docker build cache
astehlik Jul 16, 2024
6efb985
[PIMINT-171] Provide GitHub token to build action
astehlik Jul 16, 2024
ee397ca
[PIMINT-171] Improve cache scope identifier
astehlik Jul 16, 2024
c9d15be
[PIMINT-171] Fix merge
astehlik Jul 16, 2024
cf46dd7
[PIMINT-171] Add Workflow for cleaning up the registry
astehlik Jul 16, 2024
5f95b40
[PIMINT-171] Trigger cleanup Action, disable build Action
astehlik Jul 16, 2024
348d6ea
[PIMINT-171] Fix filename
astehlik Jul 16, 2024
bbfa933
[PIMINT-171] Add missin version
astehlik Jul 16, 2024
c5ce3f1
[PIMINT-171] Restructure image cleanup
astehlik Jul 16, 2024
8e3907b
[PIMINT-171] Add missing dependency
astehlik Jul 16, 2024
49a3b46
[PIMINT-171] Reformat
astehlik Jul 16, 2024
74f31ad
[PIMINT-171] Restore manifest.json
astehlik Jul 16, 2024
d4a4242
[PIMINT-171] Fix typo in jq command
astehlik Jul 16, 2024
9388d7f
[PIMINT-171] Fix jq formatting
astehlik Jul 16, 2024
5e51f1c
[PIMINT-171] Fix cleanup
astehlik Jul 16, 2024
e9a9e06
[PIMINT-171] Fix metadata extraction
astehlik Jul 16, 2024
f8e05eb
[PIMINT-171] Fix image tag separator
astehlik Jul 16, 2024
e39ed66
[PIMINT-171] Enable image cleanup
astehlik Jul 16, 2024
7f03a37
[PIMINT-171] Improve cleanup
astehlik Jul 16, 2024
17c948e
[PIMINT-171] Allow manual builds
astehlik Jul 17, 2024
dfeb0b1
[PIMINT-171] Build on pull requests without pushing and cleanup
astehlik Jul 17, 2024
afaf0b1
[PIMINT-171] Execute cleanup as dry run in PRs
astehlik Jul 17, 2024
8c2eec9
[PIMINT-171] Fix input in cleanup
astehlik Jul 17, 2024
cc816d3
[PIMINT-171] Fix cleanup dry run condition
astehlik Jul 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[*]
charset = utf-8
end_of_line = lf
indent_size = 4
indent_style = space
insert_final_newline = true
max_line_length = 120
tab_width = 4

[*.json]
indent_size = 2
160 changes: 160 additions & 0 deletions .github/workflows/build-and-push-image.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
name: 'Build and push image'

on:
workflow_call:
inputs:
image_registry:
type: string
description: 'Image registry'
required: true
image_identifier:
type: string
description: 'Image identifier'
required: true
image_name:
type: string
description: 'Fully qualified image name without registry and organization, e.g. pimcore-docker-image/php'
required: true
image_tag:
type: string
description: 'Image tag'
required: true
image_directory:
type: string
description: 'Image directory'
required: true
image_push:
type: boolean
description: 'Push image to registry'
required: false
default: false

env:
REGISTRY_IMAGE: ${{ inputs.image_registry }}/${{ github.repository_owner }}/${{ inputs.image_name }}

jobs:
build-and-push-php:
name: Build and push PHP
runs-on: ${{ matrix.architecture.runs-on }}
permissions:
contents: read
packages: write
id-token: write
strategy:
matrix:
architecture:
- runs-on: ubuntu-latest
platform: linux/amd64
identifier: linux-amd64
- runs-on: bscm-github-actions-runner-set-arm
platform: linux/arm64
identifier: linux-arm64
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log into registry ${{ inputs.image_registry }}
uses: docker/login-action@v3
with:
registry: ${{ inputs.image_registry }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY_IMAGE }}
tags: ${{ inputs.image_tag }}

- name: Build and push by digest
id: build
uses: docker/build-push-action@v6
with:
context: ${{ inputs.image_directory }}
platforms: ${{ matrix.architecture.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY_IMAGE }},push-by-digest=true,name-canonical=true,push=${{ inputs.image_push }}
cache-from: type=gha,scope=${{ inputs.image_identifier }}-${{ matrix.architecture.identifier }}
cache-to: type=gha,mode=max,scope=${{ inputs.image_identifier }}-${{ matrix.architecture.identifier }}
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Export digest and tag
run: |
mkdir -p /tmp/build-metadata//digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/build-metadata/digests/${digest#sha256:}"

mkdir -p /tmp/build-metadata/image-tags
echo "${{ inputs.image_name }}:${{ inputs.image_tag }}" > "/tmp/build-metadata/image-tags/${{ inputs.image_identifier }}"

- name: Upload build metadata
uses: actions/upload-artifact@v4
with:
name: build-metadata-${{ inputs.image_identifier }}-${{ matrix.architecture.identifier }}
path: /tmp/build-metadata/*
if-no-files-found: error
retention-days: 1

merge:
name: Merge architecture images
needs:
- build-and-push-php
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write
steps:
- name: Install cosign
uses: sigstore/[email protected]

- name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/build-metadata
pattern: build-metadata-${{ inputs.image_identifier }}-*
merge-multiple: true

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log into registry ${{ inputs.image_registry }}
uses: docker/login-action@v3
with:
registry: ${{ inputs.image_registry }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY_IMAGE }}
tags: ${{ inputs.image_tag }}

- name: Create manifest list and push
working-directory: /tmp/build-metadata/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *)
if: ${{ inputs.image_push }}

- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }}

manifestJson=$(docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}:${{ steps.meta.outputs.version }} --format "{{json .Manifest}}")
digest=$(jq -r '.digest' <<< "$manifestJson")
echo "DIGEST=$digest" >> $GITHUB_ENV
if: ${{ inputs.image_push }}

- name: Sign the published Docker image
env:
TAGS: ${{ steps.meta.outputs.tags }}
shell: bash
run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST}
if: ${{ inputs.image_push }}
45 changes: 45 additions & 0 deletions .github/workflows/build-and-push.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: 'Build and push images'

on:
pull_request:
push:
branches:
- main
# Rebuild every other day
schedule:
- cron: '0 0 */2 * *'
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build-and-push-php:
name: Build and push PHP
strategy:
matrix:
php_version:
- 8.1
- 8.2
- 8.3
php_image:
- cli
- fpm
uses: ./.github/workflows/build-and-push-image.yaml
with:
image_identifier: 'php-${{ matrix.php_image }}-${{ matrix.php_version }}'
image_registry: ghcr.io
image_name: pimcore-docker-image/php
image_tag: ${{ matrix.php_version }}-${{ matrix.php_image }}-pimcore
image_directory: dist/images/php/${{ matrix.php_version }}-${{ matrix.php_image }}-pimcore
image_push: ${{ github.ref_name == github.event.repository.default_branch }}

clean:
name: Cleanup registry
needs:
- build-and-push-php
uses: ./.github/workflows/docker-registry-cleanup.yaml
with:
image_registry: ghcr.io
dry_run: ${{ github.ref_name == github.event.repository.default_branch }}
63 changes: 63 additions & 0 deletions .github/workflows/docker-registry-cleanup.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: 'Cleanup Docker registry'

on:
workflow_call:
inputs:
image_registry:
type: string
description: 'Image registry'
required: true
dry_run:
type: boolean
description: 'Dry run'
required: false
default: true

env:
REGISTRY: ghcr.io

jobs:
cleanup:
name: Cleanup regisitry
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write
steps:
- name: Download digests
uses: actions/download-artifact@v4
with:
path: /tmp/build-metadata
pattern: build-metadata-*
merge-multiple: true

- name: Log into registry ${{ env.REGISTRY }}
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Fetch multi-platform package version SHAs
id: multi-arch-digests
working-directory: /tmp/build-metadata/image-tags
run: |
imageNames=""
for identifier in *; do
imageNameAndTag="$(cat ${identifier})"
imageName=$(echo $imageNameAndTag | cut -d: -f1)

imageNames="$imageNames $imageName"
done

unqiueImageNames=$(echo $imageNames | tr ' ' '\n' | sort -u | tr '\n' ' ')
echo "image-names=$unqiueImageNames" >> $GITHUB_OUTPUT

- uses: snok/[email protected]
with:
account: basecom
token: ${{ secrets.GITHUB_TOKEN }}
image-names: ${{ steps.multi-arch-digests.outputs.image-names }}
cut-off: 2h
dry-run: ${{ github.event.inputs.dry_run }}
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,10 @@ build-images:
php orca.phar --directory=.

build-images-debug:
php orca.phar --directory=. --debug
php orca.phar --directory=. --debug

lint:
prettier --check .github

lint-fix:
prettier --write .github
25 changes: 25 additions & 0 deletions dist/images/docker/stable/php/install_supercronic.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

set -e

# Copied from https://github.com/aptible/supercronic/releases

SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.30/supercronic-linux-amd64
SUPERCRONIC=supercronic-linux-amd64
SUPERCRONIC_SHA1SUM=9f27ad28c5c57cd133325b2a66bba69ba2235799

if [ "$(uname -m)" = "aarch64" ]; then
SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.30/supercronic-linux-arm64
SUPERCRONIC=supercronic-linux-arm64
SUPERCRONIC_SHA1SUM=d5e02aa760b3d434bc7b991777aa89ef4a503e49
fi

curl -fsSLO "$SUPERCRONIC_URL"

echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c -

chmod +x "$SUPERCRONIC"

mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}"

ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
25 changes: 25 additions & 0 deletions dist/images/docker/test/php/install_supercronic.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

set -e

# Copied from https://github.com/aptible/supercronic/releases

SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.30/supercronic-linux-amd64
SUPERCRONIC=supercronic-linux-amd64
SUPERCRONIC_SHA1SUM=9f27ad28c5c57cd133325b2a66bba69ba2235799

if [ "$(uname -m)" = "aarch64" ]; then
SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.30/supercronic-linux-arm64
SUPERCRONIC=supercronic-linux-arm64
SUPERCRONIC_SHA1SUM=d5e02aa760b3d434bc7b991777aa89ef4a503e49
fi

curl -fsSLO "$SUPERCRONIC_URL"

echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c -

chmod +x "$SUPERCRONIC"

mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}"

ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
5 changes: 5 additions & 0 deletions dist/images/php/8.1-cli-pimcore/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,11 @@ RUN curl -fsSLO "https://github.com/imagemin/pngout-bin/raw/main/vendor/linux/x6
&& chmod 0755 pngout \
&& mv pngout /usr/local/bin/pngout

# install supercronic
COPY php/install_supercronic.sh /opt/

RUN bash /opt/install_supercronic.sh

# configure xDebug
RUN echo "xdebug.idekey = PHPSTORM" >> /usr/local/etc/php/conf.d/20-xdebug.ini; \
echo "xdebug.mode = off" >> /usr/local/etc/php/conf.d/20-xdebug.ini; \
Expand Down
25 changes: 25 additions & 0 deletions dist/images/php/8.1-cli-pimcore/php/install_supercronic.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash

set -e

# Copied from https://github.com/aptible/supercronic/releases

SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.30/supercronic-linux-amd64
SUPERCRONIC=supercronic-linux-amd64
SUPERCRONIC_SHA1SUM=9f27ad28c5c57cd133325b2a66bba69ba2235799

if [ "$(uname -m)" = "aarch64" ]; then
SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.30/supercronic-linux-arm64
SUPERCRONIC=supercronic-linux-arm64
SUPERCRONIC_SHA1SUM=d5e02aa760b3d434bc7b991777aa89ef4a503e49
fi

curl -fsSLO "$SUPERCRONIC_URL"

echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c -

chmod +x "$SUPERCRONIC"

mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}"

ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
Loading