Skip to content

Commit

Permalink
Ci/release (#1967)
Browse files Browse the repository at this point in the history 
* Create manual release.yml file

Release script to repackage chainweb-node builds, tag and push chainweb docker containers, tag the release commit, generate release info to append to the changelog, and create a draft release.

This tool will undergo a few iterations, but is currently capable of repackaging chainweb-node builds

* workflow pushes ignore changes to .github directory

* trigger a (failed) run to make gh recognize new workflow

* mac runner

* disable macos gh run, change tags on nix run

* back to mac-m1 label

* testing mac run

* mac test

* revert testing to use our runner

* v2 of our config action

* Update nix.yml

* Update nix.yml

* Update nix.yml

* Update nix.yml

* Update nix.yml

* Update nix.yml

* Update nix.yml

* Update nix.yml

* Update nix.yml

* bump ghc ver, remove ubuntu 20

* specific ubuntu version

* re-enable normal mac builds

* Update nix.yml to latest setup-nix-with-cache

* Update nix.yml

* Update applications.yml

* release.yml dockerhub edits

Would rather do a remote workflow buuuut...

* do not ignore github dir changes

* improve docker syntax

* use ubuntu 20 with freeze file for release

* typo on version

* Update release.yml

* Update release.yml

* cwdocker local username

* fixing chainweb-docker build

* temp comment out to check dockerhub release

* Update release.yml

* Update release.yml

* update haskell setup action

* reference chainweb-node-docker workflow

* Update release.yml

* Update release.yml

* pull ghc version from applications.yml

* Update release.yml

* Update release.yml

* oops wrong position for needs value

* add -frozen tag to release image

* use frozen chainweb build

* Update release.yml

* better docker image references

* Add compact executable to ubuntu binary release (#1984)

* add compact executable to ubuntu binary release output

Change-Id: I0c5e2cc6fd519affc2444cbcc41ab4f0dd133ea0

* run tree

Change-Id: I05d7d0b76f5e9c206b4e89131f8be654cd05fe9d

* try

Change-Id: I6d9d9bdebe9b9f2d07bdebc35a0c122f8f5470fa

* try

Change-Id: I4d06008fa733f7f26528fe9e29606de48a41a41f

---------

Co-authored-by: Enis Bayramoğlu <[email protected]>
Co-authored-by: chessai <[email protected]>
  • Loading branch information
@DevopsGoth @enobayram @chessai
3 people authored Aug 19, 2024
1 parent 7b2fb3b commit 21f822f
Showing 1 changed file with 93 additions and 107 deletions.
200 changes: 93 additions & 107 deletions .github/workflows/release.yml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Release - chainweb-node - binary release scripts

## Release automation for chainweb-node
# Designed to run systems in chainweb/binary-release
# Designed to run systems in chainweb/binary-release

## Currently has manual version setting and creates draft releases;
# when we are confident in this process, it can auto-increment and publish non-drafts
Expand Down Expand Up @@ -34,11 +34,18 @@ on:
dry_run:
description: "dry run (debugging)"
type: boolean
default: false
default: false

jobs:
Finalize-release:
release_vars:
runs-on: ubuntu-latest
outputs:
RELEASE_SHA: ${{ steps.set-vars.outputs.RELEASE_SHA }}
SHORT_REVISION: ${{ steps.set-vars.outputs.SHORT_REVISION }}
VERSION_NEW: ${{ steps.set-vars.outputs.VERSION_NEW }}
VERSION_OLD: ${{ steps.set-vars.outputs.VERSION_OLD }}
IMAGE: ${{ steps.set-vars.outputs.IMAGE }}
GHC_VERSION: ${{ steps.set-ghc-version.outputs.GHC_VERSION }}
steps:
# if this plugin is bad we just override it
# - name: Bump version and push tag
Expand All @@ -49,76 +56,67 @@ jobs:
# default_bump: ${{ inputs.tag_default_bump }}
# tag_prefix: ""
# commit_sha: ${{ inputs.release_sha }}
# This is up here so we can adjust the way we handle package bumping etc
# might not want to use the above plugin/don't want to depend on its outputs
# yeah that thing is for rapid releasing tbh

- name: Set environment variables for release


- name: Set variables for release
id: set-vars
run: |
RELEASE_SHA=${{ github.sha }}
if [[ ! -z ${{ inputs.release_sha }} ]]; then
RELEASE_SHA=${{ inputs.release_sha }}
fi
echo "RELEASE_SHA=$RELEASE_SHA" >> $GITHUB_ENV
echo "RELEASE_SHA=$RELEASE_SHA" >> $GITHUB_OUTPUT
SHORT_REVISION=$(echo $RELEASE_SHA | cut -c 1-7)
echo "SHORT_REVISION=$SHORT_REVISION" >> $GITHUB_ENV
#echo "VERSION_NEW=${{ steps.tag_version.outputs.new_tag }}" >> $GITHUB_ENV
#echo "VERSION_OLD=${{ steps.tag_version.outputs.old_tag }}" >> $GITHUB_ENV
echo "VERSION_NEW=${{ inputs.new_tag }}" >> $GITHUB_ENV
echo "VERSION_OLD=${{ inputs.old_tag }}" >> $GITHUB_ENV
echo "IMAGE=ghcr.io/kadena-io/chainweb-node:sha-$SHORT_REVISION" >> $GITHUB_ENV
echo $GITHUB_ENV
echo "SHORT_REVISION=$SHORT_REVISION" >> $GITHUB_OUTPUT
# echo "VERSION_NEW=${{ steps.tag_version.outputs.new_tag }}" >> $GITHUB_OUTPUT
# echo "VERSION_OLD=${{ steps.tag_version.outputs.old_tag }}" >> $GITHUB_OUTPUT
echo "VERSION_NEW=${{ inputs.new_tag }}" >> $GITHUB_OUTPUT
echo "VERSION_OLD=${{ inputs.old_tag }}" >> $GITHUB_OUTPUT
echo "IMAGE=ghcr.io/kadena-io/chainweb-node:sha-$SHORT_REVISION-frozen" >> $GITHUB_OUTPUT
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
repository: kadena-io/chainweb-node-docker
path: dockerhub
ssh-key: ${{ secrets.GH_DEPLOYKEY_CHAINWEBDOCKER }}

#- name: derive release id from branch name?
# if this was a release branch we'd do this to override the inputs
# first run we are releasing from master (release branch kinda not needed if we just force release build logic during 'freeze')
# The release actions out there sometimes will just do this increment for us
- name: UBUNTU IMAGE bump end-user docker image
working-directory: dockerhub
continue-on-error: true
ref: ${{ steps.set-vars.outputs.RELEASE_SHA }}
sparse-checkout: |
.github/workflows/applications.yml
- name: Get ghc version
id: set-ghc-version
run: |
echo "DOCKERHUB_SHA=not yet updated" >> $GITHUB_ENV
git clone https://github.com/kadena-io/chainweb-node-docker
sed -i -e "s/ARG REVISION=.*/ARG REVISION=$SHORT_REVISION/" Dockerfile
docker build -t kadena/chainweb-node:$VERSION_NEW .
docker run --rm kadena/chainweb-node:$VERSION_NEW ./chainweb-node --version
git config --local user.name "github-actions[chainweb-node release]"
git config --local user.email "[email protected]"
git add Dockerfile
git commit -m "chainweb-node version $VERSION_NEW"
git tag chainweb-$VERSION_NEW
git push
git push origin chainweb-$VERSION_NEW
echo ${{ secrets.DOCKERHUB_PAT }} | docker login docker.io --username ${{ secrets.DOCKERHUB_USER }} --password-stdin
docker tag kadena/chainweb-node:$VERSION_NEW kadena/chainweb-node:latest
docker image push --all-tags kadena/chainweb-node
## spit these to output
digest1=$(docker inspect kadena/chainweb-node:$VERSION_NEW --format '{{ .RepoDigests }}')
digest2=$(docker inspect kadena/chainweb-node:latest --format '{{ .RepoDigests }}')
if [[ $digest1 != $digest2 ]]; then
echo "DOCKERHUB_SHA=sha mismatches in dockerhub image" >> $GITHUB_ENV
exit 1
fi
echo "DOCKERHUB_SHA=$digest1" >> $GITHUB_ENV
# all of this should really just be a BUMP IMAGE VERSION step in the other repo
# Now in this workflow, trigger the above, spit output into release notes file
VERSION=$(grep -Po '(?<="ghc": \[")(\d\.\d\.\d)' .github/workflows/applications.yml | head -1)
if [[ -z $VERSION ]]; then
echo "Unable to get version from chainweb node build"
exit 1
fi
echo "GHC_VERSION=$VERSION" >> $GITHUB_OUTPUT
chainweb-node-dockerhub:
uses: kadena-io/chainweb-node-docker/.github/workflows/dockerhub_release.yml@master
needs: release_vars
with:
SHORT_SHA: ${{ needs.release_vars.outputs.SHORT_REVISION }}
VERSION_NEW: ${{ needs.release_vars.outputs.VERSION_NEW }}
GHC_VERSION: ${{ needs.release_vars.outputs.GHC_VERSION }}
secrets: inherit

release_repo:
needs:
- release_vars
- chainweb-node-dockerhub
runs-on: ubuntu-latest
env:
IMAGE: ${{ needs.release_vars.outputs.IMAGE }}
SHORT_REVISION: ${{ needs.release_vars.outputs.SHORT_REVISION }}
VERSION_NEW: ${{ needs.release_vars.outputs.VERSION_NEW }}
VERSION_OLD: ${{ needs.release_vars.outputs.VERSION_OLD }}
RELEASE_SHA: ${{ needs.release_vars.outputs.RELEASE_SHA }}
GHC_VERSION: ${{ needs.release_vars.outputs.GHC_VERSION }}
steps:
- uses: actions/checkout@v3
with:
ref: ${{ env.RELEASE_SHA }}
sparse-checkout: 'CHANGELOG.md'

- name: tag git sha with release version
run: |
git tag ${{ env.VERSION_NEW }} ${{ env.RELEASE_SHA }}
Expand All @@ -130,26 +128,22 @@ jobs:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: retag ghcr docker images
continue-on-error: true
env:
IMAGE: ${{ env.IMAGE }}
SHORT_REVISION: ${{ env.SHORT_REVISION }}
VERSION_NEW: ${{ env.VERSION_NEW }}
continue-on-error: false
run: |
docker pull $IMAGE
docker run --rm $IMAGE --version
docker tag ghcr.io/kadena-io/chainweb-node:sha-$SHORT_REVISION ghcr.io/kadena-io/chainweb-node/ubuntu:$VERSION_NEW
docker tag ghcr.io/kadena-io/chainweb-node:sha-$SHORT_REVISION ghcr.io/kadena-io/chainweb-node/ubuntu:latest
docker tag ghcr.io/kadena-io/chainweb-node:sha-$SHORT_REVISION ghcr.io/kadena-io/chainweb-node:latest
docker tag $IMAGE ghcr.io/kadena-io/chainweb-node/ubuntu:$VERSION_NEW
docker tag $IMAGE ghcr.io/kadena-io/chainweb-node/ubuntu:latest
docker tag $IMAGE ghcr.io/kadena-io/chainweb-node:latest
docker image ls
docker image push --all-tags ghcr.io/kadena-io/chainweb-node/ubuntu
docker image push --all-tags ghcr.io/kadena-io/chainweb-node
digest1=$(docker inspect ghcr.io/kadena-io/chainweb-node:sha-$SHORT_REVISION --format '{{ .RepoDigests }}')
digest1=$(docker inspect $IMAGE --format '{{ .RepoDigests }}')
digest2=$(docker inspect ghcr.io/kadena-io/chainweb-node/ubuntu:$VERSION_NEW --format '{{ .RepoDigests }}')
digest3=$(docker inspect ghcr.io/kadena-io/chainweb-node/ubuntu:latest --format '{{ .RepoDigests }}')
if [[ "$digest1" != "$digest2" || "$digest2" != "$digest3" ]]; then
Expand All @@ -160,34 +154,30 @@ jobs:
- name: run release script to generate images
id: repack-binaries
env:
SHORT_REVISION: ${{ env.SHORT_REVISION }}
VERSION_NEW: ${{ env.VERSION_NEW }}
run: |
GHC_VER=9.6.5
GHC_VER=${{ env.GHC_VERSION }}
UBUNTU_20_TAR=chainweb.true.$GHC_VER.ubuntu-20.04.$SHORT_REVISION.tar.gz
UBUNTU_22_TAR=chainweb.true.$GHC_VER.ubuntu-22.04.$SHORT_REVISION.tar.gz
UBUNTU_20_TAR=chainweb.false.$GHC_VER.ubuntu-20.04.$SHORT_REVISION.tar.gz
UBUNTU_22_TAR=chainweb.false.$GHC_VER.ubuntu-22.04.$SHORT_REVISION.tar.gz
UBUNTU_20_NODE=chainweb-$VERSION_NEW.ghc-$GHC_VER.ubuntu-20.04.$SHORT_REVISION.tar.gz
UBUNTU_22_NODE=chainweb-$VERSION_NEW.ghc-$GHC_VER.ubuntu-22.04.$SHORT_REVISION.tar.gz
echo $UBUNTU_22_TAR
echo $UBUNTU_22_NODE
curl "s3.us-east-1.amazonaws.com/kadena-cabal-cache/chainweb-node/$UBUNTU_20_TAR" -o "./$UBUNTU_20_TAR"
curl "s3.us-east-1.amazonaws.com/kadena-cabal-cache/chainweb-node/$UBUNTU_22_TAR" -o "./$UBUNTU_22_TAR"
tar xvfz "$UBUNTU_20_TAR" chainweb/chainweb-node
mv chainweb/chainweb-node .
tar cvfz "$UBUNTU_20_NODE" chainweb-node
rm -fr chainweb-node chainweb
tar xvfz "$UBUNTU_22_TAR" chainweb/chainweb-node
mv chainweb/chainweb-node .
tar cvfz "$UBUNTU_22_NODE" chainweb-node
tar xvfz "$UBUNTU_20_TAR" chainweb
mv chainweb/{LICENSE,README.md,chainweb-node,compact} .
tar cvfz "$UBUNTU_20_NODE" LICENSE README.md chainweb-node compact
rm -fr chainweb-node chainweb
tar xvfz "$UBUNTU_22_TAR" chainweb
mv chainweb/{LICENSE,README.md,chainweb-node,compact} .
tar cvfz "$UBUNTU_22_NODE" LICENSE README.md chainweb-node compact
rm -fr chainweb-node chainweb
echo "UBUNTU_20_SHA=$(shasum -a 256 $UBUNTU_20_NODE)" >> $GITHUB_ENV
echo "UBUNTU_22_SHA=$(shasum -a 256 $UBUNTU_22_NODE)" >> $GITHUB_ENV
shasum -a 256 "$UBUNTU_22_NODE"
Expand All @@ -196,51 +186,47 @@ jobs:
#- name: generate release message
# could generate this from the changelog.md, though changelog.md could be itself generated...
#oom_score_adj:
#oom_score_adj:

- name: Generate release message
continue-on-error: true
env:
UBUNTU_22_NODE: ${{ env.UBUNTU_22_NODE }}
UBUNTU_20_NODE: ${{ env.UBUNTU_20_NODE }}
IMAGE: ${{ env.IMAGE }}
SHORT_REVISION: ${{ env.SHORT_REVISION }}
VERSION_NEW: ${{ env.VERSION_NEW }}
VERSION_OLD: ${{ env.VERSION_OLD }}
GHCR_DIGEST: ${{ env.GHCR_DIGEST }}
run: |
pwd
ls
sed "/$VERSION_OLD\s/,\$d" CHANGELOG.md | sed "/$VERSION_NEW (20/d" | sed '/\sChangelog/d' | tee CHANGELOG.md
echo 'Official release: https://github.com/kadena-io/chainweb-node/releases/tag/${{ env.VERSION_NEW }}
For full details refer to the [Changelog](https://github.com/kadena-io/chainweb-node/blob/master/CHANGELOG.md)
### Container Images:
* [End-user image](https://hub.docker.com/r/kadena/chainweb-node)
Image Digest: `${{ env.DOCKERHUB_SHA }}`
Image Digest: `${{ needs.chainweb-node-dockerhub.outputs.DOCKERHUB_SHA }}`
```sh
docker pull kadena/chainweb-node:${{ env.VERSION_NEW }}
```
* [Binary-only Ubuntu-20.04 ](ghcr.io/kadena-io/chainweb-node/ubuntu:${{ env.VERSION_NEW }})
Image Digest: `${{ env.GHCR_DIGEST }}`
```sh
docker pull ghcr.io/kadena-io/chainweb-node/ubuntu:${{ env.VERSION_NEW }}
```
### Ubuntu Binaries:
SHA256 Hashes
```
${{ env.UBUNTU_22_SHA }}
${{ env.UBUNTU_20_SHA }}
```
The following dependencies must be installed on the system:
* ubuntu-22.04:
```sh
apt-get install ca-certificates libgmp10 libssl3 libsnappy1v5 zlib1g liblz4-1 libbz2-1.0 libgflags2.2 zstd
Expand All @@ -251,7 +237,7 @@ jobs:
```
' >> CHANGELOG.md
cat CHANGELOG.md
- name: Release
uses: ncipollo/release-action@v1
#if: startsWith(github.ref, 'refs/tags/')
Expand Down

0 comments on commit 21f822f

Please sign in to comment.