Skip to content

Commit

Permalink
[CI] Automatically garbage collect old docker images (GPUOpen-Drivers…
Browse files Browse the repository at this point in the history
…#1885)

Run the gc script in the nightly AMDVLK wokflow, but only in the
'+clang' configuration. This is so that we do not have multiple workers
trying to delete the same images. We also expect the '+clang'
configuration to be the most robust so that it gets to the end of the
workflow consistently.

Update the gc script to be compatible with python 3.8, which is
installed by default on ubuntu 20.04.
  • Loading branch information
kuhar authored Jul 14, 2022
1 parent 0115f36 commit d7999f0
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 6 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/build-amdvlk-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,7 @@ jobs:
- name: Push the new image
run: |
docker push "$IMAGE_TAG"
- name: Garbage collect old docker images
if: matrix.feature-set == '+clang'
run: |
python3 script/gc-amdvlk-docker-images.py
12 changes: 6 additions & 6 deletions script/gc-amdvlk-docker-images.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#! /usr/bin/env python3
"""Script to garbage collect old amdvlk docker images created by the public CI on GitHub.
Requires python 3.9 or later.
Requires python 3.8 or later.
"""

import argparse
Expand All @@ -11,9 +11,9 @@
import sys

from collections import defaultdict
from typing import Any, Optional
from typing import Any, Dict, List, Optional, Tuple

def _run_cmd(cmd: list[str]) -> tuple[bool, str]:
def _run_cmd(cmd: List[str]) -> Tuple[bool, str]:
"""
Runs a shell command capturing its output.
Expand All @@ -31,7 +31,7 @@ def _run_cmd(cmd: list[str]) -> tuple[bool, str]:
return True, result.stdout


def query_images(artifact_repository_url: str) -> Optional[list[dict[str, Any]]]:
def query_images(artifact_repository_url: str) -> Optional[List[Dict[str, Any]]]:
"""
Returns a list of JSON objects representing docker images found under
|artifact_repository_url|, or None on error.
Expand All @@ -52,7 +52,7 @@ def query_images(artifact_repository_url: str) -> Optional[list[dict[str, Any]]]
return list(json.loads(text))


def find_images_to_gc(images: list[dict[str, Any]], num_last_to_keep) -> list[dict[str, Any]]:
def find_images_to_gc(images: List[Dict[str, Any]], num_last_to_keep) -> List[Dict[str, Any]]:
"""
Returns a subset of |images| that should be garbage collected. Preserves tagged
images and also the most recent |num_last_to_keep| for each package.
Expand All @@ -72,7 +72,7 @@ def find_images_to_gc(images: list[dict[str, Any]], num_last_to_keep) -> list[di
return to_gc


def delete_images(images: list[dict[str, Any]], dry_run: bool) -> None:
def delete_images(images: List[Dict[str, Any]], dry_run: bool) -> None:
"""
Deletes all |images| from the repository. When |dry_run| is True, synthesizes the delete
commands and logs but does not execute them.
Expand Down

0 comments on commit d7999f0

Please sign in to comment.