Skip to content

Commit

Permalink
Allow cleaning up openQA devel packages from the zypper cache directory
Browse files Browse the repository at this point in the history
* Add a script `openqa-clean-repo-cache` that allows to clean RPM cache
  repositories keeping a certain number of packages per version; this is
  similar to the `paccache` script from Arch Linux
* Allow running this script as part of `openqa-auto-update`
* See https://progress.opensuse.org/issues/174313
  • Loading branch information
Martchus committed Jan 10, 2025
1 parent 5b07ee4 commit 888485f
Show file tree
Hide file tree
Showing 4 changed files with 109 additions and 1 deletion.
1 change: 1 addition & 0 deletions dist/rpm/openQA.spec
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,7 @@ fi
%{_localstatedir}/lib/openqa/script
%{_localstatedir}/lib/openqa/tests
%{_datadir}/openqa/script/openqa-check-devel-repo
%{_datadir}/openqa/script/openqa-clean-repo-cache
%{_unitdir}/openqa-minion-restart.service
%{_unitdir}/openqa-minion-restart.path

Expand Down
4 changes: 3 additions & 1 deletion docs/Installing.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -1234,7 +1234,9 @@ This triggers a nightly system upgrade which first looks into configured openQA
repositories for stable packages, then conducts the upgrade and schedules
reboots during the configured reboot maintenance windows using `rebootmgr`.
As an alternative to the systemd timer the script
`/usr/share/openqa/script/openqa-auto-update` can be called when desired.
`/usr/share/openqa/script/openqa-auto-update` can be called when desired. The
script also supports cache cleanup preserving a certain number of versions per
package. Check its helptext for details.

The distribution package `openQA-continuous-update` can be used to continuously
upgrade the system. It will frequently check whether `devel:openQA` contains
Expand Down
24 changes: 24 additions & 0 deletions script/openqa-auto-update
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,29 @@
set -e
set -o pipefail

keep=${OPENQA_PACKAGE_CACHE_RETENTION_KEEP:-10}
min_mtime=${OPENQA_PACKAGE_CACHE_RETENTION_MIN_MTIME:-100}
glob=${OPENQA_PACKAGE_CACHE_REPO_GLOB:-*devel*openQA*}
usage() {
cat << EOF
Usage: openqa-auto-update
Trigger automatic system upgrade and reboot if devel:openQA packages are stable
Set OPENQA_PACKAGE_CACHE_RETENTION to also clean the zypper cache from packages
of the openQA development repository. This is useful in conjunction with the
zypper repository setting "keeppackages=1" to allow for a more fine-tuned
descicion on what packages to keep. This cleanup is influenced by further
environment variables:
ZYPPER_PACKAGES_CACHE_DIR: the zypper cache dir in case a non-standard
location is used
OPENQA_PACKAGE_CACHE_RETENTION_KEEP: the number of versions of the same
package to keep (default: $keep)
OPENQA_PACKAGE_CACHE_RETENTION_MIN_MTIME: the minimum age of packages in days
to be considered (default: $min_mtime)
OPENQA_PACKAGE_CACHE_REPO_GLOB: the glob to find relevant packages
(default: $glob)
Checkout "openqa-clean-repo-cache --help" for details.
Options:
-h, --help display this help
EOF
Expand All @@ -30,5 +48,11 @@ done
# call ref independently of dup to avoid unintended mass-removals in case ref fails (see poo#150845)
zypper -n ref
zypper -n --no-refresh dup --replacefiles --auto-agree-with-licenses --download-in-advance

if [[ $OPENQA_PACKAGE_CACHE_RETENTION ]]; then
echo 'Cleaning repository cache'
"$(dirname "${BASH_SOURCE[0]}")"/openqa-clean-repo-cache --remove --keep "$keep" --min-mtime "$min_mtime" --glob "$glob"
fi

# shellcheck disable=SC2015
needs-restarting --reboothint > /dev/null || (command -v rebootmgrctl > /dev/null && rebootmgrctl reboot || :)
81 changes: 81 additions & 0 deletions script/openqa-clean-repo-cache
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
#!/bin/bash
set -e
set -o pipefail

# define defaults for CLI arguments
dryrun='' remove='' keep=3 min_mtime='' glob=''
cache_dir=${ZYPPER_PACKAGES_CACHE_DIR:-/var/cache/zypp/packages}

# parse CLI arguments
usage() {
cat << EOF
Cleans packages from the zypper cache directory.
This script can be automatically invoked via openqa-auto-update by setting the
environment variable OPENQA_PACKAGE_CACHE_RETENTION.
Usage: openqa-clean-repo-cache <operation> [options]
Operations:
-d, --dryrun display what files are considered/kept/removed
-r, --remove remove files matching the criteria specified via options
-h, --help display this help
Options:
-h, --help display this help
-k, --keep <num> the number of versions to keep per package (default: $keep)
--min-mtime <days> keep packages with an mtime that is not older than the
specified number of days, even if this means keeping more
than specified through the '--keep' option
-c, --cache-dir <path> specifies the zypper cache directory
(default: $cache_dir)
-g, --glob <pattern> specifies the glob used to filter relevant repos/packages
EOF
exit "$1"
}
opts=$(getopt -o drhk:c:g: --long dryrun,remove,help,keep:,min-mtime:,cache-dir:,glob: -n "$0" -- "$@") || usage 1
eval set -- "$opts"
while true; do
case "$1" in
-d | --dryrun) dryrun=1; shift ;;
-r | --remove) remove=1; shift ;;
-h | --help) usage 0 ;;
-k | --keep) keep=$2; shift 2 ;;
--min-mtime) min_mtime=$2; shift 2 ;;
-c | --cache-dir) cache_dir=$2; shift 2 ;;
-g | --glob) glob=$2; shift 2 ;;
--) shift; break ;;
*) shift; break ;;
esac
done

# make commands for on specified operation
[[ $dryrun && $remove ]] && echo 'Specify either --dryrun OR --remove.' && exit 1
if [[ $dryrun ]]; then
remove_cmd=(echo 'remove') keep_cmd=(echo 'keep ')
elif [[ $remove ]]; then
remove_cmd=(rm --verbose --force) keep_cmd=()
else
echo 'No operation specified.' && exit 1
fi

# find relevant packages, sort them so newest are first
IFS=$'\n'
find_cmd=(find "$cache_dir" -type f -name '*.rpm')
[[ $glob ]] && find_cmd+=(-ipath "$glob")
[[ $min_mtime ]] && find_cmd+=(-mtime "+$min_mtime")
# shellcheck disable=SC2207
package_files=($("${find_cmd[@]}" | sort --reverse --version-sort))
previous_package_name='' package_count=0

# run commands for relevant packages considering specified number of versions to keep
for package_file in "${package_files[@]}"; do
package_name=$(rpm -q --qf "%{NAME}\n" "$package_file")
[[ $package_name != "$previous_package_name" ]] && previous_package_name=$package_name package_count=0
package_count=$((package_count + 1))
if [[ $package_count -gt "$keep" ]]; then
"${remove_cmd[@]}" "$package_file"
elif [[ ${#keep_cmd[@]} -gt 0 ]]; then
"${keep_cmd[@]}" "$package_file"
fi
done

0 comments on commit 888485f

Please sign in to comment.