Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/processor output individual records for gratia, add CPU usage and memory metrics #54

Conversation

mwestphall
Copy link
Collaborator

@mwestphall mwestphall commented May 24, 2024

  • Add a CPU usage metric based on container_cpu_usage_seconds_total.
    • NB: on old clusters using cgroups v1 this double counts results
  • Add a new memory calculation using the kube_pod_container_resource_requests{resource="memory"} metric, similar to cores.
  • Update KAPEL.py to optionally output individual APEL job records if gratia is being used for output. This aligns better with the expected GRACC record format
  • Add a simple Dockerfile for building the processor InitContainer

@rptaylor The commit history is a bit messy for this change since I separated it out from the helm chart changes in-post, let me know if you'd like it condensed down with a rebase.

@mwestphall mwestphall requested review from brianhlin and rptaylor May 24, 2024 16:39
README.md Outdated Show resolved Hide resolved
python/KAPEL.py Outdated Show resolved Hide resolved
python/KAPEL.py Outdated Show resolved Hide resolved
python/KAPEL.py Outdated Show resolved Hide resolved
@mwestphall mwestphall force-pushed the feature/processor-output-individual-records-for-gratia branch from a0e56e7 to fc4da2a Compare June 13, 2024 16:47
python/KAPEL.py Outdated Show resolved Hide resolved
Copy link
Owner

@rptaylor rptaylor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rptaylor rptaylor changed the title Feature/processor output individual records for gratia Feature/processor output individual records for gratia, add CPU usage and memory metrics Jun 21, 2024
@rptaylor rptaylor merged commit 8938f18 into rptaylor:master Jun 21, 2024
@rptaylor
Copy link
Owner

rptaylor commented Jul 5, 2024

Actually for the record I believe the cgroup-related differences we saw are due to using EL8 where the default is still cgroup v1, not because of the k8s version per se.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants