✨ Computational backend: set container limits as labels ⚠️ (devops checks on grafana dashboards!) 🚨

[sanderegg]

🚨 check when releasing to staging and prod

• legacy dynamic services need to be shutdown
• ask ANE to have a look after the release and point him to this PR

What do these changes do?

• add container cpu limits and memory limits on container (both for dynamic-sidecar services and computational services)
• uniformize labels for dynamic-sidecar services + computational services
• all labels used now are prefixed with io.simcore.runtime. (as in io.simcore.runtime.node-id or io.simcore.runtime.project-id) to properly differentiate from simcore image labels which are static
• backwards compatible with old nomenclature (which should be removed once this PR reaches production + 1)
• simplified dynamic-sidecar labels by cleaning up unused or misused ones
• ⚠️: possibly some grafana dashboards might need to be adapted. (i.e. simcore_user_agent changed to io.simcore.runtime.simcore-user-agent)
• ⚠️: any running legacy dynamic service (i.e. running via director-v0) are NOT backwards compatible. I did not take care of them as almost no one is using them for production work.

NOTE that underscores are not considered as valid docker label keys!

legacy services now:

computational service now:

dynamic-services now:

show the same labels in service and container labels.
NOTE: io.simcore.scheduler-data is still the same. I did not dare touch this now.
NOTE2: docker network/volumes labels are not changed. I will let @GitHK take care of this or when I get again an incentive to touch this.

Related issue/s

• resolves Maintenance: adjust the labels set on all services to follow inversed url notation #3638

How to test

DevOps Checklist

[codecov]

Codecov Report

Merging #4453 (1396d73) into master (bef80bb) will decrease coverage by 0.1%.
The diff coverage is 91.3%.

@@           Coverage Diff            @@
##           master   #4453     +/-   ##
========================================
- Coverage    86.7%   86.6%   -0.1%     
========================================
  Files         867     685    -182     
  Lines       39204   34142   -5062     
  Branches      745     467    -278     
========================================
- Hits        34006   29583   -4423     
+ Misses       5022    4448    -574     
+ Partials      176     111     -65     

Flag	Coverage Δ
integrationtests	78.1% <84.5%> (+10.0%)	⬆️
unittests	84.9% <87.0%> (+0.8%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
..._v2/modules/dynamic_sidecar/docker_api/__init__.py	100.0% <ø> (ø)
...modules/dynamic_sidecar/scheduler/_core/_events.py	96.7% <50.0%> (-0.1%)	⬇️
...s/dynamic_sidecar/docker_service_specs/settings.py	81.8% <78.5%> (+0.5%)	⬆️
...tor_v2/modules/dynamic_sidecar/docker_api/_core.py	95.6% <81.3%> (-0.5%)	⬇️
.../director/src/simcore_service_director/producer.py	67.4% <87.5%> (+0.2%)	⬆️
...ckages/models-library/src/models_library/docker.py	90.7% <97.2%> (+4.5%)	⬆️
...rary/src/models_library/service_settings_labels.py	97.6% <100.0%> (+0.1%)	⬆️
.../src/simcore_service_autoscaling/utils/rabbitmq.py	100.0% <100.0%> (ø)
...ctor_v2/models/schemas/dynamic_services/service.py	100.0% <100.0%> (ø)
...simcore_service_director_v2/modules/dask_client.py	92.9% <100.0%> (ø)
... and 5 more

... and 285 files with indirect coverage changes

[pcrespov]

👍

[mrnicegyu11]

very promissing, and very good for cleanup pruposes on the labels! thanks a lot!

I have only added minors and some questions for my learndings.

Please add a big verbose devops warning to the title of the PR etc. , for sure we have to modify some PromQl in the grafan dashboards, and I will prepare an ops-commit once this is in master (and I see what breaks) that shall propagate with this PR

[GitHK]

So it's looking good. I have expressed some concerns below.
Also, did you test that service composed of multiple containers like s4l work as expected?

[codeclimate]

Code Climate has analyzed commit 1396d73 and detected 0 issues on this pull request.

View more on Code Climate.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[sanderegg]

So it's looking good. I have expressed some concerns below. Also, did you test that service composed of multiple containers like s4l work as expected?

I tried iSeg