[helm] force 2 procs to JVM when cpu.requests <= 1000

[anshul98ks123]

Issue(s)

k8s - force 2 procs to JVM when cpu.requests <= 1000

Description

For small or lightly used environments, we'd like to have cpu.requests<=1000m.
Problem when we do this is java thinks there is only one proc and this hurts the parallel computation capabilities, especially in the simple create flow

Java behaviour based on cpu.requests

• Up to 1000m: 1 proc
• 1001-2000m: 2 procs
• 2001-3000m: 3 procs

When cpu.requests is smaller than or equal to 1000m, we should pass the argument
-XX:ActiveProcessorCount=2

This PR adds the logic to dynamically pass this argument in JAVA_OPTS of coordinator / scheduler / worker deployments based on the cpu request count

Testing

Tested via helm install dry run with different values for cpu requests of coordinator, scheduler and workers
helm install startree-thirdeye --dry-run --debug --generate-name --set coordinator.resources.requests.cpu=<coordinator-cpu> --set scheduler.resources.requests.cpu=<scheduler-cpu> --set worker.resources.requests.cpu=<worker-cpu>

Coordinator cpu req	Scheduler cpu req	Worker cpu req	output
0.99	1.00	1.01	decimal-val-output-1.txt
1.00	1.01	0.99	decimal-val-output-2.txt
1.01	0.99	1.00	decimal-val-output-3.txt
999m	1000m	1001m	millicore-val-output-1.txt
1000m	1001m	999m	millicore-val-output-2.txt
1001m	999m	1000m	millicore-val-output-3.txt

Snippets from helm output:

Cases when cpu requests <= 1.0 / 1000m

          - name: JAVA_OPTS
            value: >
              -XX:ActiveProcessorCount=2
              -Djavax.net.ssl.trustStore=/opt/thirdeye/server/truststore/truststore
              -Djavax.net.ssl.trustStorePassword=changeit -Xmx1G -XX:+UseG1GC

Cases when cpu requests > 1.0 / 1000m

          - name: JAVA_OPTS
            value: >
              -Djavax.net.ssl.trustStore=/opt/thirdeye/server/truststore/truststore
              -Djavax.net.ssl.trustStorePassword=changeit -Xmx1G -XX:+UseG1GC

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
thirdeye	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 28, 2024 10:44am

[cyrilou242]