fix(perf-test): restore reachable backends, update node logic, and improve observability setup #262
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added back reachable backends in the service graph to fix failing tests. - Updated node count logic to handle resource requests for 1000 services with 2 instances each. The old logic didn't provide enough nodes. Signed-off-by: Bart Smykla <[email protected]>
bartsmykla
requested review from
Automaat,
jakubdyszkiewicz and
lobkovilya
as code owners
should this be investigated in kuma? |
lukidzi
approved these changes
Jan 28, 2025
Added a new "observability" node group in EKS to keep Prometheus and other monitoring tools separate from other workloads. This helps ensure Prometheus has enough resources, especially when monitoring many services. Updated Prometheus setup to: - Use a 80GB PersistentVolumeClaim to avoid running out of space when monitoring large workloads. - Add tolerations and nodeSelector to make sure observability components run on the right node group. Increased timeout in a test from 60s to 600s, as generating certificates for 2000 services takes significantly more time. Signed-off-by: Bart Smykla <[email protected]>
bartsmykla
force-pushed
the
fix/configure-reachable-backends
branch
from
ce73ed2 to
53bdca4
Compare
This logic is not used, but was helpful when I was making sure locally that there is no huge difference between reachable services with legacy `kuma.io/service` labels and reachable backends with `MeshServices` Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Signed-off-by: Bart Smykla <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes issues with performance tests broken in #224
Changes: