Skip to content

Commit

Permalink
test/cluster: unflake test-compute-controller-metrics
Browse files Browse the repository at this point in the history
The compute controller now only refreshed its metrics once per second,
which makes it more likely that test-compute-controller-metrics, which
runs some SQL and then checks for that to be reflected in the metrics,
gets to see outdated values.

The only way we can solve this is by sleeping before querying the
metrics. Given the 1s refresh interval, sleeping for 2s should be
enough.
  • Loading branch information
teskje committed Feb 14, 2024
1 parent e160cb5 commit 52bdd72
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions test/cluster/mzcompose.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,10 @@ def workflow_default(c: Composition, parser: WorkflowArgumentParser) -> None:
for i, name in enumerate(c.workflows):
# incident-70 requires more memory, runs in separate CI step
# concurrent-connections is too flaky
# TODO: Reenable test-compute-controller-metrics when #25214 is fixed
if name in (
"default",
"test-incident-70",
"test-concurrent-connections",
"test-compute-controller-metrics",
):
continue
if buildkite.accepted_by_shard(i):
Expand Down Expand Up @@ -2427,6 +2425,9 @@ def fetch_metrics() -> Metrics:
index_id = c.sql_query("SELECT id FROM mz_indexes WHERE name = 'idx'")[0][0]
mv_id = c.sql_query("SELECT id FROM mz_materialized_views WHERE name = 'mv'")[0][0]

# Wait a bit to let the controller refresh its metrics.
time.sleep(2)

# Check that expected metrics exist and have sensible values.
metrics = fetch_metrics()

Expand Down

0 comments on commit 52bdd72

Please sign in to comment.