Global target pass-down to the Checks #50

puffitos · 2023-12-18T18:30:20Z

⚠️
⚠️ PLEASE REVIEW AFTER #49 IS MERGED.
⚠️

Motivation

Part of #44

Changes

Added Sparrow function to edit the checks' target configuration before reconciling the checks.
Renamed name to sparrowName because I think it was conflicting with the NAME variable in Linux

For additional information look at the commits.

Tests done

Unittests
E2E test with remote configuration

E2E test with remote configuration

I've let the sparrow read the global targets from our gitlab repository. One healthy target (sparrow-dev) was added to the list for the healthcheck and the latency Checks, as expected. Of course, the check failed, because of DNS issues (hint: it's always DNS). The prometheus metrics were recorded:

# HELP sparrow_health_up Health of targets
# TYPE sparrow_health_up gauge
sparrow_health_up{target="https://caas-max-sparrow.caas-t02.telekom.de/checks/health"} 1
sparrow_health_up{target="https://gitlab.devops.telekom.de"} 1
sparrow_health_up{target="https://sparrow-dev-cool.de"} 0
sparrow_health_up{target="https://www.google.com/"} 1
# HELP sparrow_latency_count Count of latency checks done
# TYPE sparrow_latency_count counter
sparrow_latency_count{target="https://example.com/"} 46
sparrow_latency_count{target="https://gitlab.devops.telekom.de"} 45
sparrow_latency_count{target="https://google.com/"} 45
sparrow_latency_count{target="https://sparrow-dev-cool.de"} 40
sparrow_latency_count{target="https://yam.telekom.de"} 45
# HELP sparrow_latency_duration Latency of targets in seconds
# TYPE sparrow_latency_duration histogram
sparrow_latency_duration_bucket{target="https://example.com/",le="0.005"} 1
sparrow_latency_duration_bucket{target="https://example.com/",le="0.01"} 1
sparrow_latency_duration_bucket{target="https://example.com/",le="0.025"} 1
sparrow_latency_duration_bucket{target="https://example.com/",le="0.05"} 1
sparrow_latency_duration_bucket{target="https://example.com/",le="0.1"} 1
sparrow_latency_duration_bucket{target="https://example.com/",le="0.25"} 40
sparrow_latency_duration_bucket{target="https://example.com/",le="0.5"} 42
sparrow_latency_duration_bucket{target="https://example.com/",le="1"} 44
sparrow_latency_duration_bucket{target="https://example.com/",le="2.5"} 44
sparrow_latency_duration_bucket{target="https://example.com/",le="5"} 45
sparrow_latency_duration_bucket{target="https://example.com/",le="10"} 45
sparrow_latency_duration_bucket{target="https://example.com/",le="+Inf"} 46
sparrow_latency_duration_sum{target="https://example.com/"} 31.556413448999997
sparrow_latency_duration_count{target="https://example.com/"} 46
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.005"} 4
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.01"} 4
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.025"} 4
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.05"} 4
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.1"} 4
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.25"} 4
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="0.5"} 45
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="1"} 45
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="2.5"} 45
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="5"} 45
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="10"} 45
sparrow_latency_duration_bucket{target="https://gitlab.devops.telekom.de",le="+Inf"} 45
sparrow_latency_duration_sum{target="https://gitlab.devops.telekom.de"} 13.747880463000003
sparrow_latency_duration_count{target="https://gitlab.devops.telekom.de"} 45
sparrow_latency_duration_bucket{target="https://google.com/",le="0.005"} 2
sparrow_latency_duration_bucket{target="https://google.com/",le="0.01"} 2
sparrow_latency_duration_bucket{target="https://google.com/",le="0.025"} 2
sparrow_latency_duration_bucket{target="https://google.com/",le="0.05"} 2
sparrow_latency_duration_bucket{target="https://google.com/",le="0.1"} 2
sparrow_latency_duration_bucket{target="https://google.com/",le="0.25"} 39
sparrow_latency_duration_bucket{target="https://google.com/",le="0.5"} 43
sparrow_latency_duration_bucket{target="https://google.com/",le="1"} 43
sparrow_latency_duration_bucket{target="https://google.com/",le="2.5"} 44
sparrow_latency_duration_bucket{target="https://google.com/",le="5"} 45
sparrow_latency_duration_bucket{target="https://google.com/",le="10"} 45
sparrow_latency_duration_bucket{target="https://google.com/",le="+Inf"} 45
sparrow_latency_duration_sum{target="https://google.com/"} 10.196599077000002
sparrow_latency_duration_count{target="https://google.com/"} 45
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.005"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.01"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.025"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.05"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.1"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.25"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="0.5"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="1"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="2.5"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="5"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="10"} 40
sparrow_latency_duration_bucket{target="https://sparrow-dev-cool.de",le="+Inf"} 40
sparrow_latency_duration_sum{target="https://sparrow-dev-cool.de"} 0
sparrow_latency_duration_count{target="https://sparrow-dev-cool.de"} 40
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.005"} 4
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.01"} 4
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.025"} 4
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.05"} 4
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.1"} 4
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.25"} 41
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="0.5"} 45
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="1"} 45
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="2.5"} 45
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="5"} 45
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="10"} 45
sparrow_latency_duration_bucket{target="https://yam.telekom.de",le="+Inf"} 45
sparrow_latency_duration_sum{target="https://yam.telekom.de"} 8.786487958000002
sparrow_latency_duration_count{target="https://yam.telekom.de"} 45
# HELP sparrow_latency_duration_seconds Latency with status information of targets
# TYPE sparrow_latency_duration_seconds gauge
sparrow_latency_duration_seconds{status="0",target="https://example.com/"} 0
sparrow_latency_duration_seconds{status="0",target="https://gitlab.devops.telekom.de"} 0
sparrow_latency_duration_seconds{status="0",target="https://google.com/"} 0
sparrow_latency_duration_seconds{status="0",target="https://sparrow-dev-cool.de"} 0
sparrow_latency_duration_seconds{status="0",target="https://yam.telekom.de"} 0
sparrow_latency_duration_seconds{status="200",target="https://example.com/"} 0.121570443
sparrow_latency_duration_seconds{status="200",target="https://gitlab.devops.telekom.de"} 0.315281729
sparrow_latency_duration_seconds{status="200",target="https://google.com/"} 0.332088165
sparrow_latency_duration_seconds{status="418",target="https://yam.telekom.de"} 0.410459357

And logs with the failures for the new taget were seen:

{"time":"2023-12-18T19:22:25.008569138+01:00","level":"ERROR","msg":"Error while checking latency","sparrow":{"latency":{"url":"https://sparrow-dev-cool.de","error":"Get \"https://sparrow-dev-cool.de\": notresolvable"}}}
{"time":"2023-12-18T19:22:26.362193547+01:00","level":"ERROR","msg":"Error while checking latency","sparrow":{"latency":{"url":"https://sparrow-dev-cool.de","error":"Get \"https://sparrow-dev-cool.de\": notresolvable"}}}
{"time":"2023-12-18T19:22:26.422631052+01:00","level":"ERROR","msg":"Http get request failed","sparrow":{"health":{"url":"https://sparrow-dev-cool.de","error":"Get \"https://sparrow-dev-cool.de\": notresolvable"}}}
{"time":"2023-12-18T19:22:26.422740367+01:00","level":"INFO","msg":"Successfully finished health check run"}
{"time":"2023-12-18T19:22:26.422777216+01:00","level":"INFO","msg":"Next health check will run after delay","sparrow":{"health":{"delay":"15s"}}}
{"time":"2023-12-18T19:22:27.389961991+01:00","level":"INFO","msg":"Successfully got remote runtime configuration"}
{"time":"2023-12-18T19:22:27.683460986+01:00","level":"ERROR","msg":"Error while checking latency","sparrow":{"latency":{"url":"https://sparrow-dev-cool.de","error":"Get \"https://sparrow-dev-cool.de\": notresolvable"}}}

TODO

I've assigned this PR to myself
I've labeled this PR correctly

Signed-off-by: Bruno Bressi <[email protected]>

pkg/sparrow/run.go

Signed-off-by: Bruno Bressi <[email protected]>

y-eight

Looks good for now, thx

puffitos added 5 commits December 18, 2023 17:55

feat: merge global targets with check targets

386b8c9

Signed-off-by: Bruno Bressi <[email protected]>

Merge branch 'feat/dns-name' into feat/target-passdown

607973b

test: tested new function

fec272b

Signed-off-by: Bruno Bressi <[email protected]>

chore: added set name to cfg loader

4114bbe

Signed-off-by: Bruno Bressi <[email protected]>

fix: fixed interface casting issues

a64099f

Signed-off-by: Bruno Bressi <[email protected]>

puffitos added the request/internal Indicates an internal feature request label Dec 18, 2023

puffitos self-assigned this Dec 18, 2023

puffitos requested review from y-eight, niklastreml, lvlcn-t and eumel8 as code owners December 18, 2023 18:30

puffitos added 2 commits December 18, 2023 19:37

chore: merged from dns-name branch

fc3f5d4

Signed-off-by: Bruno Bressi <[email protected]>

Merge remote-tracking branch 'origin/main' into feat/target-passdown

709ceb6

niklastreml requested changes Dec 19, 2023

View reviewed changes

pkg/sparrow/run.go Show resolved Hide resolved

pkg/sparrow/run.go Show resolved Hide resolved

chore: catch emptycfg slice and non-string targets

33c2363

Signed-off-by: Bruno Bressi <[email protected]>

y-eight approved these changes Dec 19, 2023

View reviewed changes

puffitos requested a review from niklastreml December 19, 2023 17:14

puffitos mentioned this pull request Dec 20, 2023

feat/target-manager-shutdown #51

Merged

4 tasks

niklastreml approved these changes Dec 20, 2023

View reviewed changes

puffitos merged commit 73b2382 into main Dec 20, 2023
7 checks passed

lvlcn-t deleted the feat/target-passdown branch December 21, 2023 14:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Global target pass-down to the Checks #50

Global target pass-down to the Checks #50

puffitos commented Dec 18, 2023 •

edited

Loading

y-eight left a comment

Global target pass-down to the Checks #50

Global target pass-down to the Checks #50

Conversation

puffitos commented Dec 18, 2023 • edited Loading

Motivation

Changes

Tests done

E2E test with remote configuration

TODO

y-eight left a comment

Choose a reason for hiding this comment

puffitos commented Dec 18, 2023 •

edited

Loading