Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable per-alert Incident Severity #1701

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

igorhrcek
Copy link
Contributor

Changes

This PR improves on existing Incident.io implementation and allows to define per-alert incident severity in Incident.io (until now severity was determined by the alert severity, however not all alerts with the critical severity should have the same Incident.io severity).

The following configurational changes were made:

  1. severity_alert_label_name - label on the alert that will contain information about Incident.io severity. Default set to robusta_incidentio_severity.
  2. severity_default - sane default in case when label is not defined
  3. dashboard_url_annotation_name - Incident.io has something called the source_url. Usually, that can be a link toward Grafana dashboard or anything else. In this case, it is a name of the alert annotation. Default set to dashboard_url.
  4. runbook_url_annotation_name - same as with the Dashboard URL, stored as metadata information that can be extracted and parsed in Incident.io interface.

Please note that severity names must be mapped in the Incident.io interface (whatever you pass - minor, major, critical) to the actual severity ID.

Risks

None

Performance impact

None

Security impact

None

How to QA

  1. Configure Incident.io Sink
  2. Send a few simulated requests
./robusta playbooks trigger prometheus_alert --namespace=monitoring alert_name="IncidentIoSeverityTesting" severity=critical labels="robusta_incidentio_severity:major"
./robusta playbooks trigger prometheus_alert --namespace=monitoring alert_name="IncidentIoSeverityTesting" severity=critical labels="robusta_incidentio_severity:major" annotations="dashboard_url=https://grafana.com,runbook_url=https://google.com"
  1. Go to Incident.io UI, find alerts and inspect payload

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant