Skip to content

Latest commit

 

History

History
28 lines (22 loc) · 2.21 KB

guide-prometheus.md

File metadata and controls

28 lines (22 loc) · 2.21 KB

Metrics Sources

We have three sources of metrics so far:

  1. Usage metrics of containers coming from the caadvisor see full list of caadvisor metrics and node usage metrics is coming from node-exporter
  2. Istio nework metrics coming from the istio exproter
  3. Seldon specific metrics coming from the Seldon core exporter

Metrics of interest

Here is the table of the metrics we used in this project and are of interest to us:

TODO make a table of metrics like full list of caadvisor metrics

Metrics

Check the following links for a better understanding of the Prometheus and PromQL language:

  1. Metrics Types vs Query Basics
  2. Understanding Prometheus Range Vectors
  3. The Anatomy of a PromQL Query

The metrics metadata are accessible from the HTTP API of the prometheus accessible at /api/v1. The api can also gives us the same information as the dashboard. However, the metadata of the metrics is only avaialable from the http API. E.g.

$PROM_SERVER=http://192.5.86.160:30090
curl -G $PROM_server/api/v1/targets/metadata --data-urlencode 'metric=container_cpu_cfs_periods_total'

returns:

{"status":"success","data":[{"target":{"endpoint":"https-metrics","instance":"10.140.81.236:10250","job":"kubelet","metrics_path":"/metrics/cadvisor","namespace":"kube-system","node":"k8s-cluster","service":"kubelet"},"type":"counter","help":"Number of elapsed enforcement period intervals.","unit":""}]}%