-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
tingwei
committed
Nov 7, 2024
1 parent
537ba2a
commit c7c5df6
Showing
40 changed files
with
2,317 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Submodule prometheus
deleted from
220171
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
skip_list: | ||
- '106' | ||
- '204' | ||
- '208' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
*.retry | ||
*.log | ||
.molecule | ||
.cache | ||
__pycache__/ | ||
.pytest_cache | ||
.tox |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
--- | ||
pull_request_rules: | ||
- name: automatic merge and new release from cloudalchemybot | ||
conditions: | ||
- "status-success=Travis CI - Pull Request" | ||
- status-success=WIP | ||
- head~=autoupdate|skeleton | ||
- author=cloudalchemybot | ||
actions: | ||
merge: | ||
method: squash | ||
strict: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
--- | ||
extends: default | ||
ignore: | | ||
.github/ | ||
meta/ | ||
|
||
rules: | ||
braces: | ||
max-spaces-inside: 1 | ||
level: error | ||
brackets: | ||
max-spaces-inside: 1 | ||
level: error | ||
line-length: disable |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
# Contributor Guideline | ||
|
||
This document provides an overview of how you can participate in improving this project or extending it. We are | ||
grateful for all your help: bug reports and fixes, code contributions, documentation or ideas. Feel free to join, we | ||
appreciate your support!! | ||
|
||
## Communication | ||
|
||
### GitHub repositories | ||
|
||
Much of the issues, goals and ideas are tracked in the respective projects in GitHub. Please use this channel to report | ||
bugs, ask questions, and request new features . | ||
|
||
## git and GitHub | ||
|
||
In order to contribute code please: | ||
|
||
1. Fork the project on GitHub | ||
2. Clone the project | ||
3. Add changes (and tests) | ||
4. Commit and push | ||
5. Create a merge-request | ||
|
||
To have your code merged, see the expectations listed below. | ||
|
||
You can find a well-written guide [here](https://help.github.com/articles/fork-a-repo). | ||
|
||
Please follow common commit best-practices. Be explicit, have a short summary, a well-written description and | ||
references. This is especially important for the merge-request. | ||
|
||
Some great guidelines can be found [here](https://wiki.openstack.org/wiki/GitCommitMessages) and | ||
[here](http://robots.thoughtbot.com/5-useful-tips-for-a-better-commit-message). | ||
|
||
## Releases | ||
|
||
We try to stick to semantic versioning and our releases are automated. Release is created by assigning a keyword (in a | ||
way similar to travis [`[ci skip]`](https://docs.travis-ci.com/user/customizing-the-build#Skipping-a-build)) to a | ||
commit with merge request. Available keywords are (square brackets are important!): | ||
|
||
* `[patch]`, `[fix]` - for PATCH version release | ||
* `[minor]`, `[feature]`, `[feat]` - for MINOR version release | ||
* `[major]`, `[breaking change]` - for MAJOR version release | ||
|
||
## Changelog | ||
|
||
Changelog is generateg automatically on every merged Pull Request and all information is taken from github issues, PRs | ||
and labels. | ||
|
||
## Expectations | ||
|
||
### Keep it simple | ||
|
||
We try to provide production ready ansible roles which should be as much zero-conf as possible but this doesn't mean to | ||
overcomplicate things. Just follow [KISS](https://en.wikipedia.org/wiki/KISS_principle). | ||
|
||
### Be explicit | ||
|
||
* Please avoid using nonsensical property and variable names. | ||
* Use self-describing attribute names for user configuration. | ||
* In case of failures, communicate what happened and why a failure occurs to the user. Make it easy to track the code | ||
or action that produced the error. Try to catch and handle errors if possible to provide improved failure messages. | ||
|
||
|
||
### Add tests | ||
|
||
We are striving to use at least two test scenarios located in [/molecule](molecule) directory. First one | ||
([default](molecule/default)) is testing default configuration without any additional variables, second one | ||
([alternative](molecule/alternative)) is testing what happens when many variables from | ||
[/defaults/main.yml](defaults/main.yml) are changed. When adding new functionalities please add tests to proper | ||
scenarios. Tests are written in testinfra framework and are located in `/tests` subdirectory of scenario directory | ||
(for example default tests are in [/molecule/default/tests](molecule/default/tests)). | ||
More information about: | ||
- [testinfra](http://testinfra.readthedocs.io/en/latest/index.html) | ||
- [molecule](https://molecule.readthedocs.io/en/latest/index.html) | ||
|
||
### Follow best practices | ||
|
||
Please follow [ansible best practices](http://docs.ansible.com/ansible/latest/playbooks_best_practices.html) and | ||
especially provide meaningful names to tasks and even comments where needed. | ||
|
||
Our test framework automatically lints code with [`yamllint`](https://yamllint.readthedocs.io) and | ||
[`ansible-lint`](https://github.com/willthames/ansible-lint) programs so be sure to follow their rules. | ||
|
||
Remember: Code is generally read much more often than written. | ||
|
||
### Use Markdown | ||
|
||
Wherever possible, please refrain from any other formats and stick to simple markdown. | ||
|
||
## Requirements regarding roles design | ||
|
||
We are trying to create the best and most secure installation method for non-containerized prometheus stack components. | ||
To accomplish this all roles need to support: | ||
|
||
- current and at least one previous ansible version (wherever possible we try to support 2 previous ansible versions) | ||
- systemd as the only available process manager | ||
- at least latest debian and CentOS distributions |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
The MIT License (MIT) | ||
|
||
Copyright (c) 2017-2018 Pawel Krupa, Roman Demachkovych | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,156 @@ | ||
<p><img src="https://cdn.worldvectorlogo.com/logos/prometheus.svg" alt="prometheus logo" title="prometheus" align="right" height="60" /></p> | ||
|
||
# Ansible Role: prometheus | ||
|
||
[](https://travis-ci.org/cloudalchemy/ansible-prometheus) | ||
[](https://opensource.org/licenses/MIT) | ||
[](https://galaxy.ansible.com/cloudalchemy/prometheus/) | ||
[](https://github.com/cloudalchemy/ansible-prometheus/tags) | ||
|
||
## Description | ||
|
||
Deploy [Prometheus](https://github.com/prometheus/prometheus) monitoring system using ansible. | ||
|
||
### Upgradability notice | ||
|
||
When upgrading from <= 2.4.0 version of this role to >= 2.4.1 please turn off your prometheus instance. More in [2.4.1 release notes](https://github.com/cloudalchemy/ansible-prometheus/releases/tag/2.4.1) | ||
|
||
## Requirements | ||
|
||
- Ansible >= 2.7 (It might work on previous versions, but we cannot guarantee it) | ||
- jmespath on deployer machine. If you are using Ansible from a Python virtualenv, install *jmespath* to the same virtualenv via pip. | ||
- gnu-tar on Mac deployer host (`brew install gnu-tar`) | ||
|
||
## Role Variables | ||
|
||
All variables which can be overridden are stored in [defaults/main.yml](defaults/main.yml) file as well as in table below. | ||
|
||
| Name | Default Value | Description | | ||
| -------------- | ------------- | -----------------------------------| | ||
| `prometheus_version` | 2.24.1 | Prometheus package version. Also accepts `latest` as parameter. Only prometheus 2.x is supported | | ||
| `prometheus_skip_install` | false | Prometheus installation tasks gets skipped when set to true. | | ||
| `prometheus_binary_local_dir` | "" | Allows to use local packages instead of ones distributed on github. As parameter it takes a directory where `prometheus` AND `promtool` binaries are stored on host on which ansible is ran. This overrides `prometheus_version` parameter | | ||
| `prometheus_config_dir` | /etc/prometheus | Path to directory with prometheus configuration | | ||
| `prometheus_db_dir` | /var/lib/prometheus | Path to directory with prometheus database | | ||
| `prometheus_web_listen_address` | "0.0.0.0:9090" | Address on which prometheus will be listening | | ||
| `prometheus_web_external_url` | "" | External address on which prometheus is available. Useful when behind reverse proxy. Ex. `http://example.org/prometheus` | | ||
| `prometheus_storage_retention` | "30d" | Data retention period | | ||
| `prometheus_storage_retention_size` | "0" | Data retention period by size | | ||
| `prometheus_config_flags_extra` | {} | Additional configuration flags passed to prometheus binary at startup | | ||
| `prometheus_alertmanager_config` | [] | Configuration responsible for pointing where alertmanagers are. This should be specified as list in yaml format. It is compatible with official [<alertmanager_config>](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alertmanager_config) | | ||
| `prometheus_alert_relabel_configs` | [] | Alert relabeling rules. This should be specified as list in yaml format. It is compatible with the official [<alert_relabel_configs>](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#alert_relabel_configs) | | ||
| `prometheus_global` | { scrape_interval: 60s, scrape_timeout: 15s, evaluation_interval: 15s } | Prometheus global config. Compatible with [official configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file) | | ||
| `prometheus_remote_write` | [] | Remote write. Compatible with [official configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#<remote_write>) | | ||
| `prometheus_remote_read` | [] | Remote read. Compatible with [official configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#<remote_read>) | | ||
| `prometheus_external_labels` | environment: "{{ ansible_fqdn \| default(ansible_host) \| default(inventory_hostname) }}" | Provide map of additional labels which will be added to any time series or alerts when communicating with external systems | | ||
| `prometheus_targets` | {} | Targets which will be scraped. Better example is provided in our [demo site](https://github.com/cloudalchemy/demo-site/blob/2a8a56fc10ce613d8b08dc8623230dace6704f9a/group_vars/all/vars#L8) | | ||
| `prometheus_scrape_configs` | [defaults/main.yml#L58](https://github.com/cloudalchemy/ansible-prometheus/blob/ff7830d06ba57be1177f2b6fca33a4dd2d97dc20/defaults/main.yml#L47) | Prometheus scrape jobs provided in same format as in [official docs](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) | | ||
| `prometheus_config_file` | "prometheus.yml.j2" | Variable used to provide custom prometheus configuration file in form of ansible template | | ||
| `prometheus_alert_rules` | [defaults/main.yml#L81](https://github.com/cloudalchemy/ansible-prometheus/blob/73d6df05a775ee5b736ac8f28d5605f2a975d50a/defaults/main.yml#L85) | Full list of alerting rules which will be copied to `{{ prometheus_config_dir }}/rules/ansible_managed.rules`. Alerting rules can be also provided by other files located in `{{ prometheus_config_dir }}/rules/` which have `*.rules` extension | | ||
| `prometheus_alert_rules_files` | [defaults/main.yml#L78](https://github.com/cloudalchemy/ansible-prometheus/blob/73d6df05a775ee5b736ac8f28d5605f2a975d50a/defaults/main.yml#L78) | List of folders where ansible will look for files containing alerting rules which will be copied to `{{ prometheus_config_dir }}/rules/`. Files must have `*.rules` extension | | ||
| `prometheus_static_targets_files` | [defaults/main.yml#L78](https://github.com/cloudalchemy/ansible-prometheus/blob/73d6df05a775ee5b736ac8f28d5605f2a975d50a/defaults/main.yml#L81) | List of folders where ansible will look for files containing custom static target configuration files which will be copied to `{{ prometheus_config_dir }}/file_sd/`. | | ||
|
||
|
||
### Relation between `prometheus_scrape_configs` and `prometheus_targets` | ||
|
||
#### Short version | ||
|
||
`prometheus_targets` is just a map used to create multiple files located in "{{ prometheus_config_dir }}/file_sd" directory. Where file names are composed from top-level keys in that map with `.yml` suffix. Those files store [file_sd scrape targets data](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#file_sd_config) and they need to be read in `prometheus_scrape_configs`. | ||
|
||
#### Long version | ||
|
||
A part of *prometheus.yml* configuration file which describes what is scraped by prometheus is stored in `prometheus_scrape_configs`. For this variable same configuration options as described in [prometheus docs](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#<scrape_config>) are used. | ||
|
||
Meanwhile `prometheus_targets` is our way of adopting [prometheus scrape type `file_sd`](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#<file_sd_config>). It defines a map of files with their content. A top-level keys are base names of files which need to have their own scrape job in `prometheus_scrape_configs` and values are a content of those files. | ||
|
||
All this mean that you CAN use custom `prometheus_scrape_configs` with `prometheus_targets` set to `{}`. However when you set anything in `prometheus_targets` it needs to be mapped to `prometheus_scrape_configs`. If it isn't you'll get an error in preflight checks. | ||
|
||
#### Example | ||
|
||
Lets look at our default configuration, which shows all features. By default we have this `prometheus_targets`: | ||
``` | ||
prometheus_targets: | ||
node: # This is a base file name. File is located in "{{ prometheus_config_dir }}/file_sd/<<BASENAME>>.yml" | ||
- targets: # | ||
- localhost:9100 # All this is a targets section in file_sd format | ||
labels: # | ||
env: test # | ||
``` | ||
Such config will result in creating one file named `node.yml` in `{{ prometheus_config_dir }}/file_sd` directory. | ||
|
||
Next this file needs to be loaded into scrape config. Here is modified version of our default `prometheus_scrape_configs`: | ||
``` | ||
prometheus_scrape_configs: | ||
- job_name: "prometheus" # Custom scrape job, here using `static_config` | ||
metrics_path: "/metrics" | ||
static_configs: | ||
- targets: | ||
- "localhost:9090" | ||
- job_name: "example-node-file-servicediscovery" | ||
file_sd_configs: | ||
- files: | ||
- "{{ prometheus_config_dir }}/file_sd/node.yml" # This line loads file created from `prometheus_targets` | ||
``` | ||
|
||
## Example | ||
|
||
### Playbook | ||
|
||
```yaml | ||
--- | ||
- hosts: all | ||
roles: | ||
- cloudalchemy.prometheus | ||
vars: | ||
prometheus_targets: | ||
node: | ||
- targets: | ||
- localhost:9100 | ||
- demo.cloudalchemy.org:9100 | ||
labels: | ||
env: demosite | ||
``` | ||
### Demo site | ||
Prometheus organization provide a demo site for full monitoring solution based on prometheus and grafana. Repository with code and links to running instances is [available on github](https://github.com/prometheus/demo-site). | ||
### Defining alerting rules files | ||
Alerting rules are defined in `prometheus_alert_rules` variable. Format is almost identical to one defined in[ Prometheus 2.0 documentation](https://prometheus.io/docs/prometheus/latest/configuration/template_examples/). | ||
Due to similarities in templating engines, every templates should be wrapped in `{% raw %}` and `{% endraw %}` statements. Example is provided in [defaults/main.yml](defaults/main.yml) file. | ||
|
||
## Local Testing | ||
|
||
The preferred way of locally testing the role is to use Docker and [molecule](https://github.com/metacloud/molecule) (v2.x). You will have to install Docker on your system. See "Get started" for a Docker package suitable to for your system. | ||
We are using tox to simplify process of testing on multiple ansible versions. To install tox execute: | ||
```sh | ||
pip3 install tox | ||
``` | ||
To run tests on all ansible versions (WARNING: this can take some time) | ||
```sh | ||
tox | ||
``` | ||
To run a custom molecule command on custom environment with only default test scenario: | ||
```sh | ||
tox -e py35-ansible28 -- molecule test -s default | ||
``` | ||
For more information about molecule go to their [docs](http://molecule.readthedocs.io/en/latest/). | ||
|
||
If you would like to run tests on remote docker host just specify `DOCKER_HOST` variable before running tox tests. | ||
|
||
## Travis CI | ||
|
||
Combining molecule and travis CI allows us to test how new PRs will behave when used with multiple ansible versions and multiple operating systems. This also allows use to create test scenarios for different role configurations. As a result we have a quite large test matrix which will take more time than local testing, so please be patient. | ||
|
||
## Contributing | ||
|
||
See [contributor guideline](CONTRIBUTING.md). | ||
|
||
## Troubleshooting | ||
|
||
See [troubleshooting](TROUBLESHOOTING.md). | ||
|
||
## License | ||
|
||
This project is licensed under MIT License. See [LICENSE](/LICENSE) for more details. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Troubleshooting | ||
|
||
|
Oops, something went wrong.