Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eliminate Warning Logs When Prometheus Metrics are Enabled #4952

Closed
Tracked by #4998
haywoodsh opened this issue Jan 22, 2024 · 1 comment · Fixed by #5008
Closed
Tracked by #4998

Eliminate Warning Logs When Prometheus Metrics are Enabled #4952

haywoodsh opened this issue Jan 22, 2024 · 1 comment · Fixed by #5008
Assignees
Labels
backlog Pull requests/issues that are backlog items bug An issue reporting a potential bug
Milestone

Comments

@haywoodsh
Copy link
Contributor

Describe the bug

Enabling Prometheus metrics in the Ingress Controller generates unwarranted warning messages in the logs.

To Reproduce
Steps to reproduce the behavior:

  1. Initiate Ingress Controller with the -enable-prometheus-metrics flag.
  2. To expose the Prometheus metrics endpoint locally, execute:
kubectl port-forward `kubectl get po -n nginx-ingress -o=jsonpath="{.items[0].metadata.name}"` 9113:9113
--namespace=nginx-ingress
  1. Navigate to http://localhost:9113 to access Prometheus metrics.
  2. Observe that the Prometheus metrics display empty labels for resource_name, resource_namespace, and resource_type.
    Example:
lnginx_ingress_nginxplus_worker_connection_idle{class="nginx",id="0",pid="43",resource_name="",resource_namespace="",resource_type=""} 1
  1. Warning messages similar to the following are logged~
level=warn ts=2024-01-17T23:12:54.546Z caller=nginx_plus.go:1258 wrongnumberoflabelsforworker%v.Forlabels%v,gotvalues:%v.Emptylabelswillbeusedinstead=0
level=warn ts=2024-01-17T23:12:54.546Z caller=nginx_plus.go:1258 wrongnumberoflabelsforworker%v.Forlabels%v,gotvalues:%v.Emptylabelswillbeusedinstead=1
level=warn ts=2024-01-17T23:12:54.546Z caller=nginx_plus.go:1258 wrongnumberoflabelsforworker%v.Forlabels%v,gotvalues:%v.Emptylabelswillbeusedinstead=2
level=warn ts=2024-01-17T23:12:54.546Z caller=nginx_plus.go:1258 wrongnumberoflabelsforworker%v.Forlabels%v,gotvalues:%v.Emptylabelswillbeusedinstead=3
level=warn ts=2024-01-17T23:12:54.546Z caller=nginx_plus.go:1258 wrongnumberoflabelsforworker%v.Forlabels%v,gotvalues:%v.Emptylabelswillbeusedinstead=4
level=warn ts=2024-01-17T23:12:54.546Z caller=nginx_plus.go:1258 wrongnumberoflabelsforworker%v.Forlabels%v,gotvalues:%v.Emptylabelswillbeusedinstead=5

Expected behavior
Prometheus metrics should not contain empty labels, and no warning messages related to this should be logged.

Your environment

  • NIC 3.4.1
  • NGINX Plus

Additional context
Remove the variable label by modifying the line in the source code at https://github.com/nginxinc/kubernetes-ingress/blob/main/cmd/nginx-ingress/main.go#L703 from

workerPIDVariableLabels := []string{"resource_type", "resource_name", "resource_namespace"}`

to

var workerPIDVariableLabels []string

This change should result in metrics formatted as follows and eliminate the warning messages:

nginx_ingress_nginxplus_worker_connection_idle{class="nginx",id="0",pid="43"} 1
Copy link

Hi @haywoodsh thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this 🙂

Cheers!

@danielnginx danielnginx added the backlog Pull requests/issues that are backlog items label Jan 29, 2024
@danielnginx danielnginx modified the milestones: v3.5.0, v3.4.3 Jan 29, 2024
@vepatel vepatel added the bug An issue reporting a potential bug label Jan 30, 2024
@danielnginx danielnginx modified the milestones: v3.5.0, v3.4.3 Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog Pull requests/issues that are backlog items bug An issue reporting a potential bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants