Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add couchbase support #404

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ NewRelic platform. Currently supported backend systems are:
- Alternative PHP Cache
- Apache HTTP Server
- CouchDB
- Couchbase
- Elasticsearch
- HAProxy
- Memcached
Expand All @@ -19,6 +20,12 @@ NewRelic platform. Currently supported backend systems are:
- Redis
- Riak
- uWSGI
- NagioRelic
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is nagiorelic related to couchbase? if not than should go to a separate pr

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True - but I cannot commit to the minodes repository any more.


+ this is not a backend
+ it just executes programs (usually bash scripts) in a directory
+ the program name is used as the metric name
+ the output of the program must be numeric
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part should probably go to a different branch, isn't related to the couchbase support part

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this referes to nagiorelic.


Base Requirements
-----------------
Expand Down Expand Up @@ -298,6 +305,12 @@ Configuration Example
#username: foo
#password: bar

couchbase:
name: localhost
host: localhost
port: 8091
path: pools/default/buckets

elasticsearch:
name: clustername
host: localhost
Expand Down Expand Up @@ -407,6 +420,9 @@ Configuration Example
port: 8098
#verify_ssl_cert: true

nagiorelic:
path: /etc/newrelic/nagiorelic

Daemon:
user: newrelic
pidfile: /var/run/newrelic/newrelic-plugin-agent.pid
Expand Down
9 changes: 9 additions & 0 deletions docker/base/newrelic-plugin-agent.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ Application:
# username: foo
# password: bar

#couchbase:
# name: localhost
# host: localhost
# port: 8091
# path: pools/default/buckets

#elasticsearch:
# name: Clustername
# host: localhost
Expand Down Expand Up @@ -151,6 +157,9 @@ Application:
# port: 1717
# path: /path/to/unix/socket

#nagiorelic:
# path: /etc/newrelic/nagiorelic

Daemon:
user: newrelic
pidfile: /var/run/newrelic/newrelic-plugin-agent.pid
Expand Down
9 changes: 9 additions & 0 deletions etc/newrelic/newrelic-plugin-agent.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ Application:
# username: foo
# password: bar

#couchbase:
# name: localhost
# host: localhost
# port: 8091
# path: pools/default/buckets

#elasticsearch:
# name: Clustername
# host: localhost
Expand Down Expand Up @@ -151,6 +157,9 @@ Application:
# port: 1717
# path: /path/to/unix/socket

#nagiorelic:
# path: /etc/newrelic/nagiorelic

Daemon:
user: newrelic
pidfile: /var/run/newrelic/newrelic-plugin-agent.pid
Expand Down
4 changes: 3 additions & 1 deletion newrelic_plugin_agent/plugins/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
available = {
'apache_httpd': 'newrelic_plugin_agent.plugins.apache_httpd.ApacheHTTPD',
'couchdb': 'newrelic_plugin_agent.plugins.couchdb.CouchDB',
'couchbase': 'newrelic_plugin_agent.plugins.couchbase.Couchbase',
'edgecast': 'newrelic_plugin_agent.plugins.edgecast.Edgecast',
'elasticsearch':
'newrelic_plugin_agent.plugins.elasticsearch.ElasticSearch',
Expand All @@ -20,4 +21,5 @@
'rabbitmq': 'newrelic_plugin_agent.plugins.rabbitmq.RabbitMQ',
'redis': 'newrelic_plugin_agent.plugins.redis.Redis',
'riak': 'newrelic_plugin_agent.plugins.riak.Riak',
'uwsgi': 'newrelic_plugin_agent.plugins.uwsgi.uWSGI'}
'uwsgi': 'newrelic_plugin_agent.plugins.uwsgi.uWSGI',
'nagiorelic': 'newrelic_plugin_agent.plugins.nagiorelic.NagioRelic'}
2 changes: 1 addition & 1 deletion newrelic_plugin_agent/plugins/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ def http_get(self, url=None):
:rtype: requests.models.Response

"""
LOGGER.debug('Polling %s Stats at %s',
LOGGER.info('Polling %s Stats at %s',
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was the log level changed entirely for all plugins?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be merged. I think I did that as a separate commit for this reason.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zeitlinger please see #457 , it builds up and what you did and fixes many bugs.

self.__class__.__name__, url or self.stats_url)
req_kwargs = self.request_kwargs
req_kwargs.update({'url': url} if url else {})
Expand Down
60 changes: 60 additions & 0 deletions newrelic_plugin_agent/plugins/couchbase.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""
couchbase

"""

from newrelic_plugin_agent.plugins import base


class Couchbase(base.JSONStatsPlugin):

GUID = 'com.meetme.couchbase'

def add_datapoints(self, stats):
for bucket_data in stats:
bucket_name = bucket_data['name']
bucket_stats = bucket_data['basicStats']
if bucket_stats:
""" Example JSON
"basicStats":{
"quotaPercentUsed":7.548735046386719,
"opsPerSec":0,
"diskFetches":0,
"itemCount":2222,
"diskUsed":229366539,
"dataUsed":203511808,
"memUsed":79154224
}
"""
self.add_gauge_bucket_metric(
bucket_name, 'quotaPercentUsed', 'percent', bucket_stats['quotaPercentUsed']
)
self.add_gauge_bucket_metric(
bucket_name, 'opsPerSec', 'ops', bucket_stats['opsPerSec']
)
self.add_gauge_bucket_metric(
bucket_name, 'diskFetches', 'count', bucket_stats['diskFetches']
)
self.add_gauge_bucket_metric(
bucket_name, 'itemCount', 'count', bucket_stats['itemCount']
)
self.add_gauge_bucket_metric(
bucket_name, 'diskUsed', 'Byte', bucket_stats['diskUsed']
)
self.add_gauge_bucket_metric(
bucket_name, 'dataUsed', 'Byte', bucket_stats['dataUsed']
)
self.add_gauge_bucket_metric(
bucket_name, 'memUsed', 'Byte', bucket_stats['memUsed']
)

# Summary metrics
self.add_gauge_value('Summary/%s/quotaPercentUsed' % bucket_name, 'percent',
bucket_stats['quotaPercentUsed'],
min_val=0, max_val=0)
self.add_gauge_value('Summary/%s/diskUsed' % bucket_name, 'byte',
bucket_stats['diskUsed'])

def add_gauge_bucket_metric(self, bucket_name, metric_name, units, metric_value):
if metric_value:
self.add_gauge_value('%s/%s' % (bucket_name, metric_name), units, metric_value)
35 changes: 35 additions & 0 deletions newrelic_plugin_agent/plugins/nagiorelic.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
"""
Nagiorelic Plugin

"""
import logging
import os
import subprocess
from subprocess import CalledProcessError

from newrelic_plugin_agent.plugins import base


LOGGER = logging.getLogger(__name__)


class NagioRelic(base.Plugin):

GUID = 'com.meetme.nagiorelic'

def poll(self):
self.initialize()

root = self.config['path']
for script in os.listdir(root):
try:
output = subprocess.check_output(os.path.join(root, script)).decode().strip()
if output.isdigit():
metric = '%s' % (os.path.splitext(os.path.basename(script))[0])
self.add_gauge_value(metric, None, int(output))
else:
LOGGER.error('command did not return a number %s: %s' % (script, output))
except CalledProcessError as e:
LOGGER.error('command returned an error %s: %s: %s' % (script, e.returncode, e.output))

self.finish()