Intermittent error "Could not get stream throughput" 404; api response "I do not have a metric called [org.graylog2.plugin.streams.Stream.5891ca6995186c0001613209.incomingMessages]." #28
Comments
|
i too have the same issue Error: Could not get streams. Error: Bad status code: 404 Not Found [object Object] |
|
I'm having a similar error. Though the API definitely working outside of graylog-cli, from the same box graylog-cli is installed. Below is an example of the graylog-cli call and result and corresponding curl call and result. SetupGraylog Version: 2.1.3+040d371 Call being made through graylog-cliResults/Error from graylog-cliCall being made through CURLResults from CURL |
|
I decide to output what the serverURL was getting set as. graylog-dashboard.js lines 88-94 // Make sure we have a protocol (default: https)
if (config.serverURL.slice(0, 4) !== 'http') config.serverURL = 'https://' + config.serverURL;
// Make sure we have a port (default REST API port is 12900)
if (!/:\d+$/.test(config.serverURL)) config.serverURL += ':12900';
// Make sure config.serverURL has a trailing slash. (computers.)
if (config.serverURL[config.serverURL.length - 1] !== '/') config.serverURL += '/';
return config;Since With that being said, if you comment out lines 89, 91, 93 (for testing). // Make sure we have a protocol (default: https)
//if (config.serverURL.slice(0, 4) !== 'http') config.serverURL = 'https://' + config.serverURL;
// Make sure we have a port (default REST API port is 12900)
//if (!/:\d+$/.test(config.serverURL)) config.serverURL += ':12900';
// Make sure config.serverURL has a trailing slash. (computers.)
//if (config.serverURL[config.serverURL.length - 1] !== '/') config.serverURL += '/';
return config;Which results in getting a different error message: Which I took a look at the URL it was creating to get the metric for said stream and apparently it does not exist. Which I'm comparing this to getting all the metrics and all the metric names. const url = `${options.serverURL}system/metrics/org.graylog2.plugin.streams.Stream.${options.streamID}.incomingMessages.1-sec-rate`In the metrics output, the closest thing I see to this is: Is there additional setup needed on that stream itself, that I missed, which is required to be configured for the stream to be used with the graylog-cli ? |
|
Hi - I plan to take a look at this soon. Would be happy to accept a PR that refactors this logic and does some better error logging. |
|
Similar problem here with a newly configured stream in latest version 1.1.1 with Graylog 2.3.0: |
|
+1 from me. Basically I'm guessing this occurs in environments where the rate can't be computed because the number of messages is low, but that's just a guess. Please fix as the tool can't be used as is :-( graylog-dashboard |
|
I ran into this issue .. but worse than this, the code doesn't seem to work if you are hitting a dedicated webui endpoint, and have multiple graylog data ingestion nodes. The nodes themselves get the metric that its looking for, but the webui endpoint doesn't expose it. I think unfortunately that fixing this would require a pretty hefty refactor.. :/ |
Issue: The API call to graylog which gets a stream throughput is sometimes failing, being given a 404 response with the message that there is no such metric. This is unpredictable, sometimes failing for a given stream but sometimes succeeding
Details of setup
Expected Results
Actual results
Error and callstack shown at the command-line:
nginx log showing the api calls made by cli-dashboard:
Note:
Double-checking in the API browser
On the graylog server (http://myprivatedevbox:12900), it is possible to load the api-browser page to examine and try out api calls. The metrics call can be retried here, again getting a 404 response:
Further notes
The problem is semi-intermittent. Most often, if I run graylog-cli-dashboard for the same stream as before, it will fail as before (or succeed as before). No obvious pattern about when and why it changes.
see further screenshots (031694-031696) showing an example of when it starts successfully, but then fails on switching to a different stream, getting the same 404 response and crashing out.
031694_screenshot
031695_screenshot
031696_screenshot
The text was updated successfully, but these errors were encountered: