Grafana Dashboards have 'dips'

[AlexLamond]

After setting up Unpoller via Docker, with a Unifi Express I am finding that the charts on Grafana are shown with 'dips'. I would expect these lines to be nice and smooth, but sadly they keep reporting zero

This is happening for the chart 'UAP Bandwidth / Wireless Devices' on Dashboard 'UniFi-Poller: Client Insights'

[davidnewhall]

Increase the poll interval.

[AlexLamond]

Maybe I'm just blind, but what is the Docker ENV tag to do that?

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[drbrain]

There appears to be UP_INFLUXDB_INTERVAL for InfluxDB, but nothing for Prometheus

I'm using prometheus and have this same problem.

[davidnewhall]

Prometheus scrape interval?

[drbrain]

Increasing the prometheus scrape interval (how often prometheus requests metrics from unpoller) does not appear to make the unpoller prometheus exporter fetch metrics from unifi more often. Some prometheus exporters are on-demand, unpoller appears to fetch metrics on its own internal interval and export the cached copy.

I haven't dug into how UP_INFLUXDB_INTERVAL is implemented, but maybe increasing it will also increase the update frequency for the prometheus exporter?

[davidnewhall]

unpoller appears to fetch metrics on its own internal interval and export the cached copy.

That's certainly not how I wrote the code and I doubt @platinummonkey changed that logic. It fetches metrics during the scrape request.

[drbrain]

I rebuilt unpoller from 96a9259eb9e9ba138a39d6cf2691d4240c8b57e7 (today's master).

My prometheus scrape interval is 15s

The top panel in this screenshot is from telegraf scraping SNMP from my unifi switch directly.

The bottom panel in this screenshot is unpoller. It scrapes a UCK G2 Plus running Unifi OS v4.0.18 and Network 8.4.62.

I expect both to look the same, but unpoller only has new metrics about once per minute, sometimes less.

Checking prometheus metrics for scraping showed nothing unusual for unpoller. the scrape duration is ~500ms and there are no anomalies.

[davidnewhall]

You're reporting a completely different behavior than this original issue. This is normal. The API doesn't update as often as the SNMP values.

[drbrain]

My understanding of prometheus, sampling, and grafana dashboard constructing lead me to believe the two behaviors are tied to one another. Zero values exist when rate() is used with a range vector that is shorter than the update interval, but there is no way to configure unpoller to fetch data more quickly.

Upthread you wrote:

Increase the poll interval.

Then:

It fetches metrics during the scrape request.

Both of which are at odds with:

The API doesn't update as often as the SNMP values.

So it appears that unpoller cannot sample often enough for prometheus rate() and friends to avoid returning 0 values with a range vector of less than about [3m]. As rate() and range vectors are the tools used to build dashboards that don't drop to 0.

I understand unpoller may not have control of how often the API updates.

Therefore I expect the configuration documentation of scrape interval or update interval note that a duration of shorter than one minute is not going to result in improved resolution.

I also expect that the pre-built dashboards would have a minimum interval set to match.

I also expect that the slow API update rate is why the pre-computed metric unpoller_device_port_receive_rate_bytes also shows a 0 byte receive rate.

I understand we may have no way to win here, but I haven't seen any documentation about this limitation, and I believe that drove the initial reporter to create this issue.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[drbrain]

Does documentation need to be written to address the issue with scrape intervals, or is there another way to resolve this issue?

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[platinummonkey]

Does documentation need to be written to address the issue with scrape intervals, or is there another way to resolve this issue?

Yes this would be the way to do this. Docs website is available here https://github.com/unpoller/unpoller.github.io/tree/master/documentation