Possibility of limiting volume-related metrics (Top K volumes or specific metrics)

[songlin-rgb]

We have thousands of volumes, and each emits tens of metrics (mainly from volume.yaml, and read_io_type_percent from workload_volume.yaml). While some of these are extremely useful, there are concerns that this causes unnecessary load for downstream systems.

Is it possible to limit this data? For example:

• Metric filtering: Can we select specific metrics within a yaml? Currently, it seems to be 'all or nothing' within a yaml.
• Top-K filtering: Can we limit to volumes that are top K in one of the configurable criteria (e.g., storage capacity consumptions, IOPS, etc.)

I’d like to understand what the current system supports, so we can identify next steps. Thanks!

[rahulguptajss]

@songlin-rgb You can modify Harvest templates to add/remove metrics collected. There are multiple ways to do that as listed below.

https://netapp.github.io/harvest/latest/configure-templates/#how-to-extend-a-restrestperfstoragegridems-collectors-existing-object-template
https://netapp.github.io/harvest/latest/configure-templates/#conf-path

Harvest dashboards support Top-K queries. The queries used in time range dashboards for Top-K queries from Prometheus can be referenced as examples.

[songlin-rgb]

Thanks for the template references!

Regarding top-K queries, I think you are referring to the ability to filter in the dashboards, right? Is it possible to filter metrics inside or before Harvest exporters, i.e., make Prometheus only scrape metrics that meet at least one of the top K criteria? I completely understand if that’s not feasible. Thanks!

[rahulguptajss]

@songlin-rgb Yes, I meant on the dashboard side, once the metrics are already ingested into Prometheus. Could you explain what you mean by topk? Is it similar to exporting only metrics that exceed a certain threshold?

Harvest only has access to the current instant metric values. Therefore, calculating topk over a time range would require Harvest to track and process a range of metric values not just a single snapshot.

[songlin-rgb]

Is it currently possible to export only metrics that exceed a certain threshold? That could be an alternative in some cases.

[rahulguptajss]

We currently do not have this functionality but it should be possible to implement using a built‑in plugin that filters such data during export. Could you please open a feature request for this?

[songlin-rgb]

I will open that if we have a concrete use case. Thanks for all the info!