[Disk Manager] Add ability to report external storage limits/usage by jkuradobery · Pull Request #6552 · ydb-platform/nbs

jkuradobery · 2026-07-17T15:29:10Z

Notes

We would like to be able to add additional snapshot metrics reporting (e.g. ydb storage usage from ydb-controlplane, S3 storage usage via api).
We want disk-manager to report it's storage metrics without us having to rely on external metrics providers.

github-actions · 2026-07-17T15:30:14Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

github-actions · 2026-07-17T15:36:40Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

Copilot

Pull request overview

This PR adds a new snapshot storage quota/usage metrics reporting hook to Disk Manager’s dataplane, wiring a quota reporter into the periodic snapshot metrics collection and extending tests/helpers to validate the new gauges are exported.

Changes:

Introduces pkg/snapshot quota reporter interface plus a stub implementation that exports used_bytes / limit_bytes gauges.
Wires reporter construction through app.Run(...) → initDataplane(...) → dataplane.RegisterForExecution(...) and calls Report() from collectSnapshotMetricsTask.
Extends facade test utilities to read Prometheus gauges and adds a metrics test asserting the new dataplane gauges exist.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cloud/disk_manager/pkg/ya.make	Adds `snapshot` package to build recursion.
cloud/disk_manager/pkg/snapshot/ya.make	New Go library target for snapshot quota reporting.
cloud/disk_manager/pkg/snapshot/quota.go	Defines quota reporter interface + constructor function type.
cloud/disk_manager/pkg/snapshot/quota_stub.go	Stub reporter that publishes quota gauges with bucket tag.
cloud/disk_manager/pkg/app/run.go	Threads quota reporter factory into app startup.
cloud/disk_manager/pkg/app/dataplane.go	Instantiates reporter and passes it into dataplane registration.
cloud/disk_manager/internal/pkg/dataplane/register.go	Extends registration to accept the quota reporter and pass it into task construction.
cloud/disk_manager/internal/pkg/dataplane/collect_snapshot_metrics_task.go	Calls `quotaReporter.Report()` on each metrics tick and logs failures.
cloud/disk_manager/internal/pkg/facade/testcommon/common.go	Adds gauge retrieval helpers (`GetGauge(s)`) for Prometheus metrics tests.
cloud/disk_manager/internal/pkg/facade/disk_service_test/disk_service_metrics_test.go	Adds assertions for the new quota gauges.
cloud/disk_manager/cmd/disk-manager/main.go	Wires the stub quota reporter into the disk-manager binary.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b9579046d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

github-actions · 2026-07-17T15:48:28Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

…or tests

github-actions · 2026-07-17T16:09:49Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 2100s): all tests PASSED for commit 0a5717a.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1948	1948	0	0	0	0	0

SvartMetal · 2026-07-20T10:54:01Z


 ////////////////////////////////////////////////////////////////////////////////

 func (c collectSnapshotMetricsTask) clearMetrics() {


It looks like we need to clear the quota metrics here, along with other metrics such as snapshots/count

I do not think we need to delete quota metrics here. We can just do max(usage) / min(limit) and call it a day.
We also do not unregister other metrics in clearMetrics, such as totalStorageSize, so I do not see the reason to delete the metric.

We also do not unregister other metrics in clearMetrics

Looks like we missed that, so it is a bug

We can just do max(usage) / min(limit) and call it a day.

Looks very kostylno and does not work correctly as well

Yes this is not a correct behavior. We can actually fail within the task for any reason, yet the disk-manager would keep on running, thus reporting metrics, which may lead to stale metrics.

SvartMetal · 2026-07-20T10:58:53Z

+import (
+	"context"
+
+	snapshot_config "github.com/ydb-platform/nbs/cloud/disk_manager/internal/pkg/dataplane/snapshot/config"


We should not reference internal packages here

See auth in the same package

github-actions · 2026-07-20T14:15:22Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

github-actions · 2026-07-20T14:31:49Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 2180s): all tests PASSED for commit 3bbd6a2.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1948	1948	0	0	0	0	0

github-actions · 2026-07-20T17:46:08Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

github-actions · 2026-07-20T17:56:02Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ disk + manager

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 3953s): all tests PASSED for commit 5b9853d.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1948	1948	0	0	0	0	0

[Disk Manager] Add ability to report external storage limits/usage

1b95790

Copilot AI review requested due to automatic review settings July 17, 2026 15:29

Copilot started reviewing on behalf of jkuradobery July 17, 2026 15:29 View session

jkuradobery added disk_manager Add this label to run only cloud/disk_manager build and tests on PR large-tests Launch large tests for PR labels Jul 17, 2026

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread cloud/disk_manager/internal/pkg/facade/disk_service_test/disk_service_metrics_test.go

Comment thread cloud/disk_manager/pkg/snapshot/quota_stub.go

chatgpt-codex-connector Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread cloud/disk_manager/internal/pkg/facade/testcommon/common.go Outdated

Comment thread cloud/disk_manager/cmd/disk-manager/main.go Outdated

Make GetGauges behave like GetCounters

01b421a

Use NewEmptySnapshotStorageQuotaReporter in main, use separate main f…

0a5717a

…or tests

SvartMetal requested changes Jul 20, 2026

View reviewed changes

fix review discussions

f7a7656

Fix review discussions

3bbd6a2

Add comment

d1540b6

Fix review discussions

5b9853d


		////////////////////////////////////////////////////////////////////////////////

		func (c collectSnapshotMetricsTask) clearMetrics() {

Uh oh!

Conversation

jkuradobery commented Jul 17, 2026

Notes

Uh oh!

github-actions Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SvartMetal Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

jkuradobery Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

SvartMetal Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SvartMetal Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkuradobery Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

SvartMetal Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

jkuradobery Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

github-actions Bot commented Jul 17, 2026 •

edited

Loading

github-actions Bot commented Jul 17, 2026 •

edited

Loading

github-actions Bot commented Jul 17, 2026 •

edited

Loading

github-actions Bot commented Jul 17, 2026 •

edited

Loading

SvartMetal Jul 22, 2026 •

edited

Loading

SvartMetal Jul 22, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading

github-actions Bot commented Jul 20, 2026 •

edited

Loading