Skip to content

Report endpoint restore failures#6558

Draft
EvgeniyKozev wants to merge 2 commits into
mainfrom
evgeniy-kozev/report-endpoint-restore-failures
Draft

Report endpoint restore failures#6558
EvgeniyKozev wants to merge 2 commits into
mainfrom
evgeniy-kozev/report-endpoint-restore-failures

Conversation

@EvgeniyKozev

Copy link
Copy Markdown
Collaborator

What changed

  • Track endpoint restore errors in TEndpointManager with an atomic counter.
  • Mark endpoint restoration as failed when any restore step fails.
  • Return E_FAIL from ListEndpoints after failed restore, including the number of failed endpoints.
  • Extend ShouldRestoreEndpointWithNbdDevice to verify the new ListEndpoints error behavior.

Why

Previously restore errors were reported to logs/counters, but the final restore stage still became Completed. As a result, clients listing endpoints could observe EndpointsWereRestored=true even when some endpoints failed to restore.

Validation

  • git diff --check
  • ./ya make -tt cloud/blockstore/libs/endpoints/ut -F TEndpointManagerTest::ShouldRestoreEndpointWithNbdDevice
  • ./ya make -tt cloud/blockstore/libs/endpoints/ut

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/ (test time: 263s): all tests PASSED for commit 42d53f0.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1215 1215 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 285s): all tests PASSED for commit 42d53f0.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1572 1572 0 0 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1433s): some tests FAILED for commit 42d53f0.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
6847 6845 0 1 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 90s): some tests FAILED for commit 42d53f0.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 92s): some tests FAILED for commit 42d53f0.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5148s): all tests PASSED for commit 42d53f0.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4345 4345 0 0 0 0 0

Y_UNUSED(ctx);
Y_UNUSED(request);

if (AtomicGet(RestoringStage) == Failed) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be it will be better to extend TListEndpointsResponse with fiels representing RestoringStage(something like EndpointsStorageState) value instead of failing request with E_FAIL? If the request could not be completed because of some underlying subsystem failure, E_INVALID_STATE seems to be more appropriate error code.

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/ (test time: 260s): all tests PASSED for commit 3031e32.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1215 1215 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 263s): all tests PASSED for commit 3031e32.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1572 1572 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1448s): all tests PASSED for commit 3031e32.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
6847 6846 0 0 0 1 0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 4924s): all tests PASSED for commit 3031e32.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4345 4345 0 0 0 0 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants