Add api and functionality to compare disk registry state with db record #4842

PeshkovMikhail · 2025-12-15T10:14:52Z

No description provided.

github-actions · 2025-12-15T10:15:04Z

Hi! Thank you for contributing!
The tests on this PR will run after a maintainer adds an ok-to-test label to this PR manually. Thank you for your patience!

github-actions · 2025-12-15T12:54:12Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f974118.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6057	0	1	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f974118.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f974118.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_compare_state.cpp

komarevtsev-d · 2025-12-15T10:47:58Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_compare_state.cpp

+    TTxDiskRegistry::TCompareDiskRegistryState& args)
+{
+    Y_UNUSED(ctx);
+    Y_UNUSED(args);


лишнее

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_compare_state.cpp

komarevtsev-d · 2025-12-15T11:45:02Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_state.cpp

+    static_assert(sizeof(*this) == 2144);
+
+    google::protobuf::util::MessageDifferencer diff;
+    NProtoBuf::string report;


std::string?

komarevtsev-d · 2025-12-15T11:50:51Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_state.cpp

+    TVector<TString> result;
+
+    if(!diff.Compare(StorageConfig->GetStorageConfigProto(), rhs.StorageConfig->GetStorageConfigProto())) {
+        std::stringstream ss;


TStringBuilder

komarevtsev-d · 2025-12-15T11:51:00Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_state.cpp

+    }
+
+    if(!diff.Compare(CurrentConfig, rhs.CurrentConfig)) {
+        std::stringstream ss;


TStringBuilder

komarevtsev-d · 2025-12-15T11:51:33Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_state.cpp

+    }
+
+    const auto& vPlacementGroups = rhs.PlacementGroups;
+    std::stringstream ss;


TStringBuilder

komarevtsev-d · 2025-12-15T13:16:34Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_state.h


        TVector<TLaggingDevice> OutdatedLaggingDevices;
+
+        TString getDiff(const TDiskState& rhs) const;


С большой буквы названия

PeshkovMikhail · 2025-12-16T07:04:17Z

cloud/blockstore/libs/storage/api/disk_registry.h

    xxx(GetAgentNodeId,                     __VA_ARGS__)                       \
    xxx(AddOutdatedLaggingDevices,          __VA_ARGS__)                       \
    xxx(GetClusterCapacity,                 __VA_ARGS__)                       \
+    xxx(CompareDiskRegistryStateWithLocalDb,           __VA_ARGS__)                       \


поздно заметил, со следующими правками исправлю

github-actions · 2025-12-16T08:17:43Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit d172d31.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6057	0	1	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit d172d31.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit d172d31.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

github-actions · 2025-12-16T13:45:38Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 6985ce8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6057	0	1	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 6985ce8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 6985ce8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

github-actions · 2025-12-16T18:08:43Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 27b77de.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6057	0	1	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 27b77de.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 27b77de.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

github-actions · 2025-12-18T13:22:14Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f898a19.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6053	0	5	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f898a19.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
10	5	0	5	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit f898a19.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
10	5	0	5	0	0	0

github-actions · 2025-12-22T08:30:29Z

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 68a07c8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6058	0	0	0	1	0

github-actions · 2025-12-23T12:40:16Z

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 3f20bfc.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6059	6058	0	0	0	1	0

github-actions · 2025-12-24T08:19:22Z

Note

This is an automated comment that will be appended during run.

🟢 linux-x86_64-relwithdebinfo: all tests PASSED for commit 54885f3.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6081	6080	0	0	0	1	0

vladstepanyuk · 2025-12-25T09:59:15Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_compare_state.cpp

+using namespace NKikimr::NTabletFlatExecutor;
+
+namespace {
+


vladstepanyuk · 2025-12-25T10:02:11Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_compare_state.cpp

+STFUNC(TCompareActor::StateCompare)
+{
+    switch (ev->GetTypeRewrite()) {
+        HFunc(


надо бы обрабатывать PoisonPill, чтобы если DR умрет, то и этот актор умер

vladstepanyuk · 2025-12-25T10:06:37Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_backup_state.cpp

        TransactionTimeTracker.GetInflightInfo(GetCycleCount()).c_str());
+
+    NProto::TBackupDiskRegistryStateResponse record;
+    if (msg->Record.GetSource() != NProto::EBackupDiskRegistryStateSource::LOCAL_DB) {


Suggested change

if (msg->Record.GetSource() != NProto::EBackupDiskRegistryStateSource::LOCAL_DB) {

if (msg->Record.GetSource() == NProto::EBackupDiskRegistryStateSource::RAM) {

vladstepanyuk · 2025-12-25T10:07:47Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_backup_state.cpp

        std::move(requestInfo),
-        msg->Record.GetBackupFilePath());
+        msg->Record.GetBackupFilePath(),
+        record);


зочем?

vladstepanyuk · 2025-12-25T10:10:14Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_actor_backup_state.cpp

    Y_UNUSED(ctx);

    TDiskRegistryDatabase db(tx.DB);
    return LoadState(db, args.Snapshot);


лучше сюда добавить
if (args.source == NProto::EBackupDiskRegistryStateSource::BOTH) {
args.RamSnapshot = State->BackupState();
}

vladstepanyuk · 2025-12-25T10:11:44Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_state_notification.h

        ui64 diskStateSeqNo,
        TVector<TDiskId> outdatedVolumes);
-
+    


убери, и пройдись по README и включи в вскоде все настройки вскода нужные чтобы таких space ов не было на пустых строках

vladstepanyuk · 2025-12-25T10:13:15Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_ut_lifecycle.cpp

        UNIT_ASSERT(!diskRegistry.Exists("nonrepl-garbage"));
    }

+    Y_UNIT_TEST(ShouldKeepDiskRegistryStateEqualWithLocalDb)


стоит добавить тест что отлавливается разъезд локальной базы и стейта

vladstepanyuk · 2025-12-25T10:15:21Z

cloud/blockstore/libs/storage/protos/disk.proto

-message TBackupDiskRegistryStateRequest
+enum EBackupDiskRegistryStateSource
 {
+    RAM = 0;


скорее что-то тип
BACKUP_DISK_REGISTRY_STATE_SOURCE_RAM
и остальные значения енума анологично

vladstepanyuk · 2025-12-25T10:18:36Z

cloud/blockstore/libs/storage/disk_registry/disk_registry_tx.h

    xxx(PurgeHostCms,                       __VA_ARGS__)                       \
    xxx(RemoveOrphanDevices,                __VA_ARGS__)                       \
    xxx(AddOutdatedLaggingDevices,          __VA_ARGS__)                       \
+    xxx(CompareDiskRegistryStateWithLocalDb,           __VA_ARGS__)                       \


можно убрать эту транзакцию теперь

github-actions · 2025-12-25T12:07:58Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 2f845a0.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6085	6049	0	35	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 2f845a0.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
70	35	0	35	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 2f845a0.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
70	35	0	35	0	0	0

github-actions · 2025-12-25T15:31:03Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit e6d8420.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
6085	6049	0	35	0	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit e6d8420.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
70	35	0	35	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit e6d8420.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
70	35	0	35	0	0	0

github-actions · 2025-12-26T15:54:53Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 97c4e58.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
5783	5780	0	1	1	1	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 97c4e58.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 97c4e58.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	1	0	1	0	0	0

komarevtsev-d self-requested a review December 15, 2025 10:30

komarevtsev-d added ok-to-test Label to approve test launch for external members large-tests Launch large tests for PR blockstore Add this label to run only cloud/blockstore build and tests on PR labels Dec 15, 2025

github-actions bot removed the ok-to-test Label to approve test launch for external members label Dec 15, 2025

komarevtsev-d reviewed Dec 15, 2025

View reviewed changes

PeshkovMikhail requested a review from komarevtsev-d December 16, 2025 06:56

PeshkovMikhail commented Dec 16, 2025

View reviewed changes

PeshkovMikhail mentioned this pull request Dec 18, 2025

Periodic comparison of the local DiskRegistryState with the state from the local database #4873

Open

Mikhail Peshkov added 8 commits December 24, 2025 12:51

Add api and functionality to compare disk registry state with db record

ff0af76

refactor

f1e8129

hashmap comparsion bug fix

7b33a2e

integration test and comment potential non persistent fields

961605e

ut diff info

d17ea10

Add DisksToCleanup and PendingCleanup to comparsion

5a8bf33

move to protobuf diff

44a3713

log info edit

54885f3

PeshkovMikhail force-pushed the drstate-compare branch from 950ceb8 to 54885f3 Compare December 24, 2025 05:57

Mikhail Peshkov added 2 commits December 25, 2025 16:44

log info edit

0bb99c8

dedicated compare actor and use backup transaction

2f845a0

vladstepanyuk requested changes Dec 25, 2025

View reviewed changes

Mikhail Peshkov added 2 commits December 25, 2025 19:47

move getting ram snapshot logic to transaction when source is both

5acd451

change disks replica count instead of diskUpdate seqno

e6d8420

fix

97c4e58


		TVector<TLaggingDevice> OutdatedLaggingDevices;

		TString getDiff(const TDiskState& rhs) const;

	if (msg->Record.GetSource() != NProto::EBackupDiskRegistryStateSource::LOCAL_DB) {
	if (msg->Record.GetSource() == NProto::EBackupDiskRegistryStateSource::RAM) {

Add api and functionality to compare disk registry state with db record #4842

Are you sure you want to change the base?

Add api and functionality to compare disk registry state with db record #4842

Uh oh!

Conversation

PeshkovMikhail commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 22, 2025

Uh oh!

github-actions bot commented Dec 23, 2025

Uh oh!

github-actions bot commented Dec 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

github-actions bot commented Dec 15, 2025 •

edited

Loading

github-actions bot commented Dec 16, 2025 •

edited

Loading

github-actions bot commented Dec 16, 2025 •

edited

Loading

github-actions bot commented Dec 16, 2025 •

edited

Loading

github-actions bot commented Dec 18, 2025 •

edited

Loading

github-actions bot commented Dec 25, 2025 •

edited

Loading

github-actions bot commented Dec 25, 2025 •

edited

Loading

github-actions bot commented Dec 26, 2025 •

edited

Loading