Skip to content

Remove empty and obsolete storage entries - #3061

Open
UnArbosSix wants to merge 5 commits into
mainfrom
codex/storage-bloat-migration
Open

Remove empty and obsolete storage entries#3061
UnArbosSix wants to merge 5 commits into
mainfrom
codex/storage-bloat-migration

Conversation

@UnArbosSix

@UnArbosSix UnArbosSix commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Remove obsolete and explicit-default storage entries that unnecessarily increase chain-state size, while preserving all live nonzero state.

Changes

  • Adds a resumable Subtensor storage cleanup that removes obsolete pre-dTAO prefixes and explicit zero/default rows.
  • Persists cleanup progress and executes bounded work from on_idle after the beta-basket migration completes.
  • Replaces the obsolete Swap V3-to-Balancer upgrade hook with cleanup-only handling for abandoned Swap V3 prefixes and zero reservoir entries.
  • Adds migration guards so completed cleanup does not restart.
  • Bumps the runtime spec_version from 443 to 444.

Files of interest

  • pallets/subtensor/src/migrations/migrate_storage_bloat_v2.rs
  • pallets/subtensor/src/macros/hooks.rs
  • pallets/swap/src/pallet/migrations/migrate_storage_cleanup_v2.rs
  • pallets/swap/src/pallet/hooks.rs
  • runtime/src/lib.rs

Behavioral and migration impact

The migration removes only abandoned prefixes or entries whose encoded value represents the storage default. Nonzero Subtensor stake/share state and nonzero Swap reservoir balances are preserved. The Subtensor sweep is spread across idle block capacity and records completion in HasMigrationRun.

Testing

Added tests covering bounded/resumable Subtensor cleanup, preservation of nonzero state, migration completion guards, preservation of root stake age, Swap legacy-prefix removal, and preservation of pending nonzero reservoir balances.

@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
subtensor Ready Ready Preview Aug 7, 2026 10:13pm

Request Review

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🛡️ AI Review — Skeptic (security review)

VERDICT: VULNERABLE

VERY HIGH scrutiny: recent account with repository write access and substantive merged contributions; final commit is authored by a different contributor; no Gittensor association found; branch correctly targets main.

Findings

Sev File Finding
HIGH pallets/swap/src/pallet/migrations/migrate_storage_cleanup_v2.rs:35 Bound the Swap cleanup across blocks inline

Prior-comment reconciliation

  • cd542575: not addressed — The Swap cleanup remains synchronous and unbounded at line 35; only a comment asserting the current mainnet row count was added.

Conclusion

The Subtensor sweep is bounded, but the Swap cleanup still performs unbounded storage deletion and iteration synchronously during runtime upgrade, risking a block-budget overrun.


📜 Previous run (superseded)
Sev File Finding Status
HIGH pallets/swap/src/pallet/migrations/migrate_storage_cleanup_v2.rs:35 Bound the Swap cleanup across blocks ➡️ Carried forward to current findings
The Swap cleanup remains synchronous and unbounded at line 35; only a comment asserting the current mainnet row count was added.

🔍 AI Review — Auditor (domain review)

VERDICT: 👎

Gittensor association UNKNOWN by the trusted allowlists; the author is a recent account with repository write access and substantial prior Subtensor activity.

PR body was empty/trivial; the Auditor has auto-filled it. Please review.

The Subtensor cleanup is resumable and tested, and the runtime spec_version is bumped to 444. However, the Swap cleanup performs thousands of unbounded deletions plus full reservoir iterations inside on_runtime_upgrade, so it does not satisfy the repository’s bounded-migration requirement.

No auto-fixes were applicable.

Findings

Sev File Finding
HIGH pallets/swap/src/pallet/migrations/migrate_storage_cleanup_v2.rs:35 Bound the Swap cleanup across blocks inline

Conclusion

The unbounded Swap migration can overrun the runtime-upgrade block budget. It must be converted to a cursor-based multi-block cleanup before merge.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review — see the sticky summary comment for the verdict and the inline comments below for specific findings.

Comment thread pallets/swap/src/pallet/migrations/migrate_storage_cleanup_v2.rs
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: SAFE Auditor: 👎

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review — see the sticky summary comment for the verdict and the inline comments below for specific findings.

Comment on lines +21 to +35
"AlphaSqrtPrice",
"CurrentTick",
"EnabledUserLiquidity",
"FeeGlobalTao",
"FeeGlobalAlpha",
"LastPositionId",
"ScrapReservoirTao",
"ScrapReservoirAlpha",
"Ticks",
"TickIndexBitmapWords",
"SwapV3Initialized",
"CurrentLiquidity",
"Positions",
] {
remove_prefix::<T>("Swap", storage_name, &mut weight);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[HIGH] Bound the Swap cleanup across blocks

Each listed prefix is cleared with an effectively unlimited u32::MAX limit, and the same upgrade also fully iterates both reservoir maps. All of this executes synchronously in on_runtime_upgrade; the stated current row count is not an enforced bound, so excessive state can overrun the upgrade block's execution budget and prevent reliable runtime activation. Convert this cleanup to a cursor-based, weight-limited multi-block migration like the Subtensor sweep.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: VULNERABLE

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review — see the sticky summary comment for the verdict and the inline comments below for specific findings.

Comment on lines +20 to +35
for storage_name in [
"AlphaSqrtPrice",
"CurrentTick",
"EnabledUserLiquidity",
"FeeGlobalTao",
"FeeGlobalAlpha",
"LastPositionId",
"ScrapReservoirTao",
"ScrapReservoirAlpha",
"Ticks",
"TickIndexBitmapWords",
"SwapV3Initialized",
"CurrentLiquidity",
"Positions",
] {
remove_prefix::<T>("Swap", storage_name, &mut weight);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[HIGH] Bound the Swap cleanup across blocks

This invokes clear_prefix(..., Some(u32::MAX)) for 13 prefixes during on_runtime_upgrade, followed by full reservoir-map iterations. The asserted current row count does not enforce a runtime bound, and the migration can exceed the upgrade block budget as state grows or differs from the snapshot. Use a persistent cursor and consume bounded on_idle weight, as the Subtensor cleanup does; mark completion only after every prefix and reservoir map is exhausted.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

🔄 AI review updated — Skeptic: VULNERABLE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants