Skip to content

Cleanup sector root cache - #1009

Merged
ChrisSchinnerl merged 5 commits into
masterfrom
nate/cleanup-contract-roots
Sep 17, 2026
Merged

ChrisSchinnerl merged 5 commits into
masterfrom
nate/cleanup-contract-roots

Conversation

@n8mgr

@n8mgr n8mgr commented Sep 15, 2026

Copy link
Copy Markdown
Member

Fixes a memory leak where the in-memory sector root cache kept the roots of every resolved, renewed, and rejected contract for the lifetime of the process.

@n8mgr
n8mgr requested review from ChrisSchinnerl, chris124567 and peterjan and a lite review from Copilot September 15, 2026 16:52
@n8mgr
n8mgr force-pushed the nate/cleanup-contract-roots branch from 039a40a to ce88512 Compare September 15, 2026 16:53
@github-project-automation github-project-automation Bot moved this to In Progress in Sia Sep 15, 2026
@n8mgr n8mgr moved this from In Progress to In Review in Sia Sep 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical stale-cache issue remains when rejected contracts are reset, along with a migration-test comment nit.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Fixes a memory leak by evicting cached sector roots after contract lifecycle transitions.

Changes:

  • Adds lifecycle tracking and migration support.
  • Implements synchronized cache expiration.
  • Updates persistence, tests, and release notes.
File summaries
File Summary
persist/sqlite/migrations.go Adds schema migration.
persist/sqlite/migrations_test.go Tests migration backfill. Nit (1 vote): Correct misplaced migration test comment.
persist/sqlite/init.sql Updates the current schema.
persist/sqlite/contracts.go Adds update tracking and expiration queries.
persist/sqlite/consensus.go Tracks lifecycle transition indices.
host/contracts/update.go Triggers cache expiration.
host/contracts/persist.go Extends the persistence interface.
host/contracts/manager.go Integrates the roots cache.
host/contracts/manager_test.go Tests cache lifecycle behavior.
host/contracts/lock.go Uses the cached roots.
host/contracts/integrity.go Uses the cached roots.
host/contracts/cache.go Implements synchronized root cleanup. Critical (1 vote): Resetting rejected contracts can leave stale cached roots attached to the replacement contract.
.changeset/release_cached_sector_roots.md Documents the release change.
Review details

Suppressed comments (1)

persist/sqlite/migrations_test.go:770

  • The comment immediately above this newly inserted test still describes TestMigrateV55, so TestMigrateV56 is documented as testing the previous migration and the actual v55 test below has no header. Please move the v55 comment below this test and add a v56 description here.
func TestMigrateV56(t *testing.T) {
  • Files reviewed: 13/13 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread host/contracts/cache.go Outdated
Copilot AI review requested due to automatic review settings September 15, 2026 17:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two critical lifecycle and reorg-buffer issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

persist/sqlite/migrations_test.go:770

  • The existing TestMigrateV55 doc comment now immediately precedes TestMigrateV56, so it documents the wrong test and leaves the v55 test undocumented. Move this new test below TestMigrateV55, or update/add the comments so each test describes its own migration.
func TestMigrateV56(t *testing.T) {
  • Files reviewed: 13/13 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread host/contracts/manager.go
Comment thread persist/sqlite/consensus.go
Copilot AI review requested due to automatic review settings September 15, 2026 17:39
@n8mgr
n8mgr force-pushed the nate/cleanup-contract-roots branch from 750e460 to eeb750c Compare September 15, 2026 17:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Terminal roots are not restored after restart/reorg, and failed-resolution reverts can leave stale failed state.

Review details

Suppressed comments (2)

host/contracts/manager.go:386

  • Because roots comes from V2SectorRoots, which excludes resolved and rejected contracts, this cache is empty for terminal contracts after every restart. If a restart is followed by a reorg within the reorg buffer, the chain update can restore such a contract to pending/active, but SectorRoots still returns nil, so validation or renewal loses the contract's sectors. Recent terminal roots need to be persisted or otherwise restored across startup.
	cm.roots = &rootsCache{
		store:             store,
		contractSectors:   roots,
		lastExpiredHeight: expireHeight,

persist/sqlite/consensus.go:1718

  • The added last_updated_* update is unreachable for failed-resolution reverts: revertFailedV2Contracts hits the continue in its state.Status == Failed branch before executing this statement. A failed proof/expiration that is reorged therefore remains marked failed with its old resolution height, so ExpireContracts can delete its cached roots even though the contract should be active again. This path needs to perform the active-state update (and timestamp it with index) before the cache expiry logic runs.
	updateStmt, err := tx.Prepare(`UPDATE contracts_v2 SET resolution_block_id=NULL, resolution_height=NULL, contract_status=?, last_updated_height=?, last_updated_block_id=? WHERE id=?`)
  • Files reviewed: 13/13 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings September 15, 2026 20:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Comment thread persist/sqlite/contracts.go Outdated
Comment thread persist/sqlite/migrations.go Outdated
Comment thread host/contracts/cache.go
Comment thread host/contracts/cache.go
@github-project-automation github-project-automation Bot moved this from In Review to In Progress in Sia Sep 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 16, 2026 21:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Shallow-tip expiry can remove roots before the reorg buffer has elapsed.

Review details

Suppressed comments (1)

host/contracts/manager.go:373

  • When the stored tip is at or below ReorgBuffer, this leaves expireHeight equal to the tip instead of zero. A restart at a shallow height therefore calls V2SectorRoots with a minimum that excludes resolved contracts from the current reorg window (and height exactly ReorgBuffer excludes every resolution below that height), so their roots disappear from the in-memory cache before the buffer has elapsed. Initialize the expiry height to zero and only subtract the buffer when the tip is higher than it.
	expireHeight := tip.Height
	if expireHeight > ReorgBuffer {
		expireHeight -= ReorgBuffer
	}
  • Files reviewed: 15/15 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@ChrisSchinnerl
ChrisSchinnerl merged commit ae0b653 into master Sep 17, 2026
14 of 15 checks passed
@ChrisSchinnerl
ChrisSchinnerl deleted the nate/cleanup-contract-roots branch September 17, 2026 12:15
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Sia Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants