Skip to content

fix(seed): wait for L2/L3 before closing pipeline - #645

Open
RerankerGuo wants to merge 1 commit into
TencentCloud:mainfrom
RerankerGuo:fix/issue-115-seed-drain-l2-l3
Open

fix(seed): wait for L2/L3 before closing pipeline#645
RerankerGuo wants to merge 1 commit into
TencentCloud:mainfrom
RerankerGuo:fix/issue-115-seed-drain-l2-l3

Conversation

@RerankerGuo

Copy link
Copy Markdown

Description | 描述

Fix the remaining seed lifecycle defect from #115: the seed runtime previously waited only for L1 to become idle, then called the gateway-oriented destroy() path. That path has an intentional 2-second shutdown budget and marks the scheduler destroyed before flushing, so a scheduled/running L2 extraction or the L3 persona task triggered by L2 could be skipped or interrupted before artifacts reached disk.

This PR adds a finite-batch completion barrier:

  • Add MemoryPipelineManager.flushAll() to drain pending L1 work, flush scheduled L2 work, then wait for L3.
  • Wait for L2 and L3 sequentially. The old concurrent wait could observe an initially-idle L3 queue before L2 enqueued the L3 task.
  • Cancel recurring L2 timers after the finite batch drain, preventing the following destroy() call from triggering a duplicate L2 pass.
  • Replace seed's final L1-only polling barrier with flushAll() so scene and persona artifacts are complete before storage resources close.
  • Add a regression test with independently blocked L2 and L3 runners. It proves the barrier does not resolve during either phase and that destroy does not run a second extraction afterward.

Scope boundary: this does not change the JSON correction/retry behavior already covered by #190, or the seed tail-batch flush behavior covered by #507.

Related Issue | 关联 Issue

Refs #115

Change Type | 修改类型

  • Bug fix | Bug 修复
  • New feature | 新功能
  • Documentation update | 文档更新
  • Code optimization | 代码优化

Self-test Checklist | 自测清单

  • Verified locally | 本地验证通过
  • No existing features affected | 无影响现有功能

Additional Notes | 其他说明

Validation (Node v26.5.0, npm v11.17.0):

npm test -- src/utils/pipeline-manager.test.ts
# 1 file, 1 test passed

npm test
# 5 files, 68 tests passed

npm run build
# plugin bundle + all three script TypeScript builds passed

git diff --check
# passed

No configuration, storage schema, or public HTTP API changes.

Expose a full pipeline flush barrier that waits for L1, then L2, then the L3 work enqueued by L2. Cancel recurring L2 timers after a finite batch drain so destroy does not trigger a duplicate extraction pass.

Use the barrier at the end of seed imports to ensure scene and persona artifacts are persisted before storage closes. Add a regression test that holds L2 and L3 independently and proves flushAll does not resolve early.

Refs TencentCloud#115

Signed-off-by: RerankerGuo <121015044+RerankerGuo@users.noreply.github.com>
@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for your interest and contributions! We will review your code and get back to you as soon as possible!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants