Skip to content

doc: 细化文档#1

Merged
GatewayJ merged 7 commits intoDemoio:mainfrom
GatewayJ:doc
Mar 2, 2026
Merged

doc: 细化文档#1
GatewayJ merged 7 commits intoDemoio:mainfrom
GatewayJ:doc

Conversation

@GatewayJ
Copy link
Member

@GatewayJ GatewayJ commented Feb 27, 2026

Note

Medium Risk
Adds a new Rust workspace with multiple service binaries and protobuf codegen; while most service logic is still stubbed, build/CI and project structure changes can break compilation and developer workflows.

Overview
Shifts the project from a single crate into a multi-crate Cargo workspace (proto/common/metadata/gateway/scheduler/cache/tape) with shared workspace dependencies and updated developer tooling (Makefile, .gitignore).

Introduces protobuf + tonic gRPC scaffolding via a new coldstore-proto crate (with build.rs and .proto definitions for Metadata/Scheduler/Cache/Tape) and adds initial service binaries for Gateway, Scheduler, Metadata, Cache, and Tape (mostly todo!() implementations).

Updates CI to run fmt/clippy/build/test for the whole workspace, installs protoc, enables cargo caching, and refreshes documentation to align on the “pure cold archive” model and the new component split; removes the old example config.yaml.example.

Written by Cursor Bugbot for commit cb0da38. This will update automatically on new commits. Configure here.

- 框架: Vue 3 → React 18 + TypeScript
- UI 库: Ant Design Vue → Ant Design 5
- 状态管理: Pinia → Zustand
- 路由: Vue Router → React Router 6
- 文件后缀: .vue → .tsx
- 新增 hooks/ 目录 (usePolling, useWebSocket)

Made-with: Cursor
validate_restore_transition 缺少该转换,导致 OfflineRequest
超时后无法将 RecallTask 标记为 Failed,任务会永久卡住。

Made-with: Cursor
- 补充 PutObject 数据暂存流程:Cache Worker 作为归档前的临时暂存区,
  新增 StagingWriteApi/StagingReadApi 接口定义(04, 05, DESIGN.md)
- 纠正 Admin Console 架构:Console 仅连 Metadata,运行时数据通过
  Worker 心跳上报(09)
- 移除 observability 中残留的"热对象"代码分支(08)
- 统一归档扫描间隔 scan_interval_secs 为 60s(05)
- 补充 DeleteObject 对已归档磁带数据的处理策略与 Tape Compaction(05)
- 新增多 Scheduler Worker 扩展占位章节(05)
- 完善 S3 Multipart Upload 协议适配设计(02)
- 精确定义 ArchiveBundle 中 FileMark 的使用方式(05)
- 修复 04-cache-layer 章节编号错误(05→6, 6→7 等)
- 改进心跳 last_heartbeat Leader 切换处理:新增宽限期机制(03)

Made-with: Cursor
- scripts/pre-commit: 可提交的 hook 脚本,依次运行 cargo fmt --check、
  clippy -D warnings、cargo check、cargo test
- Makefile: 新增 install-hooks / lint / setup targets
- proto lib.rs: 允许 tonic-build 生成代码的 doc_lazy_continuation lint

团队成员 clone 后执行 `make setup` 即可安装 hook。

Made-with: Cursor
将单体 src/ 结构迁移为 crates/ 多 crate workspace:

新增 crates:
- proto: gRPC 接口定义 (5 个 .proto 文件) + tonic-build 生成
- common: 共享配置、错误处理、领域模型
- metadata: Metadata 节点 (Raft + RocksDB) 二进制
- gateway: S3 HTTP 接入层二进制
- scheduler: 归档/取回调度器二进制
- cache: Cache Worker 二进制 (HDD 暂存后端)
- tape: Tape Worker 二进制

删除:
- src/ 旧单体代码
- config/config.yaml.example (各 crate 内置默认配置)

更新:
- Cargo.toml: workspace 根配置 + 共享依赖
- .gitignore, README.md, README_ZH.md, README_RUST.md

Made-with: Cursor
- 添加 protobuf-compiler 安装步骤,修复 tonic-build 找不到 protoc 的错误
- 使用 dtolnay/rust-toolchain 安装 stable + rustfmt/clippy
- 添加 cargo cache 加速构建
- 增加 fmt check 和 clippy 检查步骤
- 触发分支增加 doc

Made-with: Cursor
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

base_path: base,
max_size_bytes: max_size_gb * 1024 * 1024 * 1024,
next_id: AtomicU64::new(1),
})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ID counter resets on restart, causing data overwrites

High Severity

HddBackend::new always initializes next_id to 1, without scanning existing files in the meta/ directory. After a process restart, write will allocate IDs starting from 1 again via fetch_add, silently overwriting existing cached data and metadata files on disk. The list_all method exists and is documented as "启动时重建索引用" (rebuild index on startup), but it's never called during construction to recover the maximum existing ID.

Additional Locations (1)

Fix in Cursor Fix in Web


async fn available_bytes(&self) -> Result<u64> {
Ok(self.max_size_bytes)
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

available_bytes ignores actual disk usage

Medium Severity

available_bytes always returns self.max_size_bytes (the total configured capacity) instead of subtracting the space currently used by cached objects. Any consumer relying on this method for capacity-based eviction decisions will believe the cache is always empty and never trigger eviction, eventually filling the disk.

Fix in Cursor Fix in Web


# Test data
test_data/
/data
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitignore drops common IDE and OS file patterns

Low Severity

The rewrite of .gitignore removed entries for .DS_Store, Thumbs.db, .idea/, .vscode/, *~, *.rs.bk, and *.pdb. These common IDE configuration directories and OS-generated files are no longer ignored, risking accidental commits of developer-local artifacts into the repository.

Fix in Cursor Fix in Web

@GatewayJ GatewayJ merged commit 9e942bb into Demoio:main Mar 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant