feat(conf,server,db): add config file hot-reload without restart - #2924
Closed
Warm-winter wants to merge 5 commits into
Closed
feat(conf,server,db): add config file hot-reload without restart#2924Warm-winter wants to merge 5 commits into
Warm-winter wants to merge 5 commits into
Conversation
- Add fsnotify-based watcher in internal/conf/watcher.go to monitor config.json changes with SHA-256 dedup and 150ms debounce - Add reload engine in internal/bootstrap/reload.go that rebuilds the config offline and atomically swaps the global config pointer - Wire watcher startup and shutdown into bootstrap run lifecycle - Extract HTTP/HTTPS/HTTP3/Unix/S3/FTP/SFTP endpoint start logic into independently restartable functions, restarting only affected endpoints on scheme/s3/ftp/sftp config changes - Apply side effects for hot-reloadable fields without restart Co-authored-by: DeepSeek-V4-PRO-0813 <noreply@deepseek.com>
- Register MCP routes unconditionally and guard with request-time middleware that reads conf.Conf.MCP.Enable, returning 403 when disabled to match prior placeholder behavior - Apply the same guard pattern to S3 routes on the main HTTP server so s3.enable toggles take effect without restart - Read CORS, max_connections, and force_https settings at request time in middlewares instead of baking them into gin handlers Co-authored-by: DeepSeek-V4-PRO-0813 <noreply@deepseek.com>
- Reinitialize database connection when database config changes - Reinitialize bleve and meilisearch search engines on config changes - Reload static dist_dir when dist_dir config changes - Add graceful delayed shutdown for old instances Co-authored-by: DeepSeek-V4-PRO-0813 <noreply@deepseek.com>
- Add internal/db/sqlite_driver_glebarez.go using glebarez/sqlite (pure Go) as default when sqlite_cgo_compat tag is absent - Add internal/db/sqlite_driver_gorm.go using gorm sqlite (cgo) under sqlite_cgo_compat tag or legacy platforms (mips/loong64/win386) - Enable CGO_ENABLED=0 cross-compilation for Windows/macOS/Linux-musl Co-authored-by: DeepSeek-V4-PRO-0813 <noreply@deepseek.com>
- Add sed CRLF-to-LF conversion for build.sh and entrypoint.sh in Dockerfile builder stage to fix bash parse errors on Windows checkouts - Apply the same conversion to entrypoint.sh in the runtime stage Co-authored-by: DeepSeek-V4-PRO-0813 <noreply@deepseek.com>
Member
|
OpenList 当前将 config.json 视为启动时配置，而不是运行时动态配置接口。支持 HotReload 不只是增加 watcher，而是需要让路由、中间件、listener、数据库、搜索引擎等所有相关 subsystem 都具备完整的运行时重配置语义。这个 PR 本身的大范围改动已经体现了这一点。 一旦支持这种模型，今后每一个配置项都必须长期考虑 runtime reload、并发访问、旧请求与新配置的一致性、失败回滚以及 subsystem 生命周期问题。这会成为整个项目的长期架构约束，我们目前不希望引入这个设计。 当前实现也仍存在 reload 非事务化的问题：全局配置先被替换，再逐项应用副作用，因此 reload 过程中或部分操作失败时可能出现新旧状态混合；部分全局对象的并发读取/替换也缺少明确的同步边界。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary / 摘要
This PR implements hot-reload for
config.json, allowing most configuration changes to take effect without restarting the OpenList server. Previously, modifying fields such asscheme.*,s3.*,ftp.*,sftp.*,mcp.enable,cors.*,max_connections,dist_dir, database, and search engine settings required a full service restart.此 PR 为
config.json实现热重载机制，使绝大多数配置变更无需重启服务即可生效。此前修改scheme.*、s3.*、ftp.*、sftp.*、mcp.enable、cors.*、max_connections、dist_dir、数据库及搜索引擎等设置均需完整重启服务。新增基于 fsnotify 的配置文件监听器（SHA-256 去重 + 150ms 防抖）
新增离线重载引擎，原子交换全局配置指针并应用副作用
将 MCP/S3 路由改为始终注册 + 请求期守卫中间件，支持启用状态即时切换
将 CORS / max_connections / force_https 中间件改为请求期实时读取配置
抽取 HTTP/HTTPS/HTTP3/Unix/S3/FTP/SFTP 监听器为可独立启停的函数，配置变化时仅重启受影响端点
支持数据库、搜索引擎（bleve/meilisearch）、静态资源目录的子系统重初始化
新增 glebarez 纯 Go sqlite 驱动作为 cgo 不可用时的回退，支持 CGO_ENABLED=0 交叉编译
修复 Dockerfile 在 Windows 检出下因 CRLF 行尾导致的 bash 解析错误
This PR has breaking changes.
This PR changes public API, config, storage format, or migration behavior.
This PR requires corresponding changes in related repositories.
Related repository PRs / 关联仓库 PR:
OpenList-Frontend: N/A
OpenList-Docs: 可选——建议补充热重载机制说明文档
Related Issues / 关联 Issue
N/A（无直接关联 Issue；如需要可创建 feature request issue 关联）
Testing / 测试
go test ./...Manual test / 手动测试:
测试环境：WSL Ubuntu-24.04 + Docker v29.3.1，OpenList v4.2.5
Docker 镜像构建：使用项目 Dockerfile 构建
openlist:local镜像（151MB），/ping返回pong，版本信息正确MCP 热重载验证：容器内修改
config.json的mcp.enable：false→GET /mcp返回{"code":403,"message":"MCP server is not enabled"}改为
true→ 返回{"code":401,"message":"Guest user is disabled, login please"}（守卫通过，进入认证）改回
false→ 再次返回 403全程无需重启容器，2 秒内生效
浏览器全量功能测试（6 项全部通过，无控制台错误）：
多平台交叉编译：Windows amd64（PE32+）、macOS amd64/arm64（Mach-O）、Linux musl amd64/arm64（静态链接 ELF）均通过
file命令验证测试截图和详细日志见本地验证记录（未随 PR 提交）。
Checklist / 检查清单
I have read https://github.com/OpenListTeam/OpenList/blob/main/CONTRIBUTING.md .
I confirm this contribution follows the repository license, contribution policy, and code of conduct.
I have formatted the changed code with
gofmt,go fmt, orprettierwhere applicable.I have requested review from relevant maintainers or code owners where applicable.
AI Disclosure / AI 使用声明
This PR includes AI-assisted content.
Tools used / 使用工具:
ChatGPT
Codex
GitHub Copilot
Claude
Other (please specify) / 其他（请注明）: Co-authored-by: DeepSeek-V4-PRO-0813 noreply@deepseek.com
（通过 DeepSeek Harness 辅助开发）
Usage scope / 使用范围:
Code generation / 代码生成
Refactoring / 重构
Documentation / 文档
Tests / 测试
Translation / 翻译
Review assistance / 审查辅助
I have reviewed and validated all AI-assisted content included in this PR.
I have ensured that all AI-assisted commits include
Co-Authored-Byattribution.I can reproduce all AI-assisted content included in this PR without any AI tools.