fix(worker): PR1 卡死/失效 Critical 修复 6 项(限流器+id查询+TokenBucket+daily_limit+last_run_at+补偿) - #46
Merged
Merged
Conversation
1a 跨进程共享限流器:rate_limiter.py TokenBucket.acquire 改 fcntl.flock +
共享状态文件(pm_rate_{api}.state),api/worker 两容器共享同一令牌桶,
此前各持内存桶实际速率翻倍易触发 429。无写权限时降级进程内内存桶(行为同旧版)。
新增 rate_limiter_state_dir 配置(默认 /app/data 共享卷)。
1b ingest.py id: 查询改 fetch_by_ids:arxiv 不支持 id: 作为 search_query
前缀,逐个 fetch_latest(query=f"id:{mid}") 永远拿不到目标论文。改一次批量
fetch_by_ids(走 id_list 参数,正确入口)。
1c cs_feed TokenBucket 超时 bug:acquire 超时判定用 last_refill(每次 _refill
更新为 now),time.time()-last_refill 永远≈0 永不超时 → arxiv 全局限流时
acquire 卡死 worker 线程。改用 start_time 判定。
1d daily_limit 累加 + 跨天重置:cs_feed update_run_status 此前
last_run_count = count(覆盖),当日多次抓取重置配额绕过 daily_limit。
改累加;跨天先清零,避免昨天余量带进今天。
1e TopicSubscription 加 last_run_at/last_error:抓取失败此前静默无痕无法
补抓。加字段 + alembic 迁移(幂等)+ TopicRepository.update_run_status +
worker topic_dispatch_job 成功/失败后更新。
1f idle_processor skimmed 论文补偿:skim 后 read_status 变 skimmed、
deep_dive_md 仍空的论文此前无人再触发精读,永久卡在 skimmed。新增
_get_stuck_skimmed_papers + _compensate_stuck_skimmed(配额受限,默认2),
在 skim 批次跑完后单独补一次精读。
测试:新增 TopicRepository.update_run_status(持久化+截断+清空)、
CSFeedRepository.update_run_status(当日累加+跨天重置+未知静默)共 5 测试,
全套 50 passed。
🔍 OpenCode PR Review Required这是一个受保护的分支,merge 前需要进行 code review。 请运行以下命令进行 OpenCode review: 或者在 PR 页面评论 This is an automated reminder from PR Review Gate. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
Worker 抓取逻辑审计发现 6 个 Critical 问题,本 PR 为修复 3 PR 方案的 PR1(卡死/失效)。
文件集与 PR2/PR3 互斥,顺序合入。
修复清单
1a 跨进程共享限流器(Critical #5)
packages/ai/rate_limiter.pyTokenBucket.acquire改用 fcntl.flock + 共享状态文件(pm_rate_{api}.state)。此前 api/worker 两个容器各持一份进程内内存桶,实际速率是配置值的 2 倍,arxiv 全局限流时容易触发 429。现在两进程通过共享卷上的状态文件共享同一桶状态。acquire_api签名不变)rate_limiter_state_dir配置,默认/app/data(docker-compose 两服务共享的 pm_data 卷)验证:两进程共享同一桶文件 1 秒内共获取 4 个令牌(各自独立桶则会 14),跨实例
b1消耗 3 个后b2可见 2.0。1b ingest.py
id:查询改 fetch_by_ids(Critical #1)arxiv 不支持
id:作为 search_query 前缀,逐个fetch_latest(query=f"id:{mid}")永远拿不到目标论文。改一次批量fetch_by_ids(走 id_list 参数,正确入口),N 次 acquire_api 合并成 1 次。1c cs_feed TokenBucket 超时 bug(Critical #2)
acquire超时判定用last_refill(每次_refill更新为 now),time.time()-last_refill永远≈0 永不超时 → arxiv 全局限流时 acquire 卡死 worker 线程。改用start_time判定,与 rate_limiter.py 的 TokenBucket 对齐。1d daily_limit 累加 + 跨天重置(Critical #3)
cs_feed.update_run_status此前last_run_count = count(覆盖),当日多次抓取重置配额绕过 daily_limit。改累加;跨天先清零,避免昨天余量带进今天。1e TopicSubscription 加 last_run_at/last_error(Critical #4)
抓取失败此前静默无痕无法补抓。加
last_run_at/last_error字段 + alembic 迁移(幂等,PGADD COLUMN IF NOT EXISTS/ SQLite try/except,对齐 c3d4e5f6a7b8 写法)+TopicRepository.update_run_status+ workertopic_dispatch_job成功/失败后更新。1f idle_processor skimmed 论文补偿(Critical #6)
skim 后 read_status 变 skimmed、deep_dive_md 仍空的论文此前无人再触发精读,永久卡在 skimmed。新增
_get_stuck_skimmed_papers(挑 skimmed + 有 summary_md + deep_dive_md 空)+_compensate_stuck_skimmed(配额受限deep_read_compensation,默认 2),在 skim 批次跑完后单独补一次精读。繁忙时中止补偿,避免与用户请求争抢 LLM。验证
e5f6a7b8c9d0不做(诚实标注)
_active_tasks/_current_slot/_rate_limit_errors仍进程私有:纯限流只共享 bucket 即可,这些进程私有状态不影响限流正确性部署影响
alembic upgrade head(新迁移幂等)RATE_LIMITER_STATE_DIR(可选,默认 /app/data 已挂载)DEEP_READ_COMPENSATION(可选,默认 2)