Qt s0, qt_example partially merged in this branch by shepherdpp · Pull Request #272 · shepherdpp/qteasy

shepherdpp · 2026-05-11T09:08:43Z

目标：将 Trader 升级为可脱离 UI 稳定长期运行的实盘核心运行时，具备可控任务管理、可编排日程、可插拔 Broker、可审计本地记录与日志。
本轮路线：
运行时模型采用“单线程事件循环为核心，耗时 I/O 放受控 worker”。
Broker 重构采用“兼容层渐进迁移”，避免一次性切换风险。
非目标（本计划外）：一次性重写全部 UI、一次性替换所有历史 Broker 实现。
读者反馈吸收（原则层，非 qteasy_ext 实现承诺）：独立线程 ≠ 主链路可阻塞；process_trade_result 须事务化以防半提交脏账；Trader 仅通过 Broker 公开 API 消费回报/消息；任务重入与 skip 须有枚举原因便于分桶诊断；日程与「启动前校验 gate」解耦，避免带病启动污染策略输入。

…g and add tests for index_weight - Modified the 'index_weight' entry in `data_channels.py` to change the parameters for better clarity and usage. - Updated the `composite` function in `tsfuncs.py` to reflect the new parameter name for index handling. - Added unit tests in `test_channels.py` to validate argument parsing for the 'index_weight' table, ensuring correct handling of date ranges and index parameters. - Adjusted the setup method in `test_eastmoney.py` to configure the environment for testing.

…and functionality - Updated the TraderShell class to streamline the shutdown process, replacing direct task management with a more cohesive stop method. - Refactored the Trader class to implement dedicated start, stop, and join methods for managing trader and broker threads, enhancing thread lifecycle management. - Modified the CLI and TUI implementations to utilize the new start and stop methods, ensuring consistent behavior across interfaces. - Added unit tests to validate the shutdown process and runtime management, confirming correct interactions with the Trader's lifecycle methods.

…ror handling - Introduced a new `TaskSpec` dataclass to encapsulate task details, including retry logic and status tracking. - Updated task processing logic to utilize `TaskSpec`, improving clarity and maintainability. - Implemented enhanced error handling for task failures, including retry mechanisms and dead-letter task management. - Added unit tests to validate task management features, including task addition, cancellation, and failure handling.

…nhance task scheduling in Trader class - Added a static method `_build_schedule_time_kwargs_from_config` in the `Operator` class to construct trading session parameters from a configuration dictionary, improving the preparation of running schedules. - Refactored the `Trader` class to utilize the new method for generating schedule time arguments, enhancing clarity and maintainability in task scheduling. - Introduced a new `create_daily_task_plan` function in `trading_util.py` to standardize daily task creation, replacing the previous `create_daily_task_schedule` function. - Updated unit tests to validate the new scheduling features and ensure correct mapping of configuration to schedule parameters.

… support and new broker result handling - Introduced a `db_run_in_transaction` method in the `DataSource` class to facilitate transaction management, allowing for commit and rollback operations. - Updated existing database execution methods to support transaction reuse, improving efficiency and consistency in database interactions. - Enhanced trade recording functions to include a `transaction` parameter, enabling atomic updates for account balances, positions, and trade orders. - Added a new `broker_result_id` field in the trade results schema to ensure unique identification of trade results and prevent duplicate processing. - Implemented unit tests to validate transaction behavior and idempotency of trade result processing, ensuring robustness in trade handling.

- Resolved issues in the `realize` method to improve parameter validation and trading signal calculations. - Ensured proper handling of multi-parameter scenarios in the `RuleIterator` class for better state updates. - Enhanced data processing to check for NaN values in historical price data before generating trading signals.

- Introduced seven new Jupyter notebook examples demonstrating various trading strategies, including dual moving average, alpha stock selection, and grid trading. - Each example includes detailed markdown explanations, code implementations, and backtesting results for clarity and usability. - Strategies cover a range of methodologies, from multi-factor models to cross-asset arbitrage, enhancing the educational resources available for users transitioning to JoinQuant.

- Added references to new Jupyter notebook sources in the markdown files for various trading strategies, enhancing clarity and usability. - Updated the .gitignore file to exclude additional file types and directories related to the migration process, ensuring a cleaner repository. - Improved organization of example files to facilitate easier access and understanding for users transitioning to JoinQuant.

- Updated `create_headless_notebook_session` to include parameters for real-time trading and isolated data sources, improving flexibility for testing scenarios. - Added functions for configuring the notebook operator and preparing isolated replay data, enhancing the setup process for testing. - Improved documentation within the script to clarify usage and parameters, aiding user understanding and facilitating smoother integration into workflows.

- Added '/sql/' to .gitignore to exclude SQL files from version control. - Improved column validation in the DataSource class to prevent KeyError by checking for the existence of specified columns before filtering data. - Updated documentation in datatables.py to clarify the structure definitions and improve consistency in naming conventions.

…onfiguration and timeout options - Added functions to set the repository root for subprocesses and construct the environment for unittest execution, improving the flexibility of the testing setup. - Introduced a timeout parameter for subprocess execution in the `stage1_preflight_tests` function, allowing users to specify maximum wait times for tests. - Updated documentation to clarify the new parameters and their usage, enhancing user understanding and integration into workflows.

shepherdpp · 2026-05-11T09:11:36Z

架构原则与验收锚点

以下原则作为全计划与后续各阶段的验收锚点：除各阶段条目验收外，不得与已承诺原则相冲突；若某阶段暂不实现，须在计划或 PR 中显式标注 deferred 并说明风险接受范围。

时间与日历（Time & Calendar）
原则：明确时钟/时区/交易日历来源；子日频任务须定义错过窗口是否补跑、补跑次数、是否合并，以及盘中启动（catch-up）与网络抖动下的行为边界。
验收锚点：文档或代码注释中可追溯到统一参数源；同一日期与配置下日程输出可复现；盘中启动与“已过时刻”策略有测试或清单覆盖（与阶段 3 对齐）。
订单与回报幂等（Idempotency & De-duplication）
原则：贯穿 client_order_id（或等价键）；重复提交、重复回报、乱序回报下，process_result / 账本更新须幂等（重复处理不改变最终正确状态）。账本写路径须具备原子性（单事务边界），避免任一步失败留下半提交状态。
验收锚点：process_trade_result 相关写表在同一 DB 事务内提交或整体回滚；幂等键（如 result_id + delivery_status 或等价组合）在实现中写明；至少具备“同一回报处理两次”“乱序到达”的回归用例（阶段 3.5 优先收口，阶段 4/5 延续 Broker 与长稳场景）。
真相源与对账（Source of Truth & Reconcile）
原则：写明权威状态以券商回报还是本地模拟为准；周期性 reconcile（持仓/资金/在途单）为一等能力，非仅人工偶发对账。
验收锚点：存在可调用的对账入口或任务钩子；对账结果可记录（日志或表），差异有分级处理策略（与阶段 4、5 对齐）。
背压、限流与降级（Backpressure & Degradation）
原则：队列满、Broker 限速、行情源失败时须有明确策略（丢弃/合并/降频/只读等），避免无界入队拖垮进程。
验收锚点：任务队列或 worker 有上限或可观测深度；异常路径下系统可恢复至安全态（与阶段 2 为主，阶段 0 可观测性配合）。
安全与凭证（Security & Secrets）
原则：凭证不落日志、可轮换；Broker 会话/连接生命周期与 DataSource 重连解耦（各自失败不隐含对方健康）。
验收锚点：敏感字段脱敏规则或审查清单；连接/断连/重试在日志中可区分 Broker 与数据源（与阶段 4 对齐）。
可测试性（Determinism & Replay）
原则：长期目标支持可注入时钟、可重放或注入的行情/回报流，以便 CI 锁住单线程事件循环语义。
验收锚点：关键调度与任务分发路径具备不依赖真实网络的单元测试；新增行为优先补测试再改实现（各阶段持续）。
运维可观测性（Beyond File Logs）
原则：除 DEBUG/INFO 文件外，保留 health / 就绪语义与关键指标扩展位（如队列深度、任务延迟、Broker 延迟、拒单率），便于无人值守运维。
验收锚点：至少一种可机器消费的健康或指标出口（可先为结构化日志字段或轻量接口），不强制一期上完整 Prometheus（与阶段 0、5 对齐）。
人为干预与安全开关（Kill Switch）
原则：实盘须具备 kill switch / 仅平仓 / 禁止新开仓等与任务白名单、风控层的闭环，避免仅依赖粗粒度 pause。
验收锚点：开关状态可观测、可测试；触发后下单路径被阻断且记录审计（与阶段 2、现有 Risk 集成对齐）。
演进与兼容性（Schema & Checkpoint Versioning）
原则：本地表结构、断点文件、日志格式演进时须考虑版本化或向后兼容策略，避免升级后静默损坏状态。
验收锚点：破坏性变更伴随迁移说明或版本字段；文档中写明兼容承诺范围（与阶段 5 对齐）。

- Introduced a new test file `test_notebook_headless_integration.py` to validate the functionality of the headless notebook trader script. - Implemented tests for session creation and operator group validation, ensuring correct initialization and task scheduling. - Added checks for idempotent cancellation and delivery processes in the notebook's trading phases, enhancing regression testing capabilities. - Utilized isolated data sources to ensure test reliability and prevent interference with real-time data.

- Implemented a new test suite in `test_notebook_headless_integration.py` to validate the functionality of the headless notebook trader script. - Added tests for session creation, operator group validation, and task scheduling to ensure correct initialization. - Included checks for idempotent cancellation and delivery processes in trading phases, enhancing regression testing capabilities. - Utilized isolated data sources to improve test reliability and prevent interference with real-time data.

…ry policies - Updated the Broker class to include a new method `poll_messages` for asynchronous message retrieval, improving communication handling. - Modified the `poll_fills` method to check the legacy result queue when no pending fills are available, enhancing compatibility. - Introduced reentry policies for task management in the Trader class, allowing for more flexible task execution strategies. - Updated unit tests to validate the new message polling functionality and task reentry behavior, ensuring robustness in trading operations.

…ult_id column Co-authored-by: Cursor <cursoragent@cursor.com>

…proved shutdown process - Updated the `stage5_stage0to3_regression` function to include checks for broker API availability and detailed statistics on skipped and rejected tasks, enhancing diagnostic capabilities. - Modified the shutdown process to align with the TraderRuntime semantics, ensuring a more graceful termination of the trader session. - Improved logging to provide clearer insights into the state of the trader and its tasks during execution.

…(SettingWithCopyWarning) Co-authored-by: Cursor <cursoragent@cursor.com>

…y failures Co-authored-by: Cursor <cursoragent@cursor.com>

…atched table fetch - Add _hist_dnld_thread_pool_max_workers(); 0 maps to 1 for ThreadPoolExecutor - fetch_real_time_klines parallel path reads QT_CONFIG hist_dnld_parallel - fetch_batched_table_data uses same when process_count is None - Update hist_dnld_parallel config text; add TestHistDnldParallelWorkers Co-authored-by: Cursor <cursoragent@cursor.com>