Skip to content

关闭产品级最大行数限制#3003

Open
superfastbuild wants to merge 5 commits into
t8y2:mainfrom
superfastbuild:main
Open

关闭产品级最大行数限制#3003
superfastbuild wants to merge 5 commits into
t8y2:mainfrom
superfastbuild:main

Conversation

@superfastbuild

Copy link
Copy Markdown

变更说明

关闭了对 MAX_RESULT_PAGE_SIZE 做产品级收紧。当前最大可到 100,000 行,配合宽表和 DataGrid 派生数组会非常容易冲高内存的建议,有用户自定义设置可显示多少行

变更类型

  • 新功能
  • Bug 修复
  • [✅] 性能优化
  • 代码重构
  • 文档更新
  • CI / 构建

涉及前端

  • [✅] 本 PR 涉及前端改动,已附截图/录屏(见下方)
image image image

验证

编写sql查询后第一次查询10万行,在分页出设置999999,再次查询可正常查询出数据。

  • make check 通过
  • make cargo-check-fast 通过
  • [对] 相关测试通过

关联 Issue

Close #2478

@t8y2 t8y2 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这块我先 request changes,主要是 CI 现在已经红了。

packages/app-tests/settingsStore.test.ts:47 还在验证保存的 pageSize: 200000 会被归一化到 100000,但这次改动后会直接返回 200000。如果去掉 100,000 上限是预期行为,需要同步更新这个测试和对应的行为约定;如果不是预期,就需要恢复归一化里的上限保护。

另外顺手把注释掉的 import/constant 删除掉,并把 docs/performance/memory-usage-audit.md 里相邻两条互相冲突的建议合并一下会更干净。修完后我再 review 一下。

@superfastbuild superfastbuild requested a review from t8y2 July 10, 2026 01:05
@superfastbuild

Copy link
Copy Markdown
Author

已完成全部修改:

  1. normalizeResultPageSize 移除 pageSize 100000 强制上限,支持更大分页满足全量查询场景
  2. 更新 settingsStore.test.ts 对应单元测试
  3. 清理无用注释import与常量
  4. 合并 memory-usage-audit.md 冲突的性能说明
    麻烦再次 review,感谢。

@t8y2 t8y2 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

request changes: 上次提到的 CI 和测试问题已经修复,但目前还有两个正确性问题。

  1. Agent 游标路径只传了 pageSize / fetchSize,没有传 maxRowsagent_execute_query_page_params 因此仍使用固定的 MAX_ROWS = 10000,所以 JDBC、Presto 等 Agent/External 驱动即使设置 200000,结果集仍会在 10000 行处被关闭。请让该路径的 maxRows 至少与请求的 pageSize 一致,并补一个超过 10000 行的 Agent 分页回归测试。

  2. normalizeResultPageSize 现在接受任意有限正数,但 Agent 协议最终通过 Gson getAsInt() 读取。超过 Java int 范围的值会静默溢出,例如 2147483648 会变成负数,随后被当成一行分页处理。请保留一个与前后端类型和内存模型兼容的明确上限,并在前端归一化及后端边界同时校验。

@t8y2 t8y2 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes: the two correctness issues from the previous review remain unchanged after the latest merge from main.

  1. The Agent cursor request sends pageSize and fetchSize but not maxRows. agent_execute_query_page_params therefore falls back to the fixed 10,000-row safety limit, so JDBC/Presto/other Agent drivers still close the result session at 10,000 rows even when the configured page size is larger.

  2. Removing every upper bound allows values above Java's signed int range to reach the Agent protocol. Gson getAsInt() silently wraps 2147483648 to -2147483648; the resulting negative page size is then normalized into incorrect pagination behavior.

Please pass a compatible maxRows for Agent paging, keep an explicit cross-runtime upper bound, validate it at both boundaries, and add regression tests above 10,000 rows and around the Java int limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 查询数据行限制问题

2 participants