bugfix: use max_concurrent_requests for single block and linear state allocation.

[pjgao]

Summary

Cherry-pick two bugfixes from preview/qwen3.5-qwen3.6 branch to main.

Cherry-picked Commits

1. 9b64c30 bugfix: change single block manager blocks to max concurrent requests. (#1413)

   • Original Author: Yingxu Deng dengyingxu1@jd.com
   • Original Commit: 005adbd1
   • File: xllm/core/framework/block/block_manager_pool.cpp
   • Change: SingleBlockManager num_blocks from max_seqs_per_batch + 2 → max_concurrent_requests + 2

2. b2900e7 bugfix: fix high concurrency linear state overflow (#1422)

   • Original Author: Joey Gao 1783198484@qq.com
   • Original Commit: c7ec380f
   • File: xllm/core/distributed_runtime/llm_engine.cpp
   • Change: num_linear_state_blocks from max_seqs_per_batch + 2 → max_concurrent_requests + 2, update error messages

Root Cause

Both bugs use max_seqs_per_batch as the allocation basis for shared single-block and linear-state resources. When max_concurrent_requests > max_seqs_per_batch, reserved block count is insufficient, causing resource exhaustion and overflow errors under high concurrency.

Note

• Original c7ec380f also changed vlm_engine.cpp, but VLMEngine on main has been refactored and no longer contains linear-state reservation logic, so that part was dropped.
• Original authorship preserved via git cherry-pick (Author field unchanged).

[gemini-code-assist]

Code Review

This pull request replaces the use of max_seqs_per_batch from the SchedulerConfig with a global flag FLAGS_max_concurrent_requests for KV cache capacity estimation and block manager allocation. Review feedback highlights that direct access to global flags violates Rule 10 of the repository style guide, which prefers configuration structs or constructor parameters over global state. It is recommended to add max_concurrent_requests to the SchedulerConfig or pass it via the Options struct to maintain encapsulation and follow established architectural patterns.

[gemini-code-assist]

Directly accessing FLAGS_max_concurrent_requests violates the repository style guide (Rule 10), which prefers using configuration structs over global flags. Since max_seqs_per_batch is already managed via SchedulerConfig, max_concurrent_requests should also be added to SchedulerConfig and accessed via ::xllm::SchedulerConfig::get_instance().max_concurrent_requests(). This ensures consistency, centralizes configuration, and allows for proper validation during initialization. Additionally, using std::max(FLAGS_max_concurrent_requests, ::xllm::SchedulerConfig::get_instance().max_seqs_per_batch()) would be more robust to ensure sufficient blocks are reserved even if max_concurrent_requests is misconfigured to a value smaller than the batch size.

References

1. Do not overuse FLAGS_ global variables. Prefer passing configuration through constructor parameters or config structs. Only use global flags for top-level, process-wide settings. ^(link)

[gemini-code-assist]

Directly accessing FLAGS_max_concurrent_requests here violates the style guide (Rule 10) and breaks encapsulation. This value should be passed through the BlockManagerPool::Options struct, following the pattern used for other configuration parameters like num_blocks and block_size. This makes the class easier to test and decouples it from global state.

References

1. Do not overuse FLAGS_ global variables. Prefer passing configuration through constructor parameters or config structs. ^(link)

[yingxudeng]

主分支重构后，这个参数丢失了嘛？

[pjgao]

是的，看着像是#1430 这个PR重构的

[yingxudeng]

vlm_engine.cpp 需要类似修改吗

[yingxudeng]

Ah right, the qwen3.5 VLM code hasn't been merged yet. Never mind — let's revisit this after the VLM PR lands.