feat: add vllm sidecar auto-registration.#69
Conversation
Add a Python sidecar that health-gates vLLM registration, writes InstanceMetaInfo into etcd under a lease, keeps the lease alive, and deregisters on shutdown or repeated health failures. The sidecar uses etcd's v3 HTTP gateway via requests only, avoiding grpcio/protobuf native dependencies.
There was a problem hiding this comment.
Code Review
This pull request introduces a new vllm_sidecar package to automatically register and maintain the liveness of a vLLM instance in the xllm-service cluster using etcd leases. The feedback focuses on enhancing the robustness of the HTTP-based etcd client and the health prober. Key recommendations include handling pre-existing schemes in etcd endpoints, safely parsing JSON responses to prevent crashes from invalid or null fields, and using a requests.Session in the health prober to enable connection reuse and avoid socket exhaustion.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
8ed2f7f to
93fe15e
Compare
|
Thanks for the thorough review! Addressed every comment, and proactively audited the whole PR for the same classes of issues:
|
Description
Adds a standalone vLLM sidecar (
vllm_sidecar/) that auto-registers a vLLMinstance into xllm-service's etcd, replacing the manual
register_vllm.shstepfrom the previous PR. Components:
etcd_registry.py— etcd gateway client (put/lease/keepalive for the instance key)health.py— polls the vLLM/healthendpoint before registeringmeta.py— builds theInstanceMetaInfoJSON (backend_type=vllm, type=DEFAULT)sidecar.py— the runtime entrypoint tying them togetherRelated Issues
Change Type
Pull Request Checklist
<type>: <subject>.main.Reviewer Notes
register_vllm.shregistration with automatic etcd registration.