The Vanity Gateway proxies every route to a single upstream, the NVCF invocation API named by NVCF_API_ENDPOINT. There is no way to point a hostname it serves at the LLM Gateway instead, so operators running both services cannot expose the LLM Gateway OpenAI-compatible endpoints through the gateway.
Proposal: a v2config.llmGateway section mapping hosts to the LLM Gateway, with the endpoint supplied by a new LLM_GATEWAY_ENDPOINT variable. Each configured host would serve the routes the LLM Gateway registers (POST /v1/chat/completions, POST /v1/responses, POST /v1/embeddings) and proxy them unchanged, since the LLM Gateway resolves the target function from the model field the client already sends.
The Vanity Gateway proxies every route to a single upstream, the NVCF invocation API named by
NVCF_API_ENDPOINT. There is no way to point a hostname it serves at the LLM Gateway instead, so operators running both services cannot expose the LLM Gateway OpenAI-compatible endpoints through the gateway.Proposal: a
v2config.llmGatewaysection mapping hosts to the LLM Gateway, with the endpoint supplied by a newLLM_GATEWAY_ENDPOINTvariable. Each configured host would serve the routes the LLM Gateway registers (POST /v1/chat/completions,POST /v1/responses,POST /v1/embeddings) and proxy them unchanged, since the LLM Gateway resolves the target function from the model field the client already sends.