馃殌 The feature, motivation and pitch
Would it be possible to add support for embedding generation and reranking in Aphrodite? Right now, using RAG setups often means juggling both vLLM and Aphrodite, but it鈥檇 be great if Aphrodite could handle the whole process.
Something like vLLM鈥檚 pooling models (https://docs.vllm.ai/en/v0.6.5/models/pooling_models.html) could work鈥攍etting us generate embeddings efficiently and improve retrieval without switching tools. This would help speed things up and simplify deployment.
Curious if this would be doable. Appreciate your thoughts!
Alternatives
No response
Additional context
No response
馃殌 The feature, motivation and pitch
Would it be possible to add support for embedding generation and reranking in Aphrodite? Right now, using RAG setups often means juggling both vLLM and Aphrodite, but it鈥檇 be great if Aphrodite could handle the whole process.
Something like vLLM鈥檚 pooling models (https://docs.vllm.ai/en/v0.6.5/models/pooling_models.html) could work鈥攍etting us generate embeddings efficiently and improve retrieval without switching tools. This would help speed things up and simplify deployment.
Curious if this would be doable. Appreciate your thoughts!
Alternatives
No response
Additional context
No response