fix: send queries to E5 embedding models in instruct format - #338
fix: send queries to E5 embedding models in instruct format#338Devansh-awat wants to merge 1 commit into
Conversation
| TCP_CONNECT_TIMEOUT = 2.0 # seconds | ||
| # instruct task for e5 models (https://huggingface.co/intfloat/multilingual-e5-large-instruct), | ||
| # including the bundled default model multilingual-e5-large-instruct | ||
| E5_QUERY_INSTRUCT = 'Given a web search query, retrieve relevant passages that answer the query' |
There was a problem hiding this comment.
Usually we expect queries to be questions I think
The default bundled embedding model is multilingual-e5-large-instruct. Per its model card, E5-instruct queries must be formatted as "Instruct: <task>\nQuery: <query>" or retrieval quality degrades badly, with cross-language search failing almost completely. Documents were already embedded raw (correct), but queries went in raw as well. Wrap only embed_query in the instruct format; embed_documents is left untouched since documents must not get the prefix. This applies to all models because model_name (defaulting to the 'em_model' alias) does not reliably identify e5 models, and for local llama.cpp models it is unused. Fixes nextcloud#331 Assisted-by: Claude:ox-alpha Signed-off-by: Devansh-awat <Devansh-awat@users.noreply.github.com>
383ed25 to
1f839b5
Compare
|
Hello there, We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process. Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6 Thank you for contributing to Nextcloud and we hope to hear from you soon! (If you believe you should not receive this message, you can add yourself to the blocklist.) |
2 participants
Problem
The default bundled embedding model is multilingual-e5-large-instruct. Per its model card, E5-instruct queries must be formatted as:
Otherwise retrieval quality degrades badly — in particular, cross-language search (e.g. a German query against English documents) fails almost completely. The reporter of #331 reproduced this: documents were embedded raw (which is correct for E5), but queries were also sent raw, without the instruct prefix.
Change
Wrap only the query in
embed_querywith the standard E5 instruct task text (Instruct: ...\nQuery: ...).embed_documentsstays untouched since documents must not get the prefix.The wrapping applies unconditionally because the configured
model_namedefaults to theem_modelalias and cannot reliably identify E5 models (and it is unused entirely for the bundled local llama.cpp model).Verification
_get_embeddingand assertsembed_queryreceives the wrapped string whileembed_documents(batched and unbatched) sends texts raw.ruff check context_chat_backend/network_em.py— passespyright context_chat_backend/network_em.py— 0 errors (matches what CI's static-analysis workflow runs)Fixes #331
Assisted-by: Claude Code