Skip to content

fix(MetaFieldRanker): apply missing_meta when no document has the field - #12963

Merged
anakin87 merged 2 commits into
deepset-ai:mainfrom
sclfcz:fix/missing-meta-drop-when-no-field
Sep 28, 2026
Merged

anakin87 merged 2 commits into
deepset-ai:mainfrom
sclfcz:fix/missing-meta-drop-when-no-field

Conversation

@sclfcz

@sclfcz sclfcz commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Fixes #12952

What

MetaFieldRanker.run() returns the input documents unchanged when no document has the ranking field:

if len(docs_with_meta_field) == 0:
    logger.warning(...)
    return {"documents": deduplicated_documents[:top_k]}

That branch bypasses the configured missing_meta policy, so missing_meta="drop" stops filtering exactly in the case where everything should be dropped:

docs = [Document(id="a", content="a"), Document(id="b", content="b")]
MetaFieldRanker(meta_field="rating", missing_meta="drop").run(docs)["documents"]
# before: [a, b]   (expected [])
# after:  []

It is also inconsistent with the mixed case: with one rated document, the unrated ones are dropped as configured.

Change

Apply the configured policy in that branch too — drop returns an empty list, top/bottom keep returning the documents as before — and keep the existing warning, reworded to say that the policy is applied rather than that ranking is skipped. top_k is untouched for the top/bottom paths.

Testing

test/components/rankers/test_metafield.py::test_missing_meta_drop_when_no_document_has_the_field covers all three policies for the all-missing case; the mixed case is already covered by the existing tests, which are unchanged.

  • with the change: 1 passed (-k missing_meta_drop_when_no_document)
  • removing the drop branch: the same test fails
  • test/components/rankers/test_metafield.py → 47 passed with the change (full file run in CI)

AI assistance

Written with an AI coding assistant (Claude-based agent): it located the branch, ran the before/after reproduction above and wrote the test. I reviewed the diff, the reproduction output and the reasoning, and I take responsibility for the change.

When every document lacks the ranking field, run() returned the input
documents unchanged, bypassing the configured missing_meta policy: a batch with
one rated document dropped the unrated ones, while an entirely unrated batch
passed through intact even with missing_meta="drop".

Apply the policy in that branch as well (drop -> empty list, top/bottom keep
the documents, as before) and keep the warning that explains the situation.
@sclfcz
sclfcz requested a review from a team as a code owner September 25, 2026 15:40
@sclfcz
sclfcz requested review from anakin87 and removed request for a team September 25, 2026 15:40
@vercel

vercel Bot commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

@sclfcz is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Sep 25, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a minor adjustment.
Thank you

@anakin87
anakin87 enabled auto-merge (squash) September 28, 2026 13:00
@github-actions github-actions Bot added the type:documentation Improvements on the docs label Sep 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/rankers
  meta_field.py
Project Total  

This report was generated by python-coverage-comment-action

@anakin87
anakin87 merged commit c2e809e into deepset-ai:main Sep 28, 2026
22 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MetaFieldRanker ignores missing_meta="drop" when every document lacks the field

3 participants