Skip to content

Bug when computing relavant_docs (qrels) #69

@tonywu71

Description

@tonywu71

Description

At the following line, the relavant_docs dictionary is overridden instead of getting updated. This causes relavant_docs to only contain the last seen query. This is problematic in datasets with duplicate queries (e.g. DocVQA) as the output score is not correctly computed.

relevant_docs[query] = {queries2filename[query]: 1}

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions