AIDA

Tools and utilities for supporting AIDA evaluations.

Tools and utilities for Phase-II evaluation

Name	Description	Location	Latest release branch
AIDA-Evaluation Docker	Tool for applying the entire evaluation pipeline to task1 submissions	tools/aida-evaluation	AIDAED-v2020.0.1

Tools and utilities for Phase-I evaluation conducted at the end of 2019

Name	Description	Location	Latest release branch
SPAQRL Query Application Docker	The tool for applying SPARQL queries to KBs	tools/sparql-query-application	AIDAQA-v2019.2.5
AIDA-ValidateResponses	Tool for validating responses obtained after applying SPARQL queries to KBs	tools/validate-responses	AIDAVR-v2019.0.3
AIDA-ConfidenceAggregation	Tool (and docker) for aggregating confidences	tools/confidence-aggregation	AIDACA-v2019.0.2
AIDA-ScoreResponses	Tool for scoring responses	tools/scorer	AIDASR-v2019.2.1
AIF Generator	Tool for converting annotations into AIF	python/generate_aif.py	AIDAAG-v2019.0.1
AIDA-Evaluation Docker	Tool for applying the entire evaluation pipeline	tools/aida-evaluation	AIDAED-v2019.0.1

Tools and utilities for 2018 (M9) evaluation

Name	Description
utils/translate-annotations/seedling	Utility to translate LDC's seedling annotation to turtle-RDF format
tools/resolve-queries	Tool for applying SPARQL queries to a knowledge base in AIF format in order to produce output in the simplified xml response format
tools/validate-responses	Tool for validating responses for AIDA M9 evaluation

Please refer to tool/utility specific README file(s) for more details.