Skip to content

Add unit tests for TANL post-processing pipeline #2

Description

@shuntaroy

Summary

The post-processing pipeline that parses and validates TANL outputs from the LLM currently lacks test coverage. Unit tests should be added for each stage to ensure correctness and prevent regressions.

Modules to test

tanl_parser.py — Entity parsing

  • parse_tanl_entities(): parsing inline [text | type(abbrev)] format into Entity objects
  • Handling of attribute abbreviations (certainty, state, TIMEX3 type)
  • Whitespace/content mismatch detection
  • Edge cases: nested tags, malformed tags, missing closing brackets

relations.py — Relation parsing

  • parse_relations(): parsing RelationType: e1 -> e2 format
  • Phase-aware validation (time_relation vs medical_relation)
  • Deduplication of duplicate relations
  • TIMEX3 constraint checking for temporal relations
  • Invalid entity ID references

postprocess.py — Validation and correction

  • correct_entity(): normalisation of invalid attribute values (e.g. state → "other")
  • validate_entity(): schema validation (entity types, required/allowed attributes)
  • validate_document() / validate_results(): document-level validation
  • compute_statistics(): entity/relation type distribution counts

tanl_format.py — TANL format generation

  • document_to_tanl(): round-trip conversion (entities → TANL text → entities should be consistent)
  • Nested entity filtering
  • Abbreviated attribute string generation

Tasks

  • Add tests for tanl_parser.py
  • Add tests for relations.py
  • Add tests for postprocess.py
  • Add tests for tanl_format.py
  • Add round-trip tests (parse → format → parse consistency)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions