AsymVerify-ACL

Public release for AsymVerify at SemEval-2026 Task 6: Asymmetric Confidence-Gated Verification for Political Evasion Detection.

AsymVerify is a confidence-gated LLM verification system for CLARITY / SemEval-2026 Task 6. It classifies political question-answer pairs as Clear Reply, Ambivalent, or Clear Non-Reply, then selectively applies targeted verification to low-confidence boundary cases.

Result: Team AsymVerify scored 0.85 Macro F1 on the official CLARITY Subtask 1 evaluation split and placed 2nd out of 41 teams.

• Paper: paper/main.pdf
• Task site: https://konstantinosftw.github.io/CLARITY-SemEval-2026/
• Organization: https://kaons.com/

Repository Contents

Path	Description
asymverify.py	OpenRouter runner for the paper-facing Pass 1/2/3 routing.
paper/	LaTeX source, bibliography, ACL style files, figure asset, and compiled PDF.
scripts/camera_ready_analysis.py	API-free regeneration of camera-ready table fragments.
outputs/camera_ready/	Public aggregate CSV artifacts and generated LaTeX tables.
examples/demo.csv	Tiny format-only input file for smoke tests.

The repository is organized around the released system, paper source, and camera-ready analysis artifacts.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Create a local environment file if desired:

cp .env.example .env

Do not commit .env or API keys. The runner reads OPENROUTER_API_KEY from the environment.

Run AsymVerify

export OPENROUTER_API_KEY=your_key_here
python asymverify.py \
  --input examples/demo.csv \
  --output-dir outputs/demo \
  --model z-ai/glm-4.7 \
  --passes 1,2,3 \
  --threshold 0.95

Input CSVs should include a question column (question or interview_question) and an answer column (answer or interview_answer). Add --label-col clarity_label to compute accuracy and Macro F1 on a labeled file.

By default, results.json stores compact pass summaries. Use --include-traces only when you intentionally want to keep prompts and raw model responses for local debugging.

Reproduce Paper Tables

The release includes only aggregate artifacts needed to reproduce the camera-ready table fragments:

python scripts/camera_ready_analysis.py

This command reads outputs/camera_ready/*.csv, rewrites the corresponding LaTeX fragments, and does not call any model API.

Build the Paper

cd paper
latexmk -pdf -interaction=nonstopmode main.tex

The official CLARITY evaluation labels for D_eval are not distributed in this repository. Development analyses are reproduced from the public aggregate artifacts included under outputs/camera_ready/.

Citation

If you use AsymVerify, cite the system paper and the CLARITY/QEvasion task papers:

@inproceedings{kawada2026asymverify,
  title = {AsymVerify at SemEval-2026 Task 6: Asymmetric Confidence-Gated Verification for Political Evasion Detection},
  author = {Kawada, Sebastien},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026)},
  year = {2026},
  url = {https://github.com/kaons-research/AsymVerify-ACL}
}

@inproceedings{thomas-etal-2024-never,
  title = {``{I} Never Said That'': A dataset, taxonomy and baselines on response clarity classification},
  author = {Thomas, Konstantinos and Filandrianos, Giorgos and Lymperaiou, Maria and Zerva, Chrysoula and Stamou, Giorgos},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024},
  pages = {5204--5233},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.300},
  url = {https://aclanthology.org/2024.findings-emnlp.300/}
}

@misc{thomas2026semeval2026task6clarity,
  title = {SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions},
  author = {Konstantinos Thomas and Giorgos Filandrianos and Maria Lymperaiou and Chrysoula Zerva and Giorgos Stamou},
  year = {2026},
  eprint = {2603.14027},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  url = {https://arxiv.org/abs/2603.14027}
}

License

Code is released under the MIT License. The paper text and figures should be cited as scholarly work.