Skip to content

llms.txt/llms-full.txt still claim the paper freeze is active and audits are paused #437

Description

@yakew7

Where: llms.txt line 9, llms-full.txt line 7 (generated by scripts/build_explainers.py's build_llms_full(), around lines 627-628).

The gap: both files tell any AI crawler/answer engine reading them that the paper freeze is still active and new audits are paused:

Repro:

$ grep -n "peer review\|paused" llms.txt llms-full.txt scripts/build_explainers.py
llms.txt:9:> Status: the benchmark results are frozen for a research paper in peer review (tag `v1.0-paper`). New audits are paused until publication...
llms-full.txt:7:...The benchmark results are frozen for a paper in peer review (tag v1.0-paper); new audits are paused, explainers continue.
scripts/build_explainers.py:628:        "review (tag v1.0-paper); new audits are paused, explainers continue.",

vs. CLAUDE.md's current policy: "Paper freeze - lifted... New audits merge to main like anything else" - and this repo's own history shows real audits have merged since (e.g. Tenant Screening's related work, several post-freeze fixes).

Why it matters: these two files are the public-facing index served to AI crawlers/answer engines at thefaircode.xyz/llms.txt specifically so they get an accurate, current picture of the project - and they're actively telling any LLM reading them the opposite of current, repo-documented policy. Because the stale sentence is hardcoded inside the generator itself (not just the checked-in output), scripts/check_generated_files_current.py's "regenerate fresh, then diff against HEAD" check can never catch this - regenerating just reproduces the same wrong text byte-for-byte.

Suggested fix: update the hardcoded string in build_llms_full() (scripts/build_explainers.py lines ~627-628) and the hand-written llms.txt line 9 to reflect CLAUDE.md's current status, then run make build-explainers to regenerate both files.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions