Skip to content

Document mechanical preflight while preserving submission cooldowns - #150

Open
dcposch wants to merge 1 commit into
PalomarRegistry:mainfrom
dcposch:codex/infrastructure-retry-docs
Open

dcposch wants to merge 1 commit into
PalomarRegistry:mainfrom
dcposch:codex/infrastructure-retry-docs

Conversation

@dcposch

@dcposch dcposch commented Sep 2, 2026

Copy link
Copy Markdown

Companion to PalomarSubmission #130.

Summary

Agents need instructions for running the reusable mechanical verifier before submitting. Add a concise Mechanical preflight section to llms.txt: use mode: full, pin the workflow and pipeline_commit to the same commit, inspect the report, and address resource failures before retrying. Explain that preflight is advisory and excludes Challenge rendering, editorial review, and registration.

Preserve the existing exponential backoff, including after verification-error, and document cooldowns as inference/CI cost controls. Add a regression test confirming that polling a verification error preserves the complete rate record and does not reset the cooldown.

The resource profile is workflow/report metadata, so no Comparator schema extension is needed. Retain the dependency-audit fixes already in this PR: Vitest 4.1.11, js-yaml 4.3.2, and Miniflare's Sharp override at 0.35.4.

Validation

One commit on current server main (341a610). Local checks pass:

  • ESLint; 370 Node tests; 9 Worker runtime tests.
  • Full dependency audit: zero vulnerabilities.
  • Dashboard backward compatibility.
  • Formalization/browser preflight contract against current PalomarSubmission main (a013555).

The documented reusable workflow is introduced by PalomarSubmission #130; merge that companion before publishing these instructions. The server contract check now passes independently of that merge.

@dcposch

dcposch commented Sep 14, 2026

Copy link
Copy Markdown
Author

cc @kim-em , looks like tests pass except the contract test

that one depends on PalomarRegistry/PalomarSubmission#130 & should pass once that's merged

@kim-em

kim-em commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Sorry, I don't think I want to do the refund. I'm very happy to improve documentation, but the cooldown is not a penalty, it is making sure our inference/CI costs don't explode, and so who is "at fault" doesn't really matter.

Describe the full reusable verifier, pinned pipeline revision, advisory results, and actionable resource failures. Keep the existing exponential backoff for verification errors and cover it with a regression test.

Remove the Comparator profile extension to match PalomarSubmission PalomarRegistry#130. Retain dependency audit fixes for Vitest, js-yaml, and Miniflare Sharp.
@dcposch dcposch changed the title Do not penalize infrastructure verification failures Document mechanical preflight while preserving submission cooldowns Sep 15, 2026
@dcposch
dcposch force-pushed the codex/infrastructure-retry-docs branch from 2ce06eb to cac6842 Compare September 15, 2026 00:19
@dcposch

dcposch commented Sep 15, 2026

Copy link
Copy Markdown
Author

@kim-em makes sense. I've updated it to just the preflight instructions, lmk if that looks right

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants