diff --git a/.github/ISSUE_TEMPLATE/add-model.yml b/.github/ISSUE_TEMPLATE/add-model.yml new file mode 100644 index 0000000..1aa38cb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/add-model.yml @@ -0,0 +1,32 @@ +name: Add a model +description: A model this registry does not cover yet +labels: [data] +body: + - type: markdown + attributes: + value: | + Coverage is curated rather than exhaustive, because context windows and + modalities are not published in a scrapeable table by most providers. + Image, video, audio and embedding models are out of scope: this registry + is per-token pricing only. + - type: input + id: model + attributes: + label: Model id + description: The exact string the provider's API accepts + validations: + required: true + - type: dropdown + id: provider + attributes: + label: Provider + options: [openai, anthropic, google, other] + validations: + required: true + - type: input + id: source + attributes: + label: Provider page for this model + description: Ideally the page listing its context window and max output. + validations: + required: true diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000..b58aeb9 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,5 @@ +blank_issues_enabled: true +contact_links: + - name: How the data is produced + url: https://github.com/bytebrujo/modelfax/blob/main/CONTRIBUTING.md + about: Where each field comes from, and which ones the scrapers own. diff --git a/.github/ISSUE_TEMPLATE/data-correction.yml b/.github/ISSUE_TEMPLATE/data-correction.yml new file mode 100644 index 0000000..69c5435 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/data-correction.yml @@ -0,0 +1,31 @@ +name: Data correction +description: A price, context window, or lifecycle date is wrong +labels: [data] +body: + - type: input + id: model + attributes: + label: Model id + description: The `model_id` as it appears in the data file, e.g. claude-opus-5 + validations: + required: true + - type: input + id: field + attributes: + label: Which field is wrong + description: e.g. pricing.input_per_mtok, context_window, dates.retired + validations: + required: true + - type: input + id: source + attributes: + label: Provider URL showing the correct value + description: Must be on the provider's own domain. This is the whole argument. + validations: + required: true + - type: textarea + id: detail + attributes: + label: What it says now, and what it should say + validations: + required: true diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..77d9823 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,58 @@ +# Contributing + +Corrections are the most useful thing you can send. Prices and retirement dates +change without warning, and a registry that is wrong is worse than no registry. + +## Reporting wrong data + +Open an issue with the provider's own URL showing the correct value. That URL is +the whole argument; everything here is transcribed from provider documentation +and every record carries the `sources` it was read from. + +## Fixing data yourself + +Edit `data/.json` and open a pull request. Two rules: + +- Cite the provider page in `sources`. It must be https and on the provider's + own domain, which a test enforces. +- Run `make check` first. It validates every record against + `schema/model.schema.json` and checks cross-file invariants. + +Note that the scrapers own the fields they can read. If you correct a price by +hand and the provider's page still says something else, the next scrape will +change it back, which is the intended behaviour. Fix the parser instead. + +## Adding a model + +The registry covers a curated set rather than everything a provider lists, +because context windows and modalities are not published in a scrapeable table +by most providers. Adding a model is two steps in one pull request: + +1. Add its id to `tracked` in `scrapers/providers/.js`. +2. Add a complete record to `data/.json`. + +A test enforces that those two stay in sync, so you cannot do one without the +other. + +## Fixing a broken scraper + +When a provider restructures a page, the scraper fails loudly with a +`ParseError` naming the CSS selector and the expectation that broke. The fix is +one pull request containing all three of: + +1. A refreshed fixture in `fixtures//.html`. +2. The parser change. +3. The updated test expectations. + +Then run `node scrapers/run.js --offline` to re-derive `data/` from the new +fixture, because `make check` asserts the two agree. + +## Before you open a pull request + +```sh +npm ci +make check +``` + +CI runs exactly that and is a required check. `AGENTS.md` has the operational +rules, including the ones learned the hard way. diff --git a/README.md b/README.md index 2711ab8..8153596 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,20 @@ published as a static site with a free JSON API. Upstream provider pages are re-read daily; every change lands as a reviewed pull request, so `git log -- data/` is the price history. -- Site: `https://bytebrujo.github.io/modelfax/` (live after Phase 4) +- Site: - Cost calculator: `https://bytebrujo.github.io/modelfax/calculator.html` +## Try it + +```sh +# every Anthropic model, with prices per million tokens +curl -s https://bytebrujo.github.io/modelfax/data/anthropic.json | jq '.models[] | {model_id, input: .pricing.input_per_mtok, output: .pricing.output_per_mtok}' + +# everything retiring, across all three providers +curl -s https://bytebrujo.github.io/modelfax/data/{openai,anthropic,google}.json \ + | jq -s '[.[].models[]] | map(select(.dates.retired)) | sort_by(.dates.retired) | .[] | {id, retired: .dates.retired}' +``` + ## JSON API The API is the data files, served as static files. CORS is open.