Skip to content

Test the admin endpoints with credentials - #400

Merged
cmil merged 12 commits into
openapi-linksfrom
admin-tests
Oct 18, 2026
Merged

cmil merged 12 commits into
openapi-linksfrom
admin-tests

Conversation

@cmil

@cmil cmil commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

The admin endpoints were only tested without credentials. Now test/admin (pytest) walks throwaway corpora through their whole lifecycle, and the main Schemathesis run covers the admin operations with credentials. test/schemathesis_hooks.py confines their requests to the throwaway corpus scratch. The schema coverage report now lists only the webhook and the Wikidata author endpoint as untested.

The new tests found several 500s, fixed here. Admin clients may notice these changes:

  • POST /corpora responds with the metadata object instead of an array of the stored path and the metadata.
  • Requests without a body, or with an unsupported Content-Type, get 415 instead of 405.
  • Invalid values (malformed JSON or XML, a non-string field, a load other than true) get 400 instead of 500.

scratch only exists during the main Schemathesis run. The pytest admin tests run after the DTS tests, so neither can affect the test data the other tests rely on.

🤖 Generated with Claude Code

cmil and others added 9 commits October 18, 2026 13:42
POST /corpora failed with 500 for an XML body that isn't well-formed
(eXist-db passes it as a string, which couldn't be cast to
document-node()), for malformed JSON, and for JSON without name or
title, whose error wasn't caught in the JSON handler.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /corpora responded with an array of the path of the stored
corpus.xml and the metadata, while the OpenAPI document describes an
object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PUT .../tei failed with 500 for a body that isn't well-formed XML, and
POST /corpora/{corpusname} for malformed JSON. A `load` value that
isn't a boolean is now rejected as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pattern for the name of a new play didn't match the rule PUT
.../tei enforces. Unanchored, it matched any name containing a letter
or digit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tests walk throwaway corpora through creating, loading and
deleting, and add, update and delete a minimal play, checking that
the changes show up in the public endpoints. They also cover the
error responses and that unauthenticated requests change nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without a body, or with a Content-Type no function consumes, RESTXQ
answered POST /corpora, POST /corpora/{corpusname} and PUT .../tei with
405 Method Not Allowed. Fallback functions without %rest:consumes now
answer them with 415, or with 401 when not authenticated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new corpus needs a name of lower case ASCII letters and a non-empty
title, and loading a corpus only accepts `{"load": true}`. Other values
are rejected with 400.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /corpora with e.g. `"title": {}` failed with 500, as creating the
corpus took the effective boolean value of the map. Found by
Schemathesis.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Instead of only checking that anonymous requests are rejected, the
admin run now sends its requests with credentials. The ignored_auth
check still repeats each of them without credentials.

A hook makes every request that can change data use the throwaway
corpus `scratch`, as Schemathesis doesn't always use the values bound
in schemathesis.toml. The corpus is created with a play before the run
and deleted after it. The run comes after all tests that rely on the
test data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cmil
cmil added this pull request to stack #393 October 18, 2026 12:05
@cmil cmil changed the title admin tests Test the admin endpoints with credentials Oct 18, 2026
@cmil
cmil marked this pull request as ready for review October 18, 2026 12:05
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Merging this PR will leave 2 of 36 operations untested. Please review.

❌ 2 untested
🟡 1 partial
✅ 33 tested

Operations  ███████████████████░    34/36    94%
Parameters  ██████████████████░░    68/75    91%
Keywords    ██████████████████░░   90/100    90%
Responses   ██████████████████░░    74/84    88%
Examples    ██████████████████░░  289/317    91%

❌ Untested (2)

Method Path Parameters Keywords Responses Examples
❌ POST /exist/restxq/v1/webhook/github 0/5 0/7 0/1 0/0
❌ GET /exist/restxq/v1/wikidata/author/{id} 0/1 0/1 0/2 0/0

🟡 Tested (34)

Method Path Parameters Keywords Responses Examples
🟡 POST /exist/restxq/v1/corpora 1/2 8/10 4/5 0/0
🟢 GET /exist/restxq/v1/character/{id} 1/1 2/2 2/3 0/0
🟢 GET /exist/restxq/v1/corpora 1/1 2/2 2/2 0/0
🟢 DELETE /exist/restxq/v1/corpora/{corpusname} 1/1 1/1 2/3 0/7
🟢 GET /exist/restxq/v1/corpora/{corpusname} 1/1 1/1 2/2 7/7
🟢 POST /exist/restxq/v1/corpora/{corpusname} 2/2 6/6 5/6 0/7
🟢 GET /exist/restxq/v1/corpora/{corpusname}/metadata 1/1 1/1 2/2 7/7
🟢 GET /exist/restxq/v1/corpora/{corpusname}/metadata/csv 1/1 1/1 2/2 7/7
🟢 DELETE /exist/restxq/v1/corpora/{corpusname}/plays/{playname} 2/2 2/2 3/3 7/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname} 2/2 2/2 2/2 14/14
24 more
Method Path Parameters Keywords Responses Examples
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/characters 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/characters/csv 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/metrics 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/networkdata/csv 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/networkdata/gexf 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/networkdata/graphml 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/relations/csv 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/relations/gexf 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/relations/graphml 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/spoken-text 7/7 8/8 2/3 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/spoken-text-by-character 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/stage-directions 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/stage-directions-with-speakers 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/tei 2/2 2/2 2/2 14/14
🟢 PUT /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/tei 3/3 4/4 3/5 0/7
🟢 GET /exist/restxq/v1/corpora/{corpusname}/plays/{playname}/txt 2/2 2/2 2/2 14/14
🟢 GET /exist/restxq/v1/dts/collection 3/3 5/5 3/3 14/14
🟢 GET /exist/restxq/v1/dts/document 6/6 8/8 3/3 10/10
🟢 GET /exist/restxq/v1/dts/navigation 7/7 10/10 3/3 13/13
🟢 GET /exist/restxq/v1/id/{id} 1/1 1/1 2/2 0/0
🟢 GET /exist/restxq/v1/characters 0/0 0/0 1/1 0/0
🟢 GET /exist/restxq/v1/dts 0/0 0/0 1/1 0/0
🟢 GET /exist/restxq/v1/info 0/0 0/0 1/1 0/0
🟢 GET /exist/restxq/v1/wikidata/mixnmatch 0/0 0/0 1/1 0/0

Powered by TraceCov v0.25.3

schemathesis 4.31.1 | http://localhost:8081/exist/restxq/v1/openapi.yaml

Download full HTML report

cmil and others added 2 commits October 18, 2026 14:39
/corpora?include=metrics returns `"updated": null` for a corpus without
plays, which the schema didn't allow. Found by Schemathesis after it
had deleted the only play of the throwaway corpus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The schema coverage report of the action only covers the run it was
made for, so it listed the admin operations as untested. The main run
now covers all operations except the webhook, with the throwaway
corpus created before it and removed right after, before the tests
that rely on the test data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`scratch-corpus.sh create` failed with 409 when an earlier run hadn't
deleted the scratch corpus, and schemathesis.sh then went on testing
the old corpus. Now the leftover corpus is deleted first, and
schemathesis.sh stops if the corpus can't be created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cmil
cmil merged commit ba74b4b into main Oct 18, 2026
3 checks passed
@cmil
cmil deleted the admin-tests branch October 18, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant