Skip to content

Add offline unit tests for HTML and Torznab parsers - #118

Draft
markst wants to merge 6 commits into
prajwalch:mainfrom
markst:test/provider-parser-tests
Draft

markst wants to merge 6 commits into
prajwalch:mainfrom
markst:test/provider-parser-tests

Conversation

@markst

@markst markst commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Stacked on #110. This branch is built on the Jsoup → Ksoup migration, so until #110 merges the diff also shows its four commits. Review only the last two commits (Add JVM unit test infrastructure…, Add offline fixture tests…). I'll rebase onto main once #110 lands.

Split out of #110 as requested, so the migration PR stays focused.

Summary

Offline JVM unit tests for every Ksoup-backed parser: one test class per HTML search provider (38) plus the three Torznab XML parsers. 199 tests in total; run them with ./gradlew :app:testDebugUnitTest.

  • Setup: adds mockk and kotlinx-coroutines-test as test dependencies. testing/ProviderTestSupport.kt provides fixture(path) and fakeNetworkClient(url to body, …), a mockk'd NetworkClient that serves fixture bodies by exact URL and fails on any unexpected request.
  • Approach: each test calls the provider's public API (search, getDetails, getMagnetUri, latest/top where they parse a different page) and checks exact parsed values: names, sizes, seeders/peers, dates, categories, absolute URLs resolved from relative hrefs, decoded & in magnets, info hashes, uploader, description, poster and screenshot URLs. Each parser also gets one negative test (an unexpected page layout returns an empty list or null).
  • Fixtures: trimmed pages captured from each site. Where a site is Cloudflare-protected or NSFW, the fixture is hand-written to mirror the real markup, with neutral placeholder content.
  • Works with either library: the tests never reference Jsoup or Ksoup. The same 199 tests pass unchanged on main (Jsoup) and on the Migrate HTML/XML parsing from Jsoup to fleeksoft/ksoup #110 branch (Ksoup), which shows the migration doesn't change parsing output.
  • Locale: the unit-test JVM locale is pinned to en_US, because FileSizeUtils.formatBytes formats with the default locale (1,40 GB on a German machine).

Existing parser issues found while writing tests (behaviour unchanged)

The tests pin the current output and note these in comments, so fixing them will fail those assertions on purpose:

  • MegaPeer: rows with a comments column shift the fixed td:nth-child selectors, so the size is read from the comment count and seeders/peers end up null.
  • Eztv: seeders are always null, because the count sits inside <font> and ownText() on the cell is empty.
  • OxTorrent: search results get Category.Other, because the site's Logiciels icon class isn't mapped.
  • Torznab: extractAndInferCategory keeps only ids > 100000, so standard categories map to null. The comparison may be inverted.
  • AnimeTosho: on batch pages without a "File size" span, the size keeps a trailing space.

Test plan

  • ./gradlew :app:testDebugUnitTest: 199/199 pass on this branch (Ksoup)
  • The same suite on main (Jsoup): 199/199 pass
  • The same suite with a de_DE default JVM locale: 199/199 pass (the locale pin works)

Cursor and others added 6 commits October 10, 2026 14:14
Prevents module build outputs (e.g. androidApp/build) from showing up as untracked files.
Ksoup is a KMP-compatible Jsoup port, preparing HTML parsing for a future multiplatform split.
Update imports and parse calls across HTML providers, and replace nodeValue() with TextNode.text() where Ksoup's Node API differs.
Each Ksoup-backed search provider and the Torznab XML parsers are exercised
through their public API with captured (or, where blocked/NSFW, structure-
faithful hand-written) fixtures. The same suite passes unchanged against the
pre-migration Jsoup implementation, demonstrating parsing parity.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant