Skip to content

Migrate HTML/XML parsing from Jsoup to fleeksoft/ksoup - #110

Open
markst wants to merge 4 commits into
prajwalch:mainfrom
markst:migrate/jsoup-to-ksoup
Open

markst wants to merge 4 commits into
prajwalch:mainfrom
markst:migrate/jsoup-to-ksoup

Conversation

@markst

@markst markst commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

As mentioned in #108 (comment), a few steps are needed before the app can go multiplatform. This PR does the first one: it replaces Jsoup with fleeksoft/ksoup, a Kotlin Multiplatform port of Jsoup.

Summary

  • Replace the JVM-only Jsoup dependency with Ksoup 0.2.6 (latest).
  • Migrate every HTML search provider (38) and the Torznab XML parsers (capabilities, error response, results) from org.jsoup to com.fleeksoft.ksoup. The API is almost identical. The only differences:
    • nodeValue() becomes TextNode.text() (AnimeTosho, BlueRoms, Eztv).
    • The argument order of parse(xml, baseUri, parser) changes to parse(xml, parser) (Torznab results).
  • Ignore nested **/build/ directories so module build outputs aren't shown as untracked.
  • Update the README's HTML-parsing dependency entry.

Rebased onto current main, including the Torznab move to Jsoup XML parsing (100b3f2), which is now migrated to Ksoup as well. No org.jsoup references remain.

Unit tests for the migrated parsers are in a separate stacked PR, #118.

Test plan

  • ./gradlew :app:compileDebugKotlin
  • ./gradlew :app:assembleDebug :app:lintDebug: builds, no lint issues
  • ./gradlew :app:assembleStaging (R8-minified): builds
  • Staging APK on an emulator: searching "ubuntu" returned 966 results from 20 providers, and a FileMood details page parsed correctly. The only failures were network errors (DNS/timeouts for Torrentz, Dmhy, TorrentDownload).
  • Parsing parity: the offline parser test suite in Add offline unit tests for HTML and Torznab parsers #118 (199 tests, one class per HTML provider plus the Torznab parsers) passes unchanged against both main (Jsoup) and this branch (Ksoup). That covers abs:href resolution and the former nodeValue() paths in Eztv, BlueRoms and AnimeTosho.
  • androidApp/build (and other nested build/) stay ignored by git

@markst

markst commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

May be an idea to add some unit tests for each provider?

Cursor added 4 commits October 10, 2026 14:14
Prevents module build outputs (e.g. androidApp/build) from showing up as untracked files.
Ksoup is a KMP-compatible Jsoup port, preparing HTML parsing for a future multiplatform split.
Update imports and parse calls across HTML providers, and replace nodeValue() with TextNode.text() where Ksoup's Node API differs.
@markst
markst force-pushed the migrate/jsoup-to-ksoup branch from 232b0bf to 0761306 Compare October 13, 2026 03:22
@markst markst changed the title Migrate HTML parsing from Jsoup to fleeksoft/ksoup Migrate HTML/XML parsing from Jsoup to fleeksoft/ksoup Oct 13, 2026
@markst
markst marked this pull request as ready for review October 13, 2026 03:23
@markst

markst commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

A note on review scope: the migration itself is small (45 files, +197/−196), but the test commits add 136 files and ~22k lines, mostly HTML fixtures (~1.3 MB). That makes the PR much heavier to review than the change it covers. Happy to go either way:

  1. Split the tests into their own PR. This PR goes back to just the Jsoup → Ksoup migration (first four commits), and the tests (last two commits) become a follow-up PR against main. They don't reference Jsoup or Ksoup, so they pass on either and can land before or after this one.
  2. Cut the tests down to a small set that isn't per provider. Instead of a test class and captured pages for every provider, keep a few tests that cover the parsing behaviour the migration could change: abs:href resolution against the base URI, & decoding in magnet hrefs, ownText()/text()/wholeText()/TextNode.text(), html() serialisation, :containsOwn/:has/nth-child selectors, and the namespaced torznab|attr[...] selectors with Parser.xmlParser(). That's a few hundred lines with small hand-written fixtures, and per-provider coverage can come later if wanted.

Either way, the full suite has already shown that parsing behaves the same: all 199 tests pass unchanged on main (Jsoup) and on this branch (Ksoup). Let me know which you'd prefer and I'll update the PR.

@prajwalch

Copy link
Copy Markdown
Owner

Leave this PR only for Jsoup -> Ksoup migration.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants