Data Models

Data Models

🌐 English · Deutsch

Source registry (providers.py)

Every streaming source is registered as a Provider dataclass: URL patterns (series/season/episode) plus the corresponding model classes.

Source	series_cls	season_cls	episode_cls
AniWorld	AniworldSeries	AniworldSeason	AniworldEpisode
SerienStream (serienstream.to)	SerienstreamSeries	SerienstreamSeason	SerienstreamEpisode
FilmPalast	—	—	FilmPalastEpisode (movies have no season structure)
MegaKino (series)	MegakinoSeries	MegakinoSeason	MegakinoEpisode (one post = one season; synthetic …?episode=N URLs)
MegaKino (movie)	—	—	MegakinoMovie
hanime (18+)	HanimeSeries	HanimeSeason	HanimeEpisode (franchise = series; synthetic …?ep=N URLs; browser-resolved AES-128 HLS)

resolve_provider(url) normalises the URL (e.g. /serie/stream/<slug> → /serie/<slug>, trailing slash) and returns the matching provider — or raises ValueError("Unsupported URL").

from mediaforge.providers import resolve_provider

prov = resolve_provider("https://aniworld.to/anime/stream/one-piece")
series = prov.series_cls(url="https://aniworld.to/anime/stream/one-piece")

Series

Lazy-loading properties (HTML is fetched on first access and cached):

title, title_cleaned, description, genres, release_year, poster_url, directors, actors, producer, country, age_rating, rating, imdb (IMDB ID), mal_id (AniWorld only), has_movies, seasons, season_count.

Note on poster URLs: some serienstream.to series use a hash ID instead of the series slug in the poster path — the extraction is generalised accordingly.

Season

AniworldSeason(url, series=None) — properties include season_number, episodes, are_movies (AniWorld "movies" season, skipped e.g. by AutoSync).

Episode

Core class for downloads:

ep = prov.episode_cls(
    url="https://aniworld.to/anime/stream/x/staffel-1/episode-1",
    selected_language="German Dub",
    selected_provider="VOE",
    selected_path="/path/optional",   # optional
)
ep.download(cancel_event=event)        # blocking; aborts when the event is set

• provider_data (property): mapping (Audio, Subtitles) → {host → embed URL} — the basis for language checks (e.g. in AutoSync) and host selection.
• download(): invoke extractor → direct link → yt-dlp/FFmpeg → target file according to MEDIAFORGE_NAMING_TEMPLATE; sets language tags (deu/eng/jpn) on audio/subtitle tracks. The finished file path is available in _episode_path afterwards.
• Progress is published globally via models/common/common.get_ffmpeg_progress() (phase download = yt-dlp, ffmpeg = muxing) — the queue UI and /api/v1/status read from it.

Language system (config.py)

Site language keys are mapped to semantic enums:

Key	Label (aniworld)	Audio	Subtitles
1	German Dub	GERMAN	—
2	English Sub	JAPANESE	ENGLISH
3	German Sub	JAPANESE	GERMAN
4	English Dub	ENGLISH	—

serienstream.to uses the same keys with its own labels (1 = German Dub, 2 = English Dub, 3 = English Dub with German Sub). Helper maps: LANG_KEY_MAP, LANG_LABELS, LANG_CODE_MAP (ISO 639-2 for FFmpeg tags) and their inverses.

URL patterns

Defined as compiled regexes in config.py (MEDIAFORGE_SERIES_PATTERN, …_SEASON_…, …_EPISODE_…, analogous SERIENSTREAM_*, plus FILMPALAST_EPISODE_PATTERN in providers.py). AniWorld episode URLs also cover movies (/filme/film-N).

Audio / Subtitles enums

The Audio / Subtitles enums this page relies on are not one shared pair. models/s_to/episode.py defines its own (Audio.GERMAN / ENGLISH / JAPANESE, Subtitles.NONE / GERMAN), intentionally separate from the ones in config.py that AniWorld uses — s.to offers an "English audio + German subtitles" combination AniWorld's LANG_KEY_MAP does not model the same way.

Because the two enum classes are distinct types, models/common/common.py's download() tells the two apart via hasattr(self, "_normalize_language"), not isinstance.

episode_number vs file_episode_number

A real contract, and one that bites: episode_number is what the page shows and what filters use; file_episode_number is what gets matched against files on disk.

It exists because of AniWorld's absolute episode numbering (option aniworld_absolute_episodes): an episode titled … [Episode 062] stays in the season AniWorld lists it in but is written as S02E062 instead of S02E002.

• episode_number — the season-relative number from the URL. Page display, filters, everything user-facing.
• absolute_episode_number — parsed from the [Episode NNN] title marker, or None. AniWorld-only.
• file_episode_number — the number folder path and file name are built from. Equals episode_number unless absolute numbering is on and this episode actually carries a marker. Movies (/filme) are never renumbered.
• file_number_candidates — every (season, episode) pair a file for this episode may carry, current scheme first. Used for "do I already have this?" so flipping the setting does not make an existing library look empty and re-download the whole show under a second scheme.

All three are defined in models/aniworld_to/episode.py. No other provider has them — s.to, MegaKino, FilmPalast and hanime episodes only have episode_number, so every call site must use getattr(ep, "file_episode_number", ep.episode_number) and getattr(ep, "file_number_candidates", ((season, ep.episode_number),)). Real consumers today: the path/file-name building in models/aniworld_to/episode.py, web/autosync_worker.py's skip check, and the OpenSubtitles metadata builder in models/common/common.py.

models/common/

Site-agnostic pieces every episode model shares. models/common/__init__.py re-exports ProviderData, check_downloaded, clean_title, and the shared download() / watch() / syncplay() episode actions from common.py.

ProviderData

The class behind the provider_data mapping documented above. It wraps dict[(Audio, Subtitles)][provider_name] -> url and behaves like a dictionary: get((audio, subs)) returns {} for an unknown key, __getitem__ raises. __str__ renders a readable "German audio + English subtitles / - VOE -> …" block, which is what the CLI prints.

dupecheck.py — duplicate handling

Two opt-in checks that run when the episode is already on disk. Both default to off.

• dl_quality_upgrade — enumerate the source's formats (yt-dlp, metadata only, nothing downloaded) and compare against the file on disk. Deliberately through yt-dlp and not ffprobe: the resolved stream URL is usually an HLS master playlist, and ffprobe would report the variant that playlist defaults to while the download itself takes bestvideo+bestaudio/best. Only a strictly higher video height (HEIGHT_TOLERANCE 32 px) or a meaningfully higher bitrate at the same height (BITRATE_MARGIN 0.20) counts as better; anything unknown counts as not better. is_better_quality(), explain().
• dl_audio_track_merge — the episode exists in another language and the current job is a different one. With language separation on (or {language} in the naming template) that file lives under a different folder and name, so the mux path never saw it and wrote a near-duplicate. find_existing_variant() closes the gap by resolving the same episode across sibling language folders, matching on an S01E01 / 1x01 token rather than on the full name.

The subtitle chain

_gather_subtitles() in models/common/common.py runs three stages, cheapest first, so the three download branches (full / audio-only / video-only) cannot drift apart:

1. yt-dlp sidecars (subtitles.py) — every subtitle rendition the stream itself offers, written next to the video and muxed into the .mkv as tagged soft-sub tracks. Setting dl_subtitles, default on; capped at MAX_SUBTITLE_TRACKS (12). writeautomaticsub stays off on purpose — ASR captions are not the source's subtitles.
2. The hoster's player config (fetch_hoster_subtitles()) — out-of-band tracks an extractor reports that the stream does not carry.
3. OpenSubtitles.com (opensubtitles.py) — external, off by default (opensubtitles_enabled plus credentials), and only for languages still missing after 1 and 2. moviehash first (size + first/last 64 KiB — identifies the exact release, so the timing fits), title + season_number/episode_number second. The daily quota is cached per process, and the /download response's URL is checked against _ALLOWED_DOWNLOAD_HOSTS over https only, so a spoofed response cannot turn into an SSRF.

Nothing in this chain may raise: a missing subtitle is a cosmetic loss, the video is the deliverable. Language codes are normalised to ISO 639-2/B (normalize_lang(), so a ger track and a deu track are not two languages).

Stage 3 is extensible. subtitle_sources.py (core, no mediaforge.web import) offers register_subtitle_source(item_id, source_id, label, fetch) / unregister_subtitle_source(item_id); fetch(video_path, have_langs, meta) -> [Path, …] must never re-fetch a language already in have_langs and returns sidecars named <video stem>.<lang>.<ext>. opensubtitles is a reserved source_id. Registrations are keyed by item_id, so web/thirdparties/registry.py's unregister_module() removes them when a module is disabled.

models/direct_link/ — the Direct Link path

A direct link has no series/season/provider/dub-sub structure to reconcile: the user pastes a raw stream URL (typically an .m3u8 master playlist, but anything yt-dlp supports works), picks one of the probed quality variants and gives the job a filename. So this path bypasses resolve_provider() entirely — web/queue_worker.py branches on item.get("provider") == "Direct" and constructs the class directly.

episode.py — DirectLinkEpisode

DirectLinkEpisode(url, title, selected_path=None, format_id=None, source_provider=None). It exists purely to satisfy the small interface the queue worker's dispatch loop expects from an "episode" (.download(cancel_event=…), ._episode_path, title_cleaned), so the retry/watchdog/history machinery is reused unchanged. Output is always <folder>/<title_cleaned>.mkv; format_id defaults to bestvideo+bestaudio/best.

download() always re-runs the full discovery rather than reusing what was resolved at probe time — many embed hosts hand out short-lived, signed CDN URLs that expire while the job sits in the queue. The first attempt may use the 10-minute resolve cache; a retry forces a fresh resolution.

resolver.py

The "which hosts can we resolve ourselves" layer. The host list is derived at runtime, not hard-coded: every name in models/megakino_to/scraper.py's _HOSTER_DOMAINS is kept if and only if extractors/provider/ has a real (non-stub) extractor for it — _extractor_implemented() probes with an empty URL and drops anything raising NotImplementedError. fast_providers() returns them in the user's configured provider order (Settings → Provider order).

FAST_PROVIDERS is an instance of _FastProviderSet, a live view over fast_providers() rather than the hard-coded set it used to be — existing name in FAST_PROVIDERS checks keep working while picking up newly enabled providers without a restart.

resolve_stream_for_provider(name, url, timeout=12) calls that host's extractor and returns (stream_url, headers), headers seeded from PROVIDER_HEADERS_D plus DIRECT_LINK_USER_AGENT (a desktop Chrome UA; some CDNs reject yt-dlp's default outright).

probe.py

discover_and_resolve(url, timeout=12, use_cache=True) is the whole pipeline, used identically by the probe step and by the download-time re-resolution. In order, retrying across all candidates at each step (a link that merely looks like a supported host is often a dead mirror):

1. url itself, if it is already a known embed-host link (detect_fast_provider()).
2. find_candidate_urls_in_page() — a static HTML scan for embedded hoster links (iframe/href, plus URLs inside inline <script> JSON after un-escaping \/), sorted by the user's provider order. It cannot find a link a site only reveals via a click-driven AJAX call.
3. browser_sniff.sniff_media_url() — last resort.

It always returns (provider_name_or_None, stream_url, headers); on total failure (None, url, {"User-Agent": …}), so the caller can still hand the URL to yt-dlp's generic extraction.

SSRF gate. Direct Link fetches, yt-dlp-probes and even headless-renders whatever URL a logged-in user pastes, so assert_safe_url() refuses non-http(s) schemes and private/loopback/metadata addresses, raising UnsafeUrlError (a ValueError subclass). It reuses web/stream_proxy.py's is_safe_url() — the same check the HLS proxy applies — imported lazily to keep the model layer free of an import-time dependency on the web package. The gate is applied to the pasted URL, to every redirect hop (redirects are followed manually, up to MAX_PAGE_REDIRECTS, so each hop is checked before it is fetched), to every in-page candidate, and to the stream URL the host finally hands back.

probe_direct_link_formats(url) is the read-only "what's in here" step behind the format-picker modal: it returns {"title", "provider", "formats"}, where formats[0] is always the safe bestvideo+bestaudio/best default and the rest are one entry per resolution (highest bitrate wins), with +bestaudio appended for video-only HLS renditions.

browser_sniff.py

Site-agnostic fallback: render the page in a headless browser (patchright; absent → returns None), watch its network traffic and return the first .m3u8/.mp4 request that is not obvious ad noise. It nudges playback with a couple of centre clicks, since many players only issue the real media request after a click (autoplay blocked, or a "click to play" overlay). Returned headers carry a Referer of the original page. Only tried last — launching a browser costs seconds.

Books / eBooks (web/books/)

Books get an indexing pass of their own and are stored under a separate books key in the library cache, alongside titles. That split is the point: nothing in the video pipeline — scanning, stats, calendar, auto-sync, upscaling — can be reached by a book entry. Because set_library_cache() replaces the whole row, a video-only partial update has to carry books / books_version over explicitly, or the shelf silently empties until the next full scan.

The video scanner in routes/library.py is positional (a directory under the library root is a title; a file counts only if its name carries an SxxExx marker). Neither assumption survives a Calibre layout, where the top-level folder is the author and the book sits one level down in Title (id)/. So the book scan is extension-driven instead: every file under the base, at any depth, whose extension is a book format is a candidate. Where it sits only informs the metadata; it never decides whether it counts.

• scanner.py — scan_books(base). Walks the tree, builds candidates, merges them, and emits the book entries the API returns. It never opens an EPUB/MOBI/PDF; it reads the filesystem plus metadata.opf and Calibre's metadata.db where present. Entries carry key, title, sort_title, authors, series/series_index, language, isbn, published, publisher, rating, description, tags, cover_path, formats, total_size, added_at, media_kind: "book". The scan result is versioned via BOOKS_FORMAT_VERSION.
• identity.py — pure string logic (no filesystem, no DB) deciding which files are the same book. This is the part that has to be right: too eager and two books collapse into one card, too timid and the same novel appears five times — which is not hypothetical, because Calibre stores one record per format, so EPUB + MOBI + AZW3 + PDF is four folders with four metadata.opf files, plus the loose copy in the library root. normalize(), clean_title(), split_filename() (stem → title/author), split_series(), group_key(title, author) and merge_groups(candidates).
• opf.py — parse_opf() for Calibre's metadata.opf sidecar. Best metadata available without opening the book, and it carries the untruncated title that the filename does not. Size-capped at 512 KB against XML bombs.
• calibre_db.py — load_catalogue() / lookup() against metadata.db, opened strictly read-only (file:…?mode=ro&immutable=1, so no -wal/-shm sidecars and no lock Calibre would trip over). Every failure is soft. Authoritative for series/tags/rating/description, but never overrides a title or author already resolved from the file itself.
• covers.py — where the cover picture hides per format: the EPUB package document's cover (EPUB 3 properties="cover-image" and EPUB 2 <meta name="cover">), MOBI/AZW3/AZW via the converted EPUB, a sidecar cover.jpg, and nothing for PDF. Caching and downscaling live in web/covercache.py, shared with the comic shelf.
• convert.py — no browser renders Mobipocket, so the reader asks for an EPUB and the server makes one via the mobi package (GPL-3.0). Cached by (path, mtime, size) under the config directory; nothing is ever written next to the user's book, which is what makes a read-only library mount work.

How a book identity differs from a video title. A video title is identified positionally (folder under the root) and keyed by season/episode numbers. A book is identified by normalize(title)|normalize(author) — path-independent by design, so the same novel loose in the root and inside Author/Title (1234)/ merges into one entry. The formats become a formats list on that one entry (each with path, size, readable, drm) instead of separate items, and author-less entries get their own namespace so they can be attached later rather than force-merged with a same-titled book by a different author.

Extending the registry from a module

Third-party modules can add their own content source at runtime via register_provider(item_id, provider) / unregister_provider(item_id) in providers.py — the same Provider dataclass documented at the top of this page. Registrations are keyed by the module's item_id so they are cleaned up on uninstall.

Details, the full registry list and the rules a module has to follow are in the Module API — not repeated here.