safety: guard against fake releases; fix config/indexer blind spots - #5
Merged
Merged
Conversation
Fake releases (the S29E01 incident) - search + hunter drop releases whose name is an executable (BLOCK_EXECUTABLE_RELEASES); hidden counts surface in the UI - live guard (GUARD_INTERVAL_SECONDS, default 60): media torrents whose file list is executables with no video are paused, flagged, recorded, notified; wants re-queued; never deleted; resumed torrents left alone - sorter backstop: such releases are quarantined with executables renamed *.faucet-blocked; new exit code 5, handled by hook and sweep - AIR_DELAY_DAYS (default 1): new episodes are not hunted until the day after they air; settable in Settings -> Behavior Config / indexer blind spots - Settings -> Connections gains a write-only Jackett API key field - scheduler and stalls read the live config (F15): keys saved in Settings reach the hunter without a restart - hunt short-circuits with one warning when the indexer isn't configured; search failures summarized; add failures logged and recorded as grab_failed (F33) - torznab <error> documents (bad API key) raise instead of reading as "no results" - /api/search returns 503 with an explanation when unconfigured; the UI shows server error detail instead of a bare status code - dashboard indexer pill reflects the indexer (it mirrored the client); dashboard and admin /health list configuration warnings Tests - tests/test_fake_releases.py: 41 regressions - tests/test_wants_upgrades.py: use monkeypatch (direct assignment leaked a fake search into later test files)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened
On Sept 16, two things broke, and each hid the other.
1. Search and the hunter lost the Jackett key (13:27, the PR #2 deploy).
JACKETT_API_KEYhad only ever lived in the old Portainer stack environment. The trackeddeploy/stack.ymldeliberately carries no secrets, andfaucet.envnever had the key, so the recreated container came up without it.api_searchturns everySearchErrorinto a 502, andjgetthrew away the message.client_ok.index.htmlhard-codedjackett_api_key: null.scheduler.pyandstalls.pybindconfigat import, andconfig.reload()swaps in a new object (F15).2. Bait releases for South Park S29E01, minutes before the 10 p.m. premiere.
South Park S29E01 South America 1080p WEB-DL x265 NTb.exe, a 1 GB Windows executable.South+Park+S29E01+…+NTb, whose name carried no extension until its metadata arrived.REMOVE_ON_COMPLETE..exesat on the NAS share while it downloaded.What changed
Fake-release defenses (
faucet/safety.py, new)search.search()drops results whose title is an executable (.exe,.scr,.msi,.lnk,.js,.ps1,.apk, …;.comis excluded because of tracker watermarks). Every path goes through it: the search page, the hourglass, packs, the hunter and subscriptions.Results.hiddencarries the count, and the UI shows "N unsafe results hidden".BLOCK_EXECUTABLE_RELEASES=0disables it._guard_loopruns everyGUARD_INTERVAL_SECONDS(60), plus once at the start of each tick.transfer_checksand as asuspicioushistory event. It's flagged in Activity → Transfers and on the dashboard, and a notification goes out whenfailedorsuspiciousis inNOTIFY_ON.grabbed, so it's never picked again.EXIT_SUSPICIOUS = 5: in consume mode it's quarantined to_failed/with every executable renamed*.faucet-blocked.quarantine()now returns the destination path. The hook removes the torrent on rc 5 and records and notifies; the sweep counts 5 as swept.The stall handler only touches
downloadingtransfers, so a paused, flagged torrent can't be stall-removed.Air-date delay
AIR_DELAY_DAYS(default 1; 0 to 30; editable in Settings → Behavior, with validation).series.hunt_eligible()gates reconcile, the pack pre-pass (both the wanted list and its aired-episode count), and the per-episode hunt.Config and indexer visibility
/api/settingsexposes onlyJACKETT_API_KEY_SET._cfg(), which readsfaucet.config.configlive.grab_failed(F33).<error>documents (a bad API key comes back as HTTP 200) now raise instead of reading as "no results"./api/searchreturns 503 with an explanation when unconfigured, andjgetshows the server'sdetailtext.indexer.configured, and a warnings banner covers a missing key, a missing or unreachable client, and suspicious downloads. Admin/healthincludes the same warnings; public/healthstays minimal.Docs
AIR_DELAY_DAYS,BLOCK_EXECUTABLE_RELEASES,GUARD_INTERVAL_SECONDS.Tests
tests/test_fake_releases.py, 41 tests:.comwatermark and PS4.pkg.exestill filed; games untouched/health, settings round-trip and validation, the hidden countmain, 18 of them fail.tests/test_wants_upgrades.pynow usesmonkeypatch. Its directsearchmod.search = …assignments leaked a fake search into later files, which only surfaced when test order changed.Deploying
transfer_checkscreates itself.Follow-ups
tests/test_faucet.pystill assignsSCH.searchmod.search/SCH.make_clientdirectly in five places.faucet-deployrefuse to recreate the container whenJACKETT_API_KEYresolves to empty.