Repository navigation
Conversation
…e matrix [p0-live-multipart] [p1-live-soak] [p1-live-size]
…-multipart] [p1-live-soak] [p1-live-size]
…ase and API gates
…ase and API gates
…ase and API gates
…m checksum/Range acceptance
… and budget limits
… verify remote bytes
…d cross-client ACK proofs
refactor(layout): regroup tests and scripts; fix release packaging
…jobs The first real runs of the converted multiworker E2E failed inside the worker phase. Three workers must never hold concurrent MTProto connections to one StringSession, so each job now runs inside an inter-process flock with a fresh short-lived client (connect, claim/process, disconnect), and worker failures print their traceback to stderr for CI diagnosis.
…guard The worker script embedded in the USER multiworker E2E test used 'await' in a class body, which fails at compile time, so every worker died before the start gate. Compute the peer chat id before the class, and add a compile() regression test for the embedded script plus stderr diagnostics on early worker exit.
…egression The embedded worker referenced its argv account via a class body, which cannot see function locals (NameError at runtime). Use a SimpleNamespace runtime instead, and add a subprocess regression test that runs the worker against the real SQL jobs engine with a stubbed telethon package: ready gate, claim, connect, journal, upload, ack and done-state are verified without live Telegram.
The live workers hit a real GetDialogs FloodWaitError with the fail-fast flood threshold; Telethon now auto-sleeps waits up to 60 seconds in the worker and parent clients. The fake-telethon regression subprocess no longer inherits TELEDRIVE_E2E_CHANNEL_NAME/TARGET from the live workflow env, so it always resolves by the numeric target.
The previous patch only changed the parent's client; the worker kept flood_sleep_threshold=0 and died on the real 4s GetDialogs flood-wait. The worker now auto-sleeps waits up to 60s like the parent.
A connected Telethon client is bound to its event loop; the parent used separate asyncio.run calls and failed with 'event loop must not change'. The whole readback/verification/cleanup phase now runs in one async scenario under a single asyncio.run.
…plate
Credentials move from env comma-lists to a tidy config.yaml storage:
section holding the dedicated channel, round_robin/failover strategy and an
apis: list of {api_id, api_hash, session_file, account} sets. The config
section is authoritative; env vars remain a legacy fallback. A
config.example.yaml template lives at the repo root; examples/ is gone and
README/TELETHON_STORAGE/.env.example document the new layout.
feat(config): tidy storage config with apis list + docs refresh
…rategy; SSL; compose hardening Owner directive: nothing configured via env except TZ and the config path. - teledrive/settings.py central loader; web/app.py + web/s3.py + unified.py read web:/jobs:/state:/storage:/paths: sections only (env vars removed). - TELEDRIVE_STORAGE_TRANSPORT deleted; transport is always Telethon USER. - storage.strategy supports single | round_robin | failover | parallel; parallel uploads chunks concurrently across credential sets (parallel_degree + batched gather in the engine; per-label intents). - config.example.yaml is the tidy template (accounts/phone block dropped: sessions are pre-authorized); .env.example reduced to PUID/PGID/TZ/ports; docker-compose keeps only TELEDRIVE_CONFIG+TZ, adds healthcheck and json-file log rotation; service binds 0.0.0.0 in-container. - state.db_extra_packages installs drivers into /data/site-packages at startup (oracledb example). - scripts/ops/generate_ssl.py (openssl self-signed certs, config snippet, --test HTTPS verification) + web.ssl_certfile/keyfile TLS support. - WebUI Settings page gains a Storage card (channel, strategy, apis list editor) persisting to config.yaml. - Runtime tests now build config.yaml fixtures; env-based settings tests rewritten; full suite green (1083 passed / 61 skipped / 0 failed).
…tions The disposable fault harness now generates config.yaml with the web/storage/ jobs sections (the runner's disposable password and web-a/web-b hosts) and the compose fixture carries only TELEDRIVE_CONFIG. The intentional 0.0.0.0 in-container binds are marked nosec B104 with justification.
TrustedHostMiddleware captured the import-time default; a config-reloaded allowed_hosts change never applied. Replace it with a per-request middleware reading the live ALLOWED_HOSTS. ensure_config accepts an existing mounted config when chmod fails for a foreign uid but the file is already private. The CI smoke runs the container as the host uid and mounts a 0600 config.
load_config created a default legacy account whenever accounts was falsy, including explicit 'accounts: []' configs, which then failed the unified service's storage-path validation. A default account now only appears for a legacy telegram: section without an accounts key; empty accounts and absent telegram sections mean no legacy accounts.
parallel degree is exposed to the engine for concurrent chunk uploads; single strategy pins every send to the first set. Added regression tests for both strategies.
feat(config-only): all settings from config.yaml; parallel strategy; SSL; compose hardening
…ace tests + dry-run CLI)
|
Important Review skippedToo many files! This PR contains 218 files, which is 118 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configuration
⛔ Files ignored due to path filters: 13
📒 Files selected for processing: 218
You can disable this status message by setting the
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Author
|
Closing; work will be kept on the fork (babico/TeleDrive) only. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes the legacy Telethon direct-send scanner/uploader CLI. Unified upload path (REST/S3/local-file → durable job queue → Telethon transport) remains. Legacy auto-bucket scheduler removed. 1085 tests pass.