dffmpeg is a centrally-coordinated FFmpeg worker job manager.
This project is heavily inspired by joshuaboniface/rffmpeg, but re-imagined for distributed environments where clients and workers are decoupled.
dffmpeg introduces a central coordinator to manage job distribution, state tracking, and failure recovery. It features:
- Centralized State & Resilience: Durable job queues and heartbeat monitoring.
- Path Independence: Uses logical path variables instead of absolute paths, allowing flexible mount points.
- Secure & Pluggable: HMAC-signed communication and support for multiple backends (RabbitMQ, MQTT, SQLite, etc.).
For a deep dive into the system design, see Architecture.
The project is a monorepo containing:
dffmpeg-coordinator: The central server (FastAPI).dffmpeg-worker: The worker agent that runs on encoding nodes.dffmpeg-client: The CLI/Library for submitting jobs.dffmpeg-common: Shared libraries and models.
- Coordinator: Functional (API, DB, Scheduling, Janitor, Dashboard, Admin CLI).
- Worker: Functional (Polling, Execution, Mount Monitoring). Note: Capabilities detection is currently a stub.
- Client: Functional (CLI, Library, Proxy). Supports active monitoring and background/detached job submission.
This project is currently in active development.
Configuration is handled via YAML files within each package. Please refer to the specific package documentation for detailed configuration options.
For details on configuring Transports (RabbitMQ, MQTT) and required permissions, see Transport Configuration.
- Getting Started: Step-by-step guide to setting up a cluster.
- Architecture: System design, components, and scenarios (Development vs. Production).
- Security Model: Authentication and key management.
- Transport Configuration: RabbitMQ and MQTT setup details.
Tests are run using pytest. Ensure your python environment is set up.
python3 -m pytestYou can target specific tests:
python3 -m pytest packages/dffmpeg-coordinator/tests/unit/test_janitor.py