YurTube is a self‑hosted video hosting engine written in Python and built on FastAPI, PostgreSQL, CouchBase, Redis+Celery, Manticore, ffmpeg etc. UX is inspired by Youtube service and ClipBucket engine, with a modular design that runs cleanly on a single host or scales out by moving services to external nodes via configuration.
Highlights
- Accounts and authentication: local sign‑in and SSO (Google, X)
- Two video players: a custom YouTube‑style player and a simple HTML5 player
- Uploading and editing videos with automatic thumbnails, animated previews
- Sprites generation
- Generation and display captions
- Generation and display captions translations
- Watch and embed videos
- Search by two search engines
- Comments with like/dislike and persistent user votes
- Channels and subscriptions, user avatars, responsive UI
Application is WIP now. Available base functional:
- Register and authentication - local and SSO (Gmail, X)
- Two video players - custom (fully inspired by Youtube's one) and simple standard.
- View and embed videos
- Upload and edit videos:
- generation of animated previews
- Sprites generation (by separate microservice)
- Captions generation (by separate microservice)
- Converting to other video/audio formats (by separate microservice)
- Translate captions on much of languages (by separate microservice)
- Two Search engines (Manticore and Postgres FTS)
- Comments by separate microservice
- Notifications system
- Unified extendable storage system (by separate microservice)
- Admin panel for monitor and manage all YurTube app family (by separate microservice).
Design notes
- Modular by default: swap or externalize services through config without code changes
- Scales from a single instance to multi‑server deployments as your load grows
For a minimal version with limited functionality need to install the main app and ytstorage service. Other services are optional. Below is list of all services (they have separate repositories):
- ytcms - service for captions generation
- ytcms - service for video conversions
- yttrans - service for captions translations
- ytcomments - service for comments
- ytsprites - service for WebVTT sprites generation
- ytstorage - service for external storage systems
- ytadmin - admin panel as external service
These could be installed and configured separately. About this see below in the appropriate sections.
Note: This installation is designed for the Fedora distribution, but with appropriate modifications, it can be used on other distributions as well. However, some issues have been noted, such as the fact that FFMpeg is supplied in a stripped-down form in Debian-based distributions and requires a special build. This issue primarily affects external services and can be resolved through dockerization.
cd /var/www
git clone https://github.com/sphynkx/yurtube
cd yurtube
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r install/requirements.txt
deactivate
chmod a+x run.shsudo dnf install -y ffmpegIf not found, enable RPM Fusion (free + nonfree), then install:
sudo dnf install -y https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpmOptionaly - swap ffmpeg-free to full ffmpeg if your system has ffmpeg-free preinstalled
sudo dnf -y swap ffmpeg-free ffmpeg --allowerasingInstall ffmpeg (ffprobe comes with the same package)
sudo dnf install -y ffmpegVerify:
which ffmpeg && ffmpeg -version
which ffprobe && ffprobe -versionFind and configure pg_hba.conf. Add entries for the app database and user (defaults):
# YurTube app
local yt_db yt_user scram-sha-256
host yt_db yt_user 127.0.0.1/32 scram-sha-256
host yt_db yt_user ::1/128 scram-sha-256Reload PostgreSQL config:
sudo -u postgres psql -c "SELECT pg_reload_conf();"Create role and database. Replace SECRET with yt_user's password:
sudo -u postgres psql -v db_pass='SECRET' -f install/prep.sqlSet user password for DB user (replace "SECRET" with actual password):
export PGPASSWORD='SECRET'Apply schema creation and seed data (initial categories/tags):
psql -U yt_user -h 127.0.0.1 -d yt_db -f install/schema.sql
psql -U yt_user -h 127.0.0.1 -d yt_db -f install/seed.sqldnf install redis && systemctl enable --now redis
cp install/yurtube-celery-notifications.service /etc/systemd/system
cp install/yurtube-celery-notifications-beat.service /etc/systemd/system
sudo systemctl daemon-reload
sudo systemctl enable --now yurtube-celery-notifications.service yurtube-celery-notifications-beat.serviceCheck:
redis-cli pingExpect: PONG
Also see config/notifications_cfg.py - it consists default params for localhost. You may redefine them in .env.
Copy sample:
cp install/.env-sample .envand edit params/secrets:
DATABASE_URL=postgresql://yt_user:SECRET@127.0.0.1:5432/yt_db
SECRET_KEY=replace-with-strong-secret
SESSION_COOKIE_SECURE=true
./run.sh bootstrap-adminEnter username, email, password for admin-user.
Go to Google Cloud Console, create new Web-app, config it, get secrets, set them to .env also.
Go to Twitter Dev Portal, create new Web-app, config it, get secrets, set them to .env also.
Application supports two search engines. You may configure both of them and switch via .env parameter. Default is Postgres FTS.
At first:
sudo dnf install hunspell-ru hunspell-en postgresql-contrib postgresql-devel
install/dicts_prep.shApply schemas:
psql -U yt_user -h 127.0.0.1 -d yt_db -f install/postgres/ddl/videos_fts.sql
psql -U yt_user -h 127.0.0.1 -d yt_db -f install/postgres/ddl/fts_config.sql
psql -U yt_user -h 127.0.0.1 -d yt_db -f install/postgres/ddl/videos_norm_fts.sql
psql -U yt_user -h 127.0.0.1 -d yt_db -f install/postgres/ddl/videos_fuzzy_norm.sqlor via single script:
install/postgres/apply_db_schema.shIn the .env set SEARCH_BACKEND to "postgres". Restart app.
For Fedora distribution:
sudo rpm --import https://repo.manticoresearch.com/GPG-KEY-manticore
sudo dnf install https://repo.manticoresearch.com/manticore-repo.noarch.rpmEdit /etc/yum.repos.d/manticore.repo - replace "$releasever" to "9". Install packages:
sudo dnf install manticoreSwitch app to Manticore engine: in the .env set SEARCH_BACKEND to "manticore".
Create tables and check them:
mysql -h 127.0.0.1 -P 9306 -e "SOURCE install/manticore/ddl/videos_rt.sql"
mysql -h 127.0.0.1 -P 9306 -e "SOURCE install/manticore/ddl/subtitles_rt.sql"
mysql -h 127.0.0.1 -P 9306 -e "SHOW TABLES"
curl -s 'http://127.0.0.1:9308/sql?mode=raw&query=SHOW%20TABLES'Next check. Edit some video - do some modification in Meta section (for example Description) and save. Next run
mysql -h 127.0.0.1 -P 9306 -e "SELECT video_id,title,status FROM videos_rt WHERE video_id='XXXXXXXXXXXX'"where "XXXXXXXXXXXX" is ID of edited video. You could get record about modified video.
To force reindex search DB use script install/manticore/reindex_all.py:
source ../../.venv/bin/activate
python3 reindex_all.py
deactivateThis service is required to install. You may install it both same server and separate one. App uses storage abstraction layer and all filesystem operations perform via this service.
The service allows you to set up various storage configurations - a local folder or S3 storage. See details in ytstorage repository.
After install need to configure ytstorage params:
YTSTORAGE_GRPC_ADDRESS- remote host and portYTSTORAGE_GRPC_TLS- set true to use authYTSTORAGE_GRPC_TOKEN- set token same as on service side
Check services/storage/storage_proto/ytstorage.proto. It must be same as one in ytstorage installation. Otherwise you need regenerate proto files.. Just run gen_proto.sh in the same dir.
This is separate service based on gRPC+protobuf and ffmpeg, allow to convert uploading video into different video/audio formats. It installs as separate service on the same or external server. See its repo for details about it's install and configuration.
This is separate service based on gRPC+protobuf and faster-whisper. It installs as separate service on the same or external server. See its repo for details about it's install and configuration.
Default config is 127.0.0.1:9099 but you may reconfigure it to multiserver configuration and redefine other params via .env:
YTCMS_SERVERS=192.168.7.20:9099,127.0.0.1:9099,[::1]:9099
YTCMS_TOKEN=CHANGE_ME
YTCMS_HEALTH_TIMEOUT=0.7
YTCMS_SERVER_TTL=10
YTCMS_AUDIO_PREPROCESS=demux
Also make sure that file services/ytcms/ytcms_proto/captions.proto is identical with one at ytcms service. If not - you have to regenerate stubs:
cd services/ytcms_proto
./gen_proto.shThis is separate service based on gRPC+protobuf and different translation providers. It installs as separate service on the same or external server. See its repo for details about it's install and configuration.
At app config you need to set IP address (YTTRANS_HOST) and port (YTTRANS_PORT) of yttrans service, YTTRANS_TOKEN same as on service side. Also make sure that file services/yttrans/yttrans_proto/yttrans.proto is identical with one at yttrans service. If not - you have to regenerate stubs:
cd services/yttrans_proto
./gen_proto.shThis is separate microservice for work with client side comments service. It communicates with CouchBase DB and exchanges info with app's client side part. To install and configure - see ytcomments repo.
To configure app to work with service set options in the .env:
#YTCOMMENTS_ENABLED=false ## switch to legacy mech
YTCOMMENTS_ENABLED=true
YTCOMMENTS_TRANSPORT=grpc
YTCOMMENTS_ADDR=127.0.0.1:9093
YTCOMMENTS_TRANSPORT=grpc
YTCOMMENTS_TLS_ENABLED=false
YTCOMMENTS_TIMEOUT_MS=3000
YTCOMMENTS_FORCE_SERVICE_READ=1
Make sure that the services/ytcomments/ytcomments.proto is same as one for the ytcomments. Otherwise run gen_proto.sh to regenerate protobuf files.
This is separate service for generation sprites preview, based on gRPC+protobuf. It could be installed locally or on some other server. Download it from this repository, follow instructions for install, configure and run.
Default config is 127.0.0.1:9094 but you may reconfigure it to multiserver configuration and redefine other params via .env:
YTSPRITES_GRPC_ADDR="127.0.0.1:9094"
YTSPRITES_SERVERS=192.168.7.20:9094,127.0.0.1:9094,[::1]:9094
YTSPRITES_HEALTH_TIMEOUT=2.0
YTSPRITES_SERVER_TTL=10
YTSPRITES_TOKEN=CHANGE_ME
Also make sure that file services/ytsprites/ytsprites_proto/ytsprites.proto is identical with one at ytsprites service. If not - you have to regenerate stubs:
cd services/ytsprites/ytsprites_proto
./gen_proto.shOptional service for manage and monitor all YurTube app family. Recommend to install on some separate server and not configure for external access. Details see in ytadmin repo.
Create system user for app, ensure storage directory exists and is writable:
sudo useradd --system --home-dir /var/www/yurtube --shell /usr/sbin/nologin yurtube || true
sudo chown -R yurtube:yurtube /var/www/yurtubeand:
./run.shCopy systemd unit file:
cp install/yurtube.service /etc/systemd/system/yurtube.serviceMake dir for logs:
mkdir -p /var/log/uvicornReload and start:
sudo systemctl daemon-reload
sudo systemctl enable --now yurtube.service
sudo systemctl status yurtube.service
journalctl -u yurtube.service -fFirewall (if needed):
sudo firewall-cmd --add-port=8077/tcp --permanent
sudo firewall-cmd --reloadThis is case of external hosting server that proxying to another server with app. On external hosting server - create /etc/nginx/conf.d/yurtube.conf:
# upstream to VM
upstream vm_app {
server 192.168.7.3:8077;
keepalive 32;
}
# May be duplicated with nginx.conf. First check `nginx -t`
proxy_buffering on;
proxy_buffers 64 16k;
proxy_busy_buffers_size 256k;
proxy_read_timeout 120s;
#sendfile on;
#tcp_nopush on;
#tcp_nodelay on;
#keepalive_timeout 65s;
open_file_cache max=1000 inactive=60s;
open_file_cache_valid 120s;
open_file_cache_min_uses 2;
# Dont forget create appropriate dirs:
# Previews cache
proxy_cache_path /var/cache/nginx/thumbs levels=1:2 keys_zone=thumbs:50m max_size=1g inactive=30d use_temp_path=off;
# Videos cache (slice, 206)
proxy_cache_path /var/cache/nginx/video levels=1:2 keys_zone=video:200m max_size=20g inactive=7d use_temp_path=off;
# Microchache for dynamicals
proxy_cache_path /var/cache/nginx/micro levels=1:2 keys_zone=micro:20m max_size=1g inactive=30s use_temp_path=off;
map $http_range $nocache_no_range {
default 0;
"" 1;
}
server {
server_name yurtube.sphynkx.org.ua;
access_log /var/log/nginx/yurtube-access.log main;
error_log /var/log/nginx/yurtube-error.log;
# Microcache for init state of player
location /videos/react/state {
# Cache key by `video_id`
set $state_key "";
if ($args ~* "video_id=([^&]+)") {
set $state_key "state:$1";
}
proxy_cache micro;
proxy_cache_key $state_key;
proxy_cache_valid 200 5s;
proxy_cache_lock on;
add_header X-Microcache $upstream_cache_status always;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://vm_app;
}
# Slice-cache for video (Range/206)
# 1. Cache full 200
# 2. Force snt to client 206 from cache
# 3. Cache key w/o $slice_range !! Full 200
location ~* ^/internal/storage/file/.+\.(webm|mp4|mkv|mov)$ {
proxy_cache_key $uri;
proxy_cache video;
proxy_cache_methods GET HEAD;
# Full cache
proxy_cache_valid 200 7d;
# Optional 206 caching for upstream
proxy_cache_valid 206 7d;
# Force send 206 to client from cached 200
proxy_force_ranges on;
proxy_cache_lock on;
proxy_cache_background_update on;
# Allow caching ignoring Set-Cookie
proxy_ignore_headers Set-Cookie Expires;
# Dont send them to client
proxy_hide_header Set-Cookie;
add_header X-Cache $upstream_cache_status always;
add_header X-Video-Location "matched" always;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://vm_app;
}
# Long cache for animated previews and other imgs
location ~* ^/internal/storage/file/.+\.(jpg|jpeg|png|webp|gif)$ {
proxy_cache thumbs;
proxy_cache_valid 200 30d;
proxy_cache_lock on;
proxy_ignore_headers Expires Set-Cookie;
add_header Cache-Control "public, max-age=2592000, immutable" always;
add_header X-Cache $upstream_cache_status always;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://vm_app;
}
location / {
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://vm_app;
}
}
Then:
mkdir -p /var/cache/nginx/{micro,thumbs,video}
service nginx restart
letsencryptChoose subdomain and set option 2.