diff --git a/README.md b/README.md index 6764502..357afc2 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Get the developer experience of Docker without the resource cost. | **Isolation** | Heavy (Container Virtualization) | Light (Venv / Native Process) | | **RAM Usage** | High (Daemon + Container Overhead) | **Near Zero** (Native OS usage only) | | **Disk Space** | Heavy (Duplicated OS layers) | **Tiny** (Shared OS libs) | -| **Management** | `docker compose up` | `systemd-compose . up` | +| **Management** | `docker compose up` | `systemd-compose up` | | **Best For** | Cloud / Complex Deps | Raspberry Pi / Home Lab / Old PCs | --- @@ -21,10 +21,11 @@ Get the developer experience of Docker without the resource cost. ## ✨ Features * **Rootless Architecture:** Runs entirely in userspace using `systemd --user`. -* **Docker-like CLI:** Commands like `up`, `down`, `ps`, and `logs` make it easy to learn. +* **Docker-like CLI:** Commands like `up`, `down`, `ps`, `logs` and `exec` make it easy to learn. * **Isolated Environments:** Automatically creates and manages a Python `venv` for each project. * **Namespaced Services:** Every app runs as `sdc-.service`, so it can never replace one of your other user services. * **Conflict Detection:** Prevents two project folders from claiming the same app name. +* **Resource Limits:** Cap an app's memory and CPU so one runaway app can't take down your Pi. * **Crash Detection:** `up` waits until your app is actually running (and listening on its port) and shows the logs if it isn't. Crash loops end as **CRASHED** instead of restarting forever. * **Global Dashboard:** `ps` shows every app you've started, wherever its project folder lives. * **Flexible:** Runs anything installed in the venv: plain Python scripts, Uvicorn (FastAPI), Gunicorn (Flask/Django), or `python -m http.server` for a static site. @@ -54,7 +55,7 @@ The script asks before changing anything. It: 1. Installs missing packages: `python3`, `python3-venv`, `python3-pip`, `git`, `lsof`, `iproute2`, `dbus-user-session`, `libpam-systemd`. 2. Fixes minimal systems such as DietPi (masked `systemd-logind`, missing `XDG_RUNTIME_DIR`). -3. Offers to add you to the `systemd-journal` group, so `logs` and `status` can show your apps' output on systems that don't keep a per-user journal. This group can read **all** system logs. +3. Makes sure `logs` and `status` can show your apps' output. If the system journal is kept only in memory (common on DietPi and other minimal images), you can't read your own apps' logs. The script offers to store the journal on disk (`/etc/systemd/journald.conf.d/99-systemd-compose.conf`), after which each user can read their own logs. If you decline, it offers the `systemd-journal` group instead, which can read **all** system logs. 4. Enables **linger** for your user, so apps start at boot and keep running after you log out. 5. Links `systemd-compose` into `/usr/local/bin`. @@ -81,10 +82,10 @@ cd ~/my-new-bot nano config.env # set APP_NAME="my-new-bot" # Create the venv, install requirements.txt, install and start the service -systemd-compose . up +systemd-compose up systemd-compose ps # see all apps -systemd-compose . logs # follow this app's output (Ctrl+C to exit) +systemd-compose logs # follow this app's output (Ctrl+C to exit) ``` --- @@ -141,6 +142,8 @@ REQUIRE_PORT="false" | `REQUIRE_PORT` | No | `true` (default) or `false`. See above. | | `PYTHON_BIN` | No | Python used to create the venv. Default `python3`. Delete `venv/` and run `build` after changing it. | | `STARTUP_TIMEOUT` | No | Seconds to wait for the port to start listening. Default `15`. | +| `MEMORY_MAX` | No | Memory limit for the app and all its workers, e.g. `300M`, `1G` or `25%` of RAM. Above it, the app is killed and restarted. Default: no limit. | +| `CPU_QUOTA` | No | CPU limit as a percentage of one core, e.g. `50%`, or `200%` for two cores. Default: no limit. | `config.env` can use `${INSTALL_DIR}` (the project folder), `${APP_DIR}`, and any value from `.env`, such as `$PORT`. @@ -170,20 +173,27 @@ Use systemd's format: ### Project Commands -Run these with the path to a project folder (`.` for the current folder). +Run these inside a project folder, or put the folder first: `systemd-compose ~/my-app up`. | Command | Description | | --- | --- | -| `systemd-compose . up` | **Recommended.** `build` + `run` (like `docker compose up`). | -| `systemd-compose . build` | Creates the `venv` and installs `requirements.txt`. Stops if `pip` fails. | -| `systemd-compose . run` | Writes the service file and (re)starts the app, then waits until it is running. Safe to re-run while the app is running. | -| `systemd-compose . restart` | Restarts the app with the existing service file. Picks up `.env` changes. | -| `systemd-compose . stop` | Stops the app (keeps the venv and logs). It still starts again at boot. | -| `systemd-compose . down` | Stops the app, removes the service, and **deletes the venv**. | -| `systemd-compose . status` | State, PID, memory of all workers, restart count, port and recent log lines. | -| `systemd-compose . logs` | Follows the app's logs. | +| `systemd-compose up` | **Recommended.** `build` + `run` (like `docker compose up`). | +| `systemd-compose build` | Creates the `venv` and installs `requirements.txt`. Stops if `pip` fails. | +| `systemd-compose run` | Writes the service file and (re)starts the app, then waits until it is running. Safe to re-run while the app is running. | +| `systemd-compose restart` | Restarts the app with the existing service file. Picks up `.env` changes. | +| `systemd-compose stop` | Stops the app (keeps the venv and logs). It still starts again at boot. | +| `systemd-compose down` | Stops the app and removes the service. Keeps the venv, so the next `up` is fast. | +| `systemd-compose down --volumes` | Same, and also **deletes the venv**. | +| `systemd-compose status` | State, PID, memory of all workers (and the limit), restart count, port and recent log lines. | +| `systemd-compose logs` | Follows the app's logs. Add journalctl options to change that: `logs -n 200`, `logs --since "1 hour ago"`, `logs --since today -f`. | +| `systemd-compose exec ` | Runs a one-off command like the app runs: venv first on `PATH`, variables from `.env`, inside `APP_DIR`. E.g. `exec python manage.py migrate`, `exec pip list`, `exec python` for a shell. | +| `systemd-compose config` | Prints the service file `run` would write, and whether the installed one is out of date. Changes nothing. | -**`run` or `restart`?** Values in `config.env` (including `$PORT` in `ARGS`) are written into the service file. After changing `config.env` or `PORT`, use `run` or `up`. For other `.env` changes, `restart` is enough. +`systemd-compose --help` lists all commands. + +To manage a project folder whose name matches a command (e.g. a folder called `logs`), write it as a path: `systemd-compose ./logs up`. + +**`run` or `restart`?** Values in `config.env` (including `$PORT` in `ARGS`) are written into the service file. After changing `config.env` or `PORT`, use `run` or `up`. For other `.env` changes, `restart` is enough. Not sure? `systemd-compose config` tells you whether the service file is out of date. ### Global Commands @@ -238,6 +248,17 @@ ARGS="-c ${INSTALL_DIR}/${APP_DIR}/gunicorn.conf.py --bind 0.0.0.0:$PORT" * Wrap an argument containing spaces in quotes, e.g. `ARGS="--name 'My App'"`. * `%` and `$` are passed through literally (systemd-compose escapes them), so a log format like `'%(h)s %(r)s'` is safe. +### Resource Limits + +```bash +MEMORY_MAX="300M" # the app and all its workers together +CPU_QUOTA="50%" # half of one CPU core +``` + +Run `systemd-compose run` to apply. `status` shows memory use next to the limit. An app that goes over `MEMORY_MAX` is killed and restarted, so set it well above normal use. + +Limits need the kernel's memory/CPU controllers to be available to user services (cgroup v2, the default on current Debian, Ubuntu and Raspberry Pi OS). If they aren't, `run` warns that the limit won't be enforced. + ### Startup Check & Crash Handling After starting the app, `run`/`up`/`restart` watch it before reporting success: @@ -282,7 +303,7 @@ systemctl --user daemon-reload | `Name Conflict!` | Another project folder already uses this `APP_NAME`. Change `APP_NAME` in this project's `config.env`. | | `Port 8000 is already in use` | Another program (or another app) is listening on `PORT`. Change `PORT` in `.env`, or stop the other program. | | `running but not listening on port …` | The app started but isn't using `$PORT`. Check `ARGS` binds to `$PORT`, or raise `STARTUP_TIMEOUT`. | -| `logs` shows nothing / "No journal files were found" | Accept the `systemd-journal` group in `host-setup.sh`, then log out and back in. | +| `logs` shows nothing / "No journal files were found" | Re-run `./host-setup.sh` and accept storing the journal on disk. Check with `systemd-analyze cat-config systemd/journald.conf \| grep Storage=`: it should end with `persistent`, or with `auto` while `/var/log/journal` exists. If you chose the `systemd-journal` group instead, log out and back in. | | Apps stop when you log out / don't start at boot | Linger is off. Run `loginctl enable-linger $USER`. | | `Failed to connect to bus` | Re-run `./host-setup.sh` and open a new shell. | @@ -306,7 +327,7 @@ CI runs both on every pull request. ```bash # Remove each app (run in each project folder) -systemd-compose . down +systemd-compose down --volumes # Remove the command sudo rm /usr/local/bin/systemd-compose diff --git a/host-setup.sh b/host-setup.sh index 3f9d41b..73bb2b8 100644 --- a/host-setup.sh +++ b/host-setup.sh @@ -1,13 +1,5 @@ #!/bin/bash -# --- OS COMPATIBILITY CHECK --- -if ! command -v apt-get &> /dev/null; then - echo "❌ Error: This script supports Debian/Ubuntu based systems only." - echo " (Raspbian, DietPi, Ubuntu, Mint, etc.)" - echo " Reason: It relies on 'apt' for package management." - exit 1 -fi - # Dependencies: # python3-venv: Critical for creating isolated environments # python3-pip: Required for package management @@ -21,6 +13,91 @@ TARGET_LINK="/usr/local/bin/systemd-compose" SCRIPT_DIR="$(dirname "$(realpath "$0")")" SOURCE_SCRIPT="$SCRIPT_DIR/systemd-compose.sh" +JOURNAL_DIR="${JOURNAL_DIR:-/var/log/journal}" +JOURNALD_DROPIN_DIR="${JOURNALD_DROPIN_DIR:-/etc/systemd/journald.conf.d}" +JOURNALD_DROPIN="$JOURNALD_DROPIN_DIR/99-systemd-compose.conf" + +# --- HELPERS: LOG ACCESS --- + +# Prints journald's effective Storage= setting: auto, persistent, volatile or none +journal_storage_setting() { + local storage + storage=$(systemd-analyze cat-config systemd/journald.conf 2>/dev/null \ + | sed -n 's/^[[:space:]]*Storage=[[:space:]]*//p' | tail -n 1) + echo "${storage:-auto}" +} + +# journald only writes a separate journal per user (which that user may read +# without extra groups) when logs are stored on disk. +journal_is_persistent() { + case "$(journal_storage_setting)" in + persistent) return 0 ;; + auto) [ -d "$JOURNAL_DIR" ] ;; + *) return 1 ;; + esac +} + +enable_persistent_journal() { + sudo mkdir -p "$JOURNALD_DROPIN_DIR" && + printf '# Added by systemd-compose host-setup.sh\n[Journal]\nStorage=persistent\n' \ + | sudo tee "$JOURNALD_DROPIN" >/dev/null && + sudo systemctl restart systemd-journald && + sudo journalctl --flush +} + +# Makes sure `systemd-compose logs` can show the user's app logs. Prefers +# storing the journal on disk (each user reads only their own logs) over the +# systemd-journal group (which can read every log on the system). +ensure_log_access() { + if id -nG "$USER" | grep -qw "systemd-journal"; then + echo "✅ User is in the 'systemd-journal' group (logs visible)." + return 0 + fi + if journal_is_persistent; then + echo "✅ The journal is stored on disk, so you can read your apps' logs." + return 0 + fi + + echo "⚠️ The system journal is only kept in memory (Storage=$(journal_storage_setting))." + echo " In that mode you can't read your own apps' logs without extra permissions." + echo " Recommended: store the journal on disk. Each user can then read their own logs." + echo " (journald limits it to 10% of the disk, 4 GB at most.)" + read -p "❓ Store the journal on disk? (Requires sudo) [y/N] " -n 1 -r + echo "" + if [[ $REPLY =~ ^[Yy]$ ]]; then + if enable_persistent_journal && journal_is_persistent; then + echo "✅ The journal is now stored on disk ($JOURNALD_DROPIN)." + return 0 + fi + echo "❌ Couldn't switch the journal to disk. Another setting may override it:" + echo " systemd-analyze cat-config systemd/journald.conf" + fi + + echo " Alternative: join the 'systemd-journal' group. Note: it can read ALL system logs." + read -p "❓ Add user '$USER' to the group? (Requires sudo) [y/N] " -n 1 -r + echo "" + if [[ $REPLY =~ ^[Yy]$ ]]; then + sudo usermod -aG systemd-journal "$USER" + echo "✅ User added to group." + GROUPS_CHANGED=1 + else + echo "⚠️ Skipped. 'systemd-compose logs' may show nothing." + fi +} + +# The tests source this file to check the helpers above; stop here for them. +if [[ "${BASH_SOURCE[0]}" != "$0" ]]; then + return 0 +fi + +# --- OS COMPATIBILITY CHECK --- +if ! command -v apt-get &> /dev/null; then + echo "❌ Error: This script supports Debian/Ubuntu based systems only." + echo " (Raspbian, DietPi, Ubuntu, Mint, etc.)" + echo " Reason: It relies on 'apt' for package management." + exit 1 +fi + echo "=== 🛠️ Host Dependency Check ===" MISSING_DEPS=() @@ -129,21 +206,9 @@ fix_shell_environment() { ensure_logind_service fix_shell_environment -# --- ACTION: LOG PERMISSIONS --- +# --- ACTION: LOG ACCESS --- -if groups "$USER" | grep -q "systemd-journal"; then - echo "✅ User is in 'systemd-journal' group (Logs visible)." -else - echo "⚠️ User '$USER' is NOT in 'systemd-journal' group." - echo " You won't be able to see app logs without this." - read -p "❓ Add user to group now? (Requires sudo) [y/N] " -n 1 -r - echo "" - if [[ $REPLY =~ ^[Yy]$ ]]; then - sudo usermod -aG systemd-journal "$USER" - echo "✅ User added to group." - GROUPS_CHANGED=1 - fi -fi +ensure_log_access # --- ACTION: ENABLE LINGER --- diff --git a/systemd-compose.sh b/systemd-compose.sh index b9c9cbc..d88bf9f 100644 --- a/systemd-compose.sh +++ b/systemd-compose.sh @@ -21,10 +21,29 @@ RESERVED_ENV_KEYS=" PATH HOME USER SHELL IFS PWD OLDPWD UID EUID PPID SHLVL SCRI # --- HELPER FUNCTIONS --- +PROJECT_COMMANDS=" up down build run start stop restart status logs exec config " +GLOBAL_COMMANDS=" ps stop-all start-all " + usage() { echo "Usage:" - echo " Global: systemd-compose [ps | stop-all | start-all]" - echo " Project: systemd-compose [up|down|build|run|stop|restart|status|logs]" + echo " systemd-compose [project_folder] [options]" + echo "" + echo "Project commands (project_folder defaults to the current folder):" + echo " up Build the venv, then install and start the app" + echo " build Create the venv and install requirements.txt" + echo " run Write the service file and (re)start the app" + echo " restart Restart the app with the existing service file" + echo " stop Stop the app" + echo " down [--volumes] Stop and remove the service (--volumes also deletes the venv)" + echo " status Show state, PID, memory, restarts and port" + echo " logs [options] Follow the logs, or pass journalctl options (e.g. -n 100)" + echo " exec [args] Run a command with the app's venv and .env (e.g. exec python manage.py migrate)" + echo " config Print the service file that 'run' would write" + echo "" + echo "Global commands:" + echo " ps List all installed apps" + echo " stop-all Stop all installed apps" + echo " start-all Start all installed apps" } die() { @@ -32,10 +51,10 @@ die() { exit 1 } -# Reads a KEY=VALUE file the same way systemd's EnvironmentFile= does: -# nothing is executed, surrounding quotes are stripped, '#'/';' lines are comments. -# Each key becomes a (non-exported) shell variable, so config.env can use e.g. $PORT. -load_env_file() { +# Reads a KEY=VALUE file the same way systemd's EnvironmentFile= does and +# prints one KEY=VALUE per line: nothing is executed, surrounding quotes are +# stripped, '#'/';' lines are comments. Problems are reported on stderr. +parse_env_file() { local file="$1" line key value lineno=0 while IFS= read -r line || [ -n "$line" ]; do lineno=$((lineno + 1)) @@ -58,12 +77,22 @@ load_env_file() { if [[ "$value" =~ ^\"(.*)\"$ || "$value" =~ ^\'(.*)\'$ ]]; then value="${BASH_REMATCH[1]}" fi + printf '%s=%s\n' "$key" "$value" + done < "$file" +} + +# Turns each key in a .env file into a (non-exported) shell variable, so +# config.env can use e.g. $PORT. +load_env_file() { + local pair key + while IFS= read -r pair; do + key="${pair%%=*}" if [[ "$RESERVED_ENV_KEYS" == *" $key "* || "$key" == BASH* ]]; then - echo "⚠️ $file:$lineno: '$key' is reserved; systemd-compose will not read it." >&2 + echo "⚠️ $1: '$key' is reserved; systemd-compose will not read it." >&2 continue fi - printf -v "$key" '%s' "$value" - done < "$file" + printf -v "$key" '%s' "${pair#*=}" + done < <(parse_env_file "$1") } # Loads a project's .env and config.env into the current shell. @@ -88,6 +117,8 @@ load_project_config() { ARGS="${ARGS:-}" DESCRIPTION="${DESCRIPTION:-$APP_NAME}" STARTUP_TIMEOUT="${STARTUP_TIMEOUT:-15}" + MEMORY_MAX="${MEMORY_MAX:-}" + CPU_QUOTA="${CPU_QUOTA:-}" if ! [[ "$STARTUP_TIMEOUT" =~ ^[0-9]+$ ]]; then STARTUP_TIMEOUT=15 fi @@ -111,24 +142,56 @@ port_listening() { # --- ARGUMENT PARSING --- -# List of global commands that DO NOT require a project directory +# Accepts "systemd-compose " (current folder) as well as +# "systemd-compose ". Anything after the command +# is kept in EXTRA_ARGS for commands that take options (logs, down). +PROJECT_DIR_ARG="" case "${1:-}" in - ps|stop-all|start-all) - COMMAND="$1" + "") + usage + exit 1 + ;; + -h|--help|help) + usage + exit 0 ;; *) - if [ -z "${1:-}" ] || [ -z "${2:-}" ]; then + if [[ "$GLOBAL_COMMANDS" == *" $1 "* ]]; then + COMMAND="$1" + shift + elif [[ "$PROJECT_COMMANDS" == *" $1 "* ]]; then + PROJECT_DIR_ARG="." + COMMAND="$1" + shift + elif [ -n "${2:-}" ]; then + PROJECT_DIR_ARG="$1" + COMMAND="$2" + shift 2 + elif [ -d "$1" ]; then usage exit 1 + else + echo "Unknown command: $1" + usage + exit 1 + fi + ;; +esac +EXTRA_ARGS=("$@") + +# Only these commands take options; catch typos like "stop --all" +case "$COMMAND" in + logs|down|exec) ;; + *) + if [ ${#EXTRA_ARGS[@]} -gt 0 ]; then + die "'$COMMAND' does not take extra arguments: ${EXTRA_ARGS[*]}" fi - PROJECT_DIR_ARG="$1" - COMMAND="$2" ;; esac # --- PROJECT CONTEXT LOADING --- -if [[ "$COMMAND" != "ps" && "$COMMAND" != "stop-all" && "$COMMAND" != "start-all" ]]; then +if [ -n "$PROJECT_DIR_ARG" ]; then if [ ! -d "$PROJECT_DIR_ARG" ]; then die "Project directory '$PROJECT_DIR_ARG' not found." fi @@ -242,12 +305,22 @@ render_unit() { if [[ "$INSTALL_DIR" == *[\"\\]* ]]; then die "The project path can't contain '\"' or '\\': $INSTALL_DIR" fi + if [ -n "$MEMORY_MAX" ] && ! [[ "$MEMORY_MAX" =~ ^[0-9]+[KMGT]?$|^[0-9]+%$|^infinity$ ]]; then + die "MEMORY_MAX='$MEMORY_MAX' is invalid. Use a size like 300M or 1G, a percentage like 25%, or leave it empty." + fi + if [ -n "$CPU_QUOTA" ] && ! [[ "$CPU_QUOTA" =~ ^[0-9]+%$ ]]; then + die "CPU_QUOTA='$CPU_QUOTA' is invalid. Use a percentage of one core, like 50% (or 200% for two cores)." + fi while IFS= read -r line || [ -n "$line" ]; do key="${line%%=*}" - for var in DESCRIPTION INSTALL_DIR ENTRYPOINT APP_DIR ARGS PORT; do + for var in DESCRIPTION INSTALL_DIR ENTRYPOINT APP_DIR ARGS PORT MEMORY_MAX CPU_QUOTA; do value="${!var}" - value="${value//%/%%}" + # MemoryMax=/CPUQuota= take "%" literally (no specifiers); their + # values are validated above, so they are written as-is. + if [[ "$var" != "MEMORY_MAX" && "$var" != "CPU_QUOTA" ]]; then + value="${value//%/%%}" + fi if [ "$key" == "ExecStart" ]; then value="${value//\$/\$\$}" fi @@ -288,6 +361,22 @@ wait_for_startup() { done } +# Limits only work when the kernel lets the user manager control memory/CPU +# (cgroup v2 with the controllers delegated). Warn instead of failing silently. +warn_unenforced_limits() { + local uid controllers_file controllers + uid=$(id -u) + controllers_file="/sys/fs/cgroup/user.slice/user-${uid}.slice/user@${uid}.service/cgroup.controllers" + [ -r "$controllers_file" ] || return 0 + controllers=" $(cat "$controllers_file") " + if [ -n "$MEMORY_MAX" ] && [[ "$controllers" != *" memory "* ]]; then + echo "⚠️ MEMORY_MAX is set, but the memory controller isn't available to user services, so it won't be enforced." + fi + if [ -n "$CPU_QUOTA" ] && [[ "$controllers" != *" cpu "* ]]; then + echo "⚠️ CPU_QUOTA is set, but the cpu controller isn't available to user services, so it won't be enforced." + fi +} + # Memory of the whole service (all workers), not just the main process. service_memory() { local bytes cgroup pids @@ -349,6 +438,7 @@ run() { echo "--- 🚀 Starting ${APP_NAME} ---" warn_config_exports + warn_unenforced_limits if ! check_binary; then exit 1; fi # If this app is already installed, stop it first so a port it holds @@ -427,7 +517,7 @@ status() { if [ "$IS_ACTIVE" == "active" ]; then MAIN_PID=$(systemctl --user show --property MainPID --value "${UNIT}.service") echo "Main PID: $MAIN_PID" - echo "Memory Usage: $(service_memory)" + echo "Memory Usage: $(service_memory)${MEMORY_MAX:+ (limit $MEMORY_MAX)}" echo "Restarts: $(systemctl --user show --property NRestarts --value "${UNIT}.service")" fi @@ -451,21 +541,79 @@ up() { } down() { + local remove_venv=0 arg + for arg in "${EXTRA_ARGS[@]}"; do + case "$arg" in + -v|--volumes) remove_venv=1 ;; + *) die "Unknown option for down: $arg (did you mean --volumes?)" ;; + esac + done + echo "=== DOWN: ${APP_NAME} ===" stop systemctl --user disable --quiet "${UNIT}.service" 2>/dev/null rm -f "$SERVICE_FILE" systemctl --user daemon-reload systemctl --user reset-failed "${UNIT}.service" 2>/dev/null - if [ -d "$INSTALL_DIR/venv" ]; then + if [ "$remove_venv" == "1" ] && [ -d "$INSTALL_DIR/venv" ]; then echo "Removing venv..." rm -rf "$INSTALL_DIR/venv" fi echo "Cleanup complete." } +# With no options, follow the logs. Otherwise pass the options to journalctl, +# e.g. "logs -n 100" or "logs --since today -f". logs() { - journalctl --user -u "${UNIT}.service" -f + if [ ${#EXTRA_ARGS[@]} -eq 0 ]; then + EXTRA_ARGS=(-f) + fi + journalctl --user -u "${UNIT}.service" "${EXTRA_ARGS[@]}" +} + +# Runs a one-off command the way the service runs the app: venv first on +# PATH, the variables from .env, in the app's working directory. +exec_command() { + if [ ${#EXTRA_ARGS[@]} -eq 0 ]; then + die "Usage: systemd-compose [project_folder] exec [args...]" + fi + if [ ! -d "$INSTALL_DIR/venv" ]; then + die "No venv in $INSTALL_DIR. Run 'build' first." + fi + + local env_vars=() pair + if [ -f "$INSTALL_DIR/.env" ]; then + while IFS= read -r pair; do + env_vars+=("$pair") + done < <(parse_env_file "$INSTALL_DIR/.env" 2>/dev/null) + fi + + cd "$INSTALL_DIR/$APP_DIR" || die "Can't enter $INSTALL_DIR/$APP_DIR" + # Same order as the service: .env comes last, so it wins (as in systemd) + exec env -- \ + "PATH=$INSTALL_DIR/venv/bin:$PATH" \ + "VIRTUAL_ENV=$INSTALL_DIR/venv" \ + PYTHONUNBUFFERED=1 \ + PYTHONDONTWRITEBYTECODE=1 \ + "${env_vars[@]}" \ + "${EXTRA_ARGS[@]}" +} + +# Prints the service file "run" would write, and whether it differs from +# the installed one. Nothing is changed. +show_config() { + local unit_content + unit_content=$(render_unit) || exit 1 + echo "# $SERVICE_FILE" + printf '%s\n' "$unit_content" + echo "" + if [ ! -f "$SERVICE_FILE" ]; then + echo "# Not installed yet. Run 'up' or 'run' to install it." + elif [ "$(cat "$SERVICE_FILE")" == "$unit_content" ]; then + echo "# The installed service file is up to date." + else + echo "# The installed service file is different. Run 'run' to apply these changes." + fi } # --- GLOBAL DASHBOARD COMMANDS --- @@ -572,9 +720,10 @@ ps_dashboard() { # --- DISPATCHER --- -# Every project command acts on the service named APP_NAME, so make sure -# that name is not owned by a different project folder first. -if [ -n "${SERVICE_FILE:-}" ]; then +# Every command that manages the service acts on the service named APP_NAME, +# so make sure that name is not owned by a different project folder first. +# exec and config don't touch the service. +if [ -n "${SERVICE_FILE:-}" ] && [[ "$COMMAND" != "exec" && "$COMMAND" != "config" ]]; then if ! check_name_collision; then exit 1; fi fi @@ -587,6 +736,8 @@ case "$COMMAND" in restart) restart ;; status) status ;; logs) logs ;; + exec) exec_command ;; + config) show_config ;; ps) ps_dashboard ;; stop-all) stop_all ;; start-all) start_all ;; diff --git a/templates/service.unit b/templates/service.unit index 2d47164..87d5b5d 100644 --- a/templates/service.unit +++ b/templates/service.unit @@ -29,6 +29,10 @@ ExecStart="${INSTALL_DIR}/venv/bin/${ENTRYPOINT}" ${ARGS} # Working directory can be root (.) or a subfolder (app) WorkingDirectory=${INSTALL_DIR}/${APP_DIR} +# Resource limits from MEMORY_MAX / CPU_QUOTA in config.env (empty = no limit) +MemoryMax=${MEMORY_MAX} +CPUQuota=${CPU_QUOTA} + # Restart after crashes, not after a clean exit or `systemd-compose stop` Restart=on-failure RestartSec=5 diff --git a/tests/features.bats b/tests/features.bats new file mode 100644 index 0000000..1b60c6f --- /dev/null +++ b/tests/features.bats @@ -0,0 +1,189 @@ +#!/usr/bin/env bats +# Default project folder, help, logs/down options, exec, config, limits + +# Single quotes below are intentional: "$VAR" must be expanded by the tool. +# shellcheck disable=SC2016 + +load test_helper + +@test "the project folder defaults to the current folder" { + make_app "$TEST_ROOT/app" app false + cd "$TEST_ROOT/app" + run bash "$SDC" up + [ "$status" -eq 0 ] + [ -f "$UNIT_DIR/sdc-app.service" ] + run bash "$SDC" status + [ "$status" -eq 0 ] + assert_output_contains "Service State: active" +} + +@test "-h and --help print usage and succeed" { + for flag in -h --help help; do + run bash "$SDC" "$flag" + [ "$status" -eq 0 ] + assert_output_contains "Project commands" + done +} + +@test "a folder without a command prints usage and fails" { + mkdir -p "$TEST_ROOT/app" + run bash "$SDC" "$TEST_ROOT/app" + [ "$status" -eq 1 ] + assert_output_contains "Usage:" +} + +@test "an unknown single word is an unknown command" { + run bash "$SDC" frobnicate + [ "$status" -eq 1 ] + assert_output_contains "Unknown command: frobnicate" +} + +@test "commands without options reject extra arguments" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" stop --all + [ "$status" -eq 1 ] + assert_output_contains "does not take extra arguments" + run bash "$SDC" ps extra + [ "$status" -eq 1 ] +} + +@test "logs follows by default" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" logs + [ "$status" -eq 0 ] + assert_output_contains "[journal: --user -u sdc-app.service -f]" +} + +@test "logs passes options to journalctl" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" logs -n 100 --since today + [ "$status" -eq 0 ] + assert_output_contains "[journal: --user -u sdc-app.service -n 100 --since today]" +} + +@test "down keeps the venv" { + make_app "$TEST_ROOT/app" app false + bash "$SDC" "$TEST_ROOT/app" run + run bash "$SDC" "$TEST_ROOT/app" down + [ "$status" -eq 0 ] + [ ! -f "$UNIT_DIR/sdc-app.service" ] + [ -d "$TEST_ROOT/app/venv" ] +} + +@test "down --volumes also deletes the venv" { + make_app "$TEST_ROOT/app" app false + bash "$SDC" "$TEST_ROOT/app" run + run bash "$SDC" "$TEST_ROOT/app" down --volumes + [ "$status" -eq 0 ] + [ ! -d "$TEST_ROOT/app/venv" ] +} + +@test "down rejects unknown options without removing anything" { + make_app "$TEST_ROOT/app" app false + bash "$SDC" "$TEST_ROOT/app" run + run bash "$SDC" "$TEST_ROOT/app" down --volume + [ "$status" -eq 1 ] + assert_output_contains "did you mean --volumes" + [ -f "$UNIT_DIR/sdc-app.service" ] +} + +@test "exec runs in the app folder with the venv and .env" { + make_app "$TEST_ROOT/app" app false 8000 src + printf 'SECRET="s3cret"\n' >> "$TEST_ROOT/app/.env" + printf '#!/bin/sh\necho "tool: $*"\n' > "$TEST_ROOT/app/venv/bin/mytool" + chmod +x "$TEST_ROOT/app/venv/bin/mytool" + + run bash "$SDC" "$TEST_ROOT/app" exec mytool migrate --yes + [ "$status" -eq 0 ] + assert_output_contains "tool: migrate --yes" + + run bash "$SDC" "$TEST_ROOT/app" exec sh -c 'echo "$PWD|$SECRET|$PORT|$VIRTUAL_ENV"' + [ "$status" -eq 0 ] + [ "$output" == "$TEST_ROOT/app/src|s3cret|8000|$TEST_ROOT/app/venv" ] +} + +@test "exec returns the command's exit code" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" exec sh -c 'exit 7' + [ "$status" -eq 7 ] +} + +@test "exec without a command or venv gives a clear error" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" exec + [ "$status" -eq 1 ] + assert_output_contains "Usage: systemd-compose [project_folder] exec" + + rm -rf "$TEST_ROOT/app/venv" + run bash "$SDC" "$TEST_ROOT/app" exec python + [ "$status" -eq 1 ] + assert_output_contains "Run 'build' first" +} + +@test "config prints the service file without installing it" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" config + [ "$status" -eq 0 ] + assert_output_contains "ExecStart=" + assert_output_contains "Not installed yet" + [ ! -f "$UNIT_DIR/sdc-app.service" ] +} + +@test "config says whether the installed service is up to date" { + make_app "$TEST_ROOT/app" app false + bash "$SDC" "$TEST_ROOT/app" run + run bash "$SDC" "$TEST_ROOT/app" config + assert_output_contains "up to date" + + sed -i 's/^DESCRIPTION=.*/DESCRIPTION="changed"/' "$TEST_ROOT/app/config.env" + run bash "$SDC" "$TEST_ROOT/app" config + assert_output_contains "Run 'run' to apply" +} + +@test "MEMORY_MAX and CPU_QUOTA end up in the service file" { + make_app "$TEST_ROOT/app" app false + printf 'MEMORY_MAX="300M"\nCPU_QUOTA="50%%"\n' >> "$TEST_ROOT/app/config.env" + run bash "$SDC" "$TEST_ROOT/app" run + [ "$status" -eq 0 ] + grep -q '^MemoryMax=300M$' "$UNIT_DIR/sdc-app.service" + grep -q '^CPUQuota=50%$' "$UNIT_DIR/sdc-app.service" + + run bash "$SDC" "$TEST_ROOT/app" status + assert_output_contains "(limit 300M)" +} + +@test "systemd parses the limits (percentages are not escaped)" { + if ! command -v systemd-analyze >/dev/null; then + skip "systemd-analyze is not installed" + fi + make_app "$TEST_ROOT/app" app false + ln -sf "$(type -P true)" "$TEST_ROOT/app/venv/bin/python" + printf 'MEMORY_MAX="25%%"\nCPU_QUOTA="50%%"\n' >> "$TEST_ROOT/app/config.env" + bash "$SDC" "$TEST_ROOT/app" run + cp "$UNIT_DIR/sdc-app.service" "$TEST_ROOT/sdc-app.service" + + run env SYSTEMD_LOG_LEVEL=debug systemd-analyze verify "$TEST_ROOT/sdc-app.service" + refute_output_contains "Invalid" + assert_output_contains "CPUQuotaPerSecSec: 500ms" +} + +@test "no limits by default" { + make_app "$TEST_ROOT/app" app false + run bash "$SDC" "$TEST_ROOT/app" run + grep -q '^MemoryMax=$' "$UNIT_DIR/sdc-app.service" + grep -q '^CPUQuota=$' "$UNIT_DIR/sdc-app.service" +} + +@test "invalid limits are rejected" { + make_app "$TEST_ROOT/app" app false + echo 'MEMORY_MAX="lots"' >> "$TEST_ROOT/app/config.env" + run bash "$SDC" "$TEST_ROOT/app" run + [ "$status" -eq 1 ] + assert_output_contains "MEMORY_MAX='lots' is invalid" + + sed -i '/^MEMORY_MAX=/d' "$TEST_ROOT/app/config.env" + echo 'CPU_QUOTA="50"' >> "$TEST_ROOT/app/config.env" + run bash "$SDC" "$TEST_ROOT/app" run + [ "$status" -eq 1 ] + assert_output_contains "CPU_QUOTA='50' is invalid" +} diff --git a/tests/host_setup.bats b/tests/host_setup.bats new file mode 100644 index 0000000..7728d3f --- /dev/null +++ b/tests/host_setup.bats @@ -0,0 +1,98 @@ +#!/usr/bin/env bats +# host-setup.sh: making app logs readable (persistent journal vs. group) + +# Single quotes below are intentional: the fake tools expand "$@" themselves. +# shellcheck disable=SC2016 + +load test_helper + +setup() { + TEST_ROOT="$(mktemp -d)" + export STUB_STATE="$TEST_ROOT/state" + export JOURNAL_DIR="$TEST_ROOT/var/log/journal" + export JOURNALD_DROPIN_DIR="$TEST_ROOT/etc/journald.conf.d" + export JOURNALD_CONF="$TEST_ROOT/etc/journald.conf" + export STUB_GROUPS="user" + mkdir -p "$STUB_STATE" "$TEST_ROOT/bin" "$TEST_ROOT/etc" + printf '[Journal]\n#Storage=auto\n' > "$JOURNALD_CONF" + + # Fakes for the system tools ensure_log_access uses + cat > "$TEST_ROOT/bin/systemd-analyze" <<'EOF' +#!/bin/bash +# Effective config = main file, then drop-ins in name order (like systemd) +cat "$JOURNALD_CONF" +for f in "$JOURNALD_DROPIN_DIR"/*.conf; do [ -f "$f" ] && cat "$f"; done +exit 0 +EOF + printf '#!/bin/bash\necho "sudo $*" >> "$STUB_STATE/calls"\nexec "$@"\n' > "$TEST_ROOT/bin/sudo" + printf '#!/bin/bash\necho "systemctl $*" >> "$STUB_STATE/calls"\n' > "$TEST_ROOT/bin/systemctl" + printf '#!/bin/bash\necho "usermod $*" >> "$STUB_STATE/calls"\n' > "$TEST_ROOT/bin/usermod" + printf '#!/bin/bash\nif [ "$1" == "-nG" ]; then echo "$STUB_GROUPS"; else exec /usr/bin/id "$@"; fi\n' > "$TEST_ROOT/bin/id" + chmod +x "$TEST_ROOT/bin/"* + export PATH="$TEST_ROOT/bin:$REPO_DIR/tests/stubs:$PATH" +} + +# Runs ensure_log_access with the given answers to its y/N prompts +log_access() { + run bash -c "source '$REPO_DIR/host-setup.sh' && ensure_log_access" <<< "$1" +} + +@test "sourcing host-setup.sh only defines functions" { + run bash -c "source '$REPO_DIR/host-setup.sh' && type -t ensure_log_access" + [ "$status" -eq 0 ] + [ "$output" == "function" ] +} + +@test "Storage=auto with /var/log/journal counts as persistent" { + mkdir -p "$JOURNAL_DIR" + log_access "" + [ "$status" -eq 0 ] + assert_output_contains "stored on disk, so you can read" + [ ! -f "$STUB_STATE/calls" ] +} + +@test "members of systemd-journal are not asked anything" { + STUB_GROUPS="user systemd-journal" log_access "" + [ "$status" -eq 0 ] + assert_output_contains "'systemd-journal' group (logs visible)" +} + +@test "an in-memory journal is switched to disk when accepted" { + log_access "y" + [ "$status" -eq 0 ] + assert_output_contains "only kept in memory (Storage=auto)" + assert_output_contains "now stored on disk" + grep -q '^Storage=persistent$' "$JOURNALD_DROPIN_DIR/99-systemd-compose.conf" + grep -q 'systemctl restart systemd-journald' "$STUB_STATE/calls" + refute_output_contains "Alternative" +} + +@test "an explicit Storage=volatile is overridden by the drop-in" { + printf '[Journal]\nStorage=volatile\n' > "$JOURNALD_CONF" + log_access "y" + assert_output_contains "(Storage=volatile)" + assert_output_contains "now stored on disk" +} + +@test "a later drop-in that keeps volatile storage is reported" { + mkdir -p "$JOURNALD_DROPIN_DIR" + printf '[Journal]\nStorage=volatile\n' > "$JOURNALD_DROPIN_DIR/zz-ramlog.conf" + log_access "yn" + assert_output_contains "Couldn't switch the journal to disk" + assert_output_contains "Alternative: join the 'systemd-journal' group" +} + +@test "declining disk storage offers the group as a fallback" { + log_access "ny" + [ "$status" -eq 0 ] + assert_output_contains "can read ALL system logs" + grep -q 'usermod -aG systemd-journal' "$STUB_STATE/calls" + [ ! -f "$JOURNALD_DROPIN_DIR/99-systemd-compose.conf" ] +} + +@test "declining both changes nothing" { + log_access "nn" + [ "$status" -eq 0 ] + assert_output_contains "may show nothing" + [ ! -f "$STUB_STATE/calls" ] +}