diff --git a/AGENTS.md b/AGENTS.md index e2767aae2..ee5525e88 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -42,7 +42,7 @@ uv run cli db-backup # Backup database ### Tech Stack - **Backend**: Python 3.12+, Django 6.1+, PostgreSQL 17 -- **Search**: pg_vector (semantic), pg_search (full-text), hybrid ranking +- **Search**: PostgreSQL full-text search (tsvector), pg_vector (semantic), hybrid RRF ranking - **Async**: Daphne (ASGI), Django Channels, Procrastinate (task queue) - **Frontend**: Django templates, Cotton components, HTMX, Alpine.js, Bootstrap 5 - **LLM**: External OpenAI-compatible API endpoint @@ -86,7 +86,7 @@ Analysis operations follow a Job -> Task pattern (similar to ADIT): - **default_worker**: General background task processor (Procrastinate queue: `default`) - **llm_worker**: LLM-specific task processor (Procrastinate queue: `llm`) - **embeddings_worker**: Embedding task processor (Procrastinate queue: `embeddings`) -- **postgres**: PostgreSQL 17 with pg_vector and pg_search extensions (port 5432) +- **postgres**: PostgreSQL 17 with the pg_vector extension (port 5432) ### LLM Endpoint @@ -155,6 +155,9 @@ Labeling uses the shared core LLM client (`radis.core.utils.llm_client`); its ti - **Comments**: only where the code cannot speak for itself; explain *why*, not *what* - **No history in comments**: describe the code as it is, not how it changed — that belongs in the commit message (docstrings too) +- **Keep the docs in sync**: when a change adds a feature or alters behaviour that the + docs describe (`README.md`, `docs/`, this file, in-app help templates), update them in + the same PR ## Key Dependencies @@ -219,7 +222,7 @@ reports = response.json() ### Search Not Returning Expected Results -- Check PostgreSQL extensions are installed: `pg_vector`, `pg_search` +- Check the `vector` extension is installed in PostgreSQL - Verify report has `body` text indexed - Check search provider is configured in settings - Review QueryParser syntax for complex queries diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1d9dcca89..61e9e7c32 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing to Our Project -We're excited that you're interested in contributing to our project! This document outlines the +Thank you for contributing to RADIS. This document outlines the guidelines for contributing to our codebase. We follow the Google Python Style Guide to maintain consistency and readability across our project. @@ -17,7 +17,7 @@ cp ./example.env ./.env # adjust the environment variables to your needs uv run cli compose-up -- --watch ``` -The development server of the example project will be started on +The development server is then available on File changes will be automatically detected and the servers will be restarted. When library dependencies are changed, the containers will automatically be rebuilt and restarted. diff --git a/README.md b/README.md index d46704dd6..f870e1c5b 100644 --- a/README.md +++ b/README.md @@ -35,28 +35,29 @@ The detailed documentation of RADIS can be found at ` under Docker Swarm. + +**Web Container (`web`)**: Runs Django application serving web UI and REST API. Ports: 8000 (dev), 80/443 (prod with SSL). Handles authentication, serves static files, enqueues tasks, and manages database connections. In production, runs with 3 replicas for high availability. -**PostgreSQL Container (`radis-postgres-1`)**: PostgreSQL database storing all data (users, reports, collections, subscriptions, tasks, logs, Procrastinate queue). Port 5432. Uses Docker volumes for persistence. +**PostgreSQL Container (`postgres`)**: PostgreSQL database storing all data (users, reports, collections, subscriptions, tasks, logs, Procrastinate queue). Port 5432. Uses Docker volumes for persistence. -**Default Worker Container (`radis-default_worker-1`)**: Processes background tasks in the default queue (e.g., extraction job preparation, subscription job preparation, periodic subscription launcher, disk space checks, database backups). +**Default Worker Container (`default_worker`)**: Processes background tasks in the default queue (job preparation for extractions, subscriptions and labeling, the periodic tasks listed above, database backups). -**LLM Worker Container (`radis-llm_worker-1`)**: Executes AI-intensive tasks from the llm queue (extraction tasks, subscription tasks). Uses ChatClient to communicate with the configured LLM endpoint. +**LLM Worker Container (`llm_worker`)**: Executes AI-intensive tasks from the llm queue (extraction, subscription and labeling tasks). Uses `LLMClient` (`radis.core.utils.llm_client`) to talk to the configured LLM endpoint. -**Embeddings Worker Container (`radis-embeddings_worker-1`)**: Drains the embeddings +**Embeddings Worker Container (`embeddings_worker`)**: Drains the embeddings queue — generating and storing report vectors for hybrid search, including operator backfills started by `./manage.py embed_pending` or the admin action. @@ -77,7 +82,7 @@ directly instead of the client truncating a larger vector. `EMBEDDINGS_BASE_URL` both when the provider multiplexes models (OpenAI, Ollama, a gateway), while a self-hosted vLLM or SGLang serves one model per process and needs the override. The service has its own rate-limit gate — a 429 from the embedding gateway must not pause -inference — and its own worker (`radis-embeddings_worker-1`) draining the `embeddings` +inference — and its own worker (`embeddings_worker`) draining the `embeddings` queue, so a million-report backfill cannot starve extractions. ## Search Architecture @@ -86,22 +91,26 @@ RADIS uses a modular search architecture allowing different search providers to **Search Provider Interface**: Defines search, retrieval, and indexing operations -- **PgSearch Provider**: Default implementation using PostgreSQL full-text search with pg_search and pg_vector extensions -- **Alternative Providers**: Vespa and ElasticSearch can be integrated through the same interface +- **PgSearch Provider**: Default implementation using PostgreSQL full-text search and pg_vector +- **Alternative Providers**: Other engines such as Vespa or ElasticSearch can be integrated through the same interface **Query Parser**: Parses user queries with support for: -- AND/OR operators +- `AND`, `OR` and `NOT` operators (uppercase; implicit AND between terms) - Phrase search ("exact match") -- Exclusion (-term) +- Grouping with parentheses - Case-insensitive matching +Unsupported syntax (`-term`, `field:value`, wildcards) is stripped and the corrected query is shown to the user. + **Search Filters**: Applied on top of query: - Language, modalities, study date range - Study description, patient sex, patient age range -- Patient ID, group access -- Created after timestamp (for subscriptions) +- Patient ID, labels, group access +- Created before/after and updated after timestamps (for subscriptions and labeling) + +**Hybrid ranking**: When `EMBEDDINGS_MODEL` is set, the provider runs the full-text query (up to `HYBRID_FTS_MAX_RESULTS` candidates) and a cosine-distance nearest-neighbour query on the report vectors (`HYBRID_VECTOR_TOP_K`), then fuses both lists with reciprocal rank fusion (`HYBRID_RRF_K`). `NOT` terms are removed before the query is embedded, and query embeddings are cached. If the embedding service fails, the search degrades to full-text only. **Text-search configurations**: reports are indexed with the PostgreSQL configuration for their own language, so stemming matches the text — an English report stores "effusion" as @@ -114,3 +123,11 @@ does no stemming and therefore matches literally. An unset language filter is no to subscriptions' "All" choice: the search form's language field is optional with no explicit empty option, so a bookmarked or shared search URL missing `language=` — or the search page's first, filter-less load — resolves the same way. + +## Auto-Labeling + +The `labels` app classifies reports against administrator-defined labels. A `LabelGroup` has a Yes/No gate question; a report only gets the group's labels classified when the gate answers Yes. Each `Label` result is one of `PRESENT`, `LIKELY`, `POSSIBLE`, `ABSENT` or `UNMENTIONED`; the first three appear as badges and in the search filter. Work runs as `LabelingJob`/`LabelingTask` through the usual job/task pattern, either as a manual backfill from Django Admin or through the nightly `incremental_label_scan`, which labels reports updated since the `LabelingScanCheckpoint`. A result is stale when its label or report changed after it was generated. + +## Worker Crash Recovery + +A task left `IN_PROGRESS` by a killed worker is repaired by `radis.core.utils.recovery`: it is reset to `PENDING` and re-enqueued once its worker has sent no heartbeat for `ANALYSIS_STALLED_WORKER_GRACE_SECONDS`. The sweep runs on worker startup (`sweep_stale_tasks`) and periodically (`ANALYSIS_SWEEP_CRON`). A job whose preparation was interrupted is retried by the queue and starts preparation over, discarding half-created tasks. Embedding subjobs are plain queue jobs and rely on Procrastinate retries instead. diff --git a/docs/dev-docs/contributing.md b/docs/dev-docs/contributing.md index ecefb946b..f036a6200 100644 --- a/docs/dev-docs/contributing.md +++ b/docs/dev-docs/contributing.md @@ -6,14 +6,15 @@ consistency and readability across our project. Code Style We adhere to the Google Python Style [Guide](https://google.github.io/styleguide/pyguide.html). -This repository includes a [Dev Container](https://code.visualstudio.com/docs/devcontainers/create-dev-container). -If you open the project in VS Code after cloning, you should see a prompt: +This repository includes a [Dev Container](https://code.visualstudio.com/docs/devcontainers/create-dev-container) +with Python, uv and the Docker CLI. If you open the project in VS Code after cloning, you should see a prompt: “Reopen in Dev Container” -Click it, and VS Code will automatically build and open the development environment. - -The development server of the example project will be started on +Click it, and VS Code builds and opens the development environment. The dev container does not +run Docker itself: it talks to the Docker daemon of your machine, so the RADIS containers started +below run on your host next to the dev container. Docker Desktop, OrbStack or a native Docker +installation is required either way. ## Getting Started @@ -25,7 +26,7 @@ cp ./example.env ./.env # adjust the environment variables to your needs uv run cli compose-up -- --watch ``` -File changes will be automatically detected and the servers will be restarted. When library dependencies are changed, the containers will automatically be rebuilt and restarted. +The development server is then available on . File changes are detected automatically and the servers restart. When library dependencies change, the containers are rebuilt and restarted. ### LLM Setup @@ -190,9 +191,8 @@ uv run cli compose-up # restart containers (migrations run automatically) - For major database schema changes, consider backing up first: `uv run cli db-backup` !!! note "Development vs Production" - -**Development**: Use `uv run cli compose-up` for local development -**Production**: Use `uv run cli stack-deploy` for production deployment with Docker Swarm + **Development**: Use `uv run cli compose-up` for local development. + **Production**: Use `uv run cli stack-deploy` for production deployment with Docker Swarm. ## Reporting Issues diff --git a/docs/index.md b/docs/index.md index e1db8906a..574df9da5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -48,11 +48,12 @@ RADIS serves as a comprehensive platform for managing radiology reports, enablin **RADIS acts as an intelligent hub** for radiology reports: 1. **Report Storage**: Securely store radiology reports with structured metadata -2. **Advanced Search**: Use hybrid search combining traditional text search with semantic understanding -3. **AI-Powered Analysis**: Leverage large language models for intelligent filtering and categorization -4. **Collections**: Organize reports into custom collections for easy access and review -5. **Subscription**: Subscribe to searches and get notified when new matching reports arrive -6. **Notes**: Add personal notes to reports for annotations and context +2. **Advanced Search**: Full-text search with filters, optionally combined with semantic search for meaning-based retrieval +3. **Automated Labeling**: Large language models assign administrator-defined labels to reports for browsing and filtering +4. **Data Extraction**: Extract structured fields from many reports at once and export them as CSV +5. **Subscriptions**: Subscribe to searches and get notified when new matching reports arrive, with optional AI-based filtering and extraction +6. **Collections and Notes**: Organize reports into custom collections and add personal notes for context +7. **Chats**: Ask an AI assistant questions about a report **Ready to modernize your radiology workflow?** RADIS combines traditional database functionality with cutting-edge AI to make radiology reports more accessible and actionable. diff --git a/docs/maintenance.md b/docs/maintenance.md index 27a0c0e51..741ff359d 100644 --- a/docs/maintenance.md +++ b/docs/maintenance.md @@ -8,11 +8,6 @@ There are different things that can be upgraded: - Check outdated Python packages: `uv run cli show-outdated` (check Python section in output) - `uv lock --upgrade` will update packages according to their version range in `pyproject.toml` - Other upgrades (e.g. major versions) must be upgraded by modifying the version range in `pyproject.toml` before calling `uv lock --upgrade` -- Javascript dependencies - - Check outdated Javascript packages: `uv run cli show-outdated` (check Javascript section in output) - - `npm update` will update packages according to their version range in `package.json` - - Other upgrades (e.g. major versions) must be upgraded by modifying the version range in `packages.json` before calling `npm update` - - After an upgrade make sure the files in `static/vendor` still link to the correct files in `node_modules`1 - Python and uv in `Dockerfile` that builds the container where RADIS runs in - Dependent services in `docker-compose.base.yml`, like PostgreSQL - Github Codespaces development container dependencies in `.devcontainer/devcontainer.json` and `.devcontainer/Dockerfile` @@ -31,8 +26,8 @@ the compose files no longer define: uv run cli compose-down -- --remove-orphans ``` -A model cache volume outlives them and can be reclaimed with -`docker volume rm radis_dev_models_data`. +A model cache volume of such a container outlives it; find it with `docker volume ls` and +remove it with `docker volume rm`. **Production (Docker Swarm)**: pass `--prune`, which removes services the deployment no longer declares. Only do so when deploying the complete set of compose files, since Swarm @@ -45,14 +40,16 @@ uv run cli stack-deploy -- --prune To look first, or to clean up without redeploying: ```terminal -docker service ls | grep llm # e.g. radis_llm_gpu -docker service rm radis_llm_gpu # use your stack name if it is not 'radis' -docker volume rm radis_models_data # on each node that holds model files +docker service ls # look for a service the compose files do not define +docker service rm +docker volume ls # and its model volume, on each node that holds one +docker volume rm ``` ## Search language configs RADIS reads available text search configs from Postgres (`pg_ts_config`) and auto-maps language codes to matching configs (falling back to `simple`). If new dictionaries/configs -are installed in Postgres, restart RADIS to refresh the config cache, and reindex reports -to apply the new config to existing data. +are installed in Postgres, run `./manage.py refresh_search_configs` to clear the cache (no +restart needed). Existing reports keep their old index until they are saved again; there is +no reindex command. diff --git a/docs/user-docs/admin-guide.md b/docs/user-docs/admin-guide.md index 0b0d680b6..2fbe26360 100644 --- a/docs/user-docs/admin-guide.md +++ b/docs/user-docs/admin-guide.md @@ -4,28 +4,119 @@ The Admin Guide is intended for system administrators and technical staff respon ## Installation +On the server RADIS lives in a production folder that holds the checkout and the `.env` file. The name is up to you; `radis_prod` below is only an example, use whatever you called your production folder wherever this guide says "production folder": + ```terminal -Clone the repository: git clone https://github.com/openradx/radis.git -cd radis +git clone https://github.com/openradx/radis.git radis_prod +cd radis_prod # or whatever you named your production folder uv sync -cp ./example.env ./.env # copy example environment to .env -uv run cli stack-deploy # builds and starts Docker containers +cp ./example.env ./.env # set ENVIRONMENT=production and adjust the variables (see below) +uv run cli compose-pull # pulls the Docker image (RADIS_IMAGE, default ghcr.io/openradx/radis:latest) +uv run cli stack-deploy # starts the Docker Swarm stack +``` + +`stack-deploy` refuses to run unless `ENVIRONMENT=production` is set in `.env`. It does not build anything; the stack runs the pre-built image named by `RADIS_IMAGE`. + +### Environment Variables + +All settings are read from `.env`; the comments in `example.env` describe every variable. For production at least set: + +- `ENVIRONMENT=production` +- Secrets: `DJANGO_SECRET_KEY`, `POSTGRES_PASSWORD`, `TOKEN_AUTHENTICATION_SALT`, `SUPERUSER_PASSWORD`, `SUPERUSER_AUTH_TOKEN` (generate with `uv run cli generate-django-secret-key`, `generate-secure-password`, `generate-auth-token`) +- Hosts: `DJANGO_ALLOWED_HOSTS`, `DJANGO_CSRF_TRUSTED_ORIGINS`, `SITE_DOMAIN`, `SITE_NAME` +- LLM: `LLM_BASE_URL`, `LLM_DEFAULT_MODEL` (both required, RADIS refuses to start without them; see [LLM and Embeddings Configuration](#llm-and-embeddings-configuration)). The `host.docker.internal` default from `example.env` is a development convenience that is not injected into the production stack, so point `LLM_BASE_URL` at an endpoint the stack's nodes can reach. `EMBEDDINGS_MODEL` is optional; leave it empty for full-text search only +- SSL: `SSL_SERVER_CERT_FILE`, `SSL_SERVER_KEY_FILE`, `SSL_SERVER_CHAIN_FILE` (`uv run cli generate-certificate-chain` builds the chain from your CA-signed certificate) +- Email: `DJANGO_EMAIL_URL`, `DJANGO_SERVER_EMAIL`, `DJANGO_ADMIN_EMAIL`, `DJANGO_ADMIN_FULL_NAME`, `SUPPORT_EMAIL` +- Folders: `BACKUP_DIR` (see [Backups](../backups.md)) + +Optional tuning: `RADIS_IMAGE` and `STACK_NAME` (a second stack such as staging on the same host), `BACKUP_CRON` and `BACKUP_ENABLED`, `TIME_ZONE`, the `EMBEDDINGS_*` group (see [Embeddings](#embeddings-hybrid-search)), the `LABELING_*` group (see [Auto-Labeling](#auto-labeling)), `ANALYSIS_STALLED_WORKER_GRACE_SECONDS` and `ANALYSIS_SWEEP_CRON` (see [Background Workers](#background-workers-and-crash-recovery)), `OTEL_EXPORTER_OTLP_ENDPOINT`, and `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY` behind a proxy. + +!!! warning "No quotes in .env" + Values must not be wrapped in quotes; Docker Swarm treats them as part of the value, and `stack-deploy` refuses to run when it finds any. + +### Email + +RADIS sends mail for finished jobs, account registrations awaiting approval, subscription notifications, and critical server errors: + +- `DJANGO_EMAIL_URL`: SMTP server as a URL, e.g. `smtp://user:password@mail.example.com:587?tls=True`. Only used in production; in development mails are printed to the `web` log instead +- `DJANGO_SERVER_EMAIL`: Sender address of all mail +- `DJANGO_ADMIN_EMAIL`, `DJANGO_ADMIN_FULL_NAME`: Receive critical error reports and the "account awaiting approval" mails +- `SUPPORT_EMAIL`: Shown to users as the address to contact for help + +Subjects are prefixed with `[RADIS]`. To check the settings, run the `send_test_mail` management command (optionally with a recipient address; it defaults to `DJANGO_ADMIN_EMAIL`). + +### Running Management Commands + +Some sections below ask you to run a management command (`embed_pending`, `labels_status`, `sweep_stale_tasks`, `send_test_mail`). They run inside the `web` container. In production the stack runs under Docker Swarm, so find the container first: + +```terminal +docker exec -it $(docker ps -q -f name=radis_prod_web | head -n1) ./manage.py labels_status ``` +`radis_prod` is the stack name (`STACK_NAME`, default `radis_prod`). In development `docker compose exec web ./manage.py labels_status` does the same. + ## Updating RADIS Follow these steps to safely update your RADIS installation: 1. **Verify no active jobs** -2. **Enable maintenance mode**: In Django Admin, navigate to **Common** → **Project Settings** and check the "Maintenance mode" checkbox, then save -3. Navigate to Production folder -4. **Backup database**: Run `uv run cli db-backup` to create a database backup +2. **Enable maintenance mode**: In Django Admin, navigate to **Common** → **Project settings**, check "Maintenance" and save +3. **Navigate to the production folder** (e.g. `radis_prod`, or whatever you named it) +4. **Backup database**: Run `uv run cli db-backup` to create a database backup (see [Backups](../backups.md)) 5. **Remove stack**: Run `uv run cli stack-rm` to remove all Docker containers and services 6. **Pull latest changes**: Run `git pull origin main` to fetch the latest code updates -7. **Update environment**: Compare `example.env` with your `.env` file and add any new environment variables or update changed values +7. **Update environment**: Compare `example.env` with your `.env` file and add any new environment variables or update changed values. Pay particular attention to the `LLM_*`, `EMBEDDINGS_*`, `LABELING_*`, and `ANALYSIS_*` groups, which are explained below. Keep `STACK_NAME` unchanged, otherwise a second stack is deployed next to the old one 8. **Pull Docker images**: Run `uv run cli compose-pull` to download the latest Docker images -9. **Deploy stack**: Run `uv run cli stack-deploy` to rebuild and start all services with the updated code -10. **Disable maintenance mode**: In Django Admin, navigate to **Common** → **Project Settings** and uncheck the "Maintenance mode" checkbox, then save +9. **Deploy stack**: Run `uv run cli stack-deploy` to start all services with the updated image +10. **Disable maintenance mode**: Uncheck "Maintenance" in **Project settings** and save + +Depending on what changed, one of these follow-up steps may be needed after the stack is up: + +- If you enabled or changed the embedding model, backfill the vectors of existing reports (see [Embeddings](#embeddings-hybrid-search)) +- If you added or changed labels, run a labeling backfill (see [Auto-Labeling](#auto-labeling)) + +Tasks that were interrupted by the restart are repaired automatically (see [Background Workers](#background-workers-and-crash-recovery)). + +## LLM and Embeddings Configuration + +All AI features (chats, extractions, subscriptions, labeling, query generation) send their requests to a single OpenAI-compatible endpoint. RADIS ships no inference server; you point it at one you operate (e.g. Ollama, vLLM, SGLang, an LLM gateway) or at a hosted API. The endpoint must be reachable from inside the containers on every node of the stack; `host.docker.internal` only works in development. + +### LLM Settings + +Set these in `.env`: + +- `LLM_BASE_URL` (required): Base URL of the endpoint, e.g. `http://llm.internal:11434/v1` +- `LLM_API_KEY`: API key for the endpoint. Optional; many self-hosted providers ignore it +- `LLM_DEFAULT_MODEL` (required): The model every feature uses unless overridden +- `LLM_CHATS_MODEL`, `LLM_QUERY_GENERATION_MODEL`, `LLM_EXTRACTIONS_MODEL`, `LLM_SUBSCRIPTIONS_MODEL`, `LLM_LABELING_MODEL`: Per-feature overrides. Leave blank to use the default model. Useful to spend a stronger model where it matters (e.g. labeling) and a fast one for chat + +Every model setting takes the form `model[?param=value&...]`. The parameters are merged into each request body, so both standard fields and provider extensions work: + +```text +LLM_DEFAULT_MODEL=qwen3:8b?temperature=0&chat_template_kwargs.enable_thinking=false +``` + +Values are read as JSON where possible (`temperature=0` sends a number, `reasoning_effort=none` sends the string `"none"`), and a dotted key becomes a nested object. Drop parameters the provider does not know if it rejects requests with a 400 error. + +Request timeout, rate-limit handling, and transient retries have sensible defaults (`LLM_REQUEST_TIMEOUT_SECONDS`, `LLM_RATE_LIMIT_*`, `LLM_TRANSIENT_RETRY_*`); override them only if needed. + +### Embeddings (Hybrid Search) + +By default RADIS runs full-text search only. Setting `EMBEDDINGS_MODEL` switches on hybrid search, where full-text and semantic (vector) results are fused into one ranking: + +- `EMBEDDINGS_MODEL`: The embedding model, in the same `model[?param=value]` form. Leave empty for full-text search only +- `EMBEDDINGS_BASE_URL`, `EMBEDDINGS_API_KEY`: Default to `LLM_BASE_URL` and `LLM_API_KEY`. Set them only when embeddings are served from a different endpoint, which is the normal case for vLLM and SGLang since they serve one model per process +- `EMBEDDINGS_DIM`: Vector width (default `1024`). This is coupled to the database schema: changing it after deployment requires dropping the embedding column, re-migrating, and re-embedding all reports. If the model spec also sets `dimensions`, both must agree; RADIS checks this at startup +- `EMBEDDINGS_QUERY_INSTRUCTION`: Model-specific instruction prefix for search queries (some models, e.g. Qwen3-Embedding, want one) +- `EMBEDDINGS_QUERY_CACHE_TIMEOUT_SECONDS`: How long a query's vector stays cached (default 900 s). Lower it temporarily after swapping the model, or the old vectors keep serving stale results for up to this long + +New and updated reports are embedded automatically by the `embeddings_worker` service. Reports that existed before the model was configured have no vector and are found by full-text search only until you backfill them with the `embed_pending` management command (see [Running Management Commands](#running-management-commands)). It is idempotent and resumable, refuses to start while another backfill is running or `EMBEDDINGS_MODEL` is empty, and takes `--limit` and `--subjob-size` options; `embed_cancel` stops a running backfill. + +The same is available in Django Admin under **Pgsearch** → **Report search indexes**: the page shows the state of the embedding pipeline, has a **Cancel queued backfill** button, and offers the actions "Enqueue embedding for selected rows (NULL only)" and "Clear embeddings (NULL them)" on selected reports. **Embedding backfill runs** lists past and current backfills. + +When the embedding service is unreachable or rate limiting, searches silently fall back to full-text results and a warning is logged in the `web` service. + +See the [Contributing guide](../dev-docs/contributing.md) for how to run Ollama with a chat and an embedding model, and the troubleshooting sections there for connection problems. ## User and Group Management @@ -66,54 +157,88 @@ RADIS uses a group-based permission system: Each user has an **active group** that determines which reports they can currently access: -- Only reports associated with the active group are visible in searches and collections +- Only reports associated with the active group are visible in searches and report views. Collections are per user and not filtered by group: a report bookmarked while one group was active stays visible in the collection after switching to another - This ensures proper data isolation between different departments or projects +- Users need an active group to create subscriptions and extraction jobs; the job or subscription is bound to that group ## Report Management ### Report Import and Management -Administrators can import reports programmatically via the RADIS API or using the RADIS Client library. See the **RADIS Client** section below for details. +Administrators can import reports programmatically via the RADIS API or using the RADIS Client library. The reports API only accepts tokens of staff users. See the **RADIS Client** section below for details. + +Every import or update of a report re-indexes it for search, queues it for embedding (if configured), and marks its labels for re-labeling on the next scan (see below). This also applies to updates that do not change the report text, so avoid re-pushing an unchanged corpus if you want to keep LLM load down. + +## Auto-Labeling + +RADIS can label reports automatically with an LLM. Labels are defined system-wide by administrators in Django Admin; there is no in-app editor. Users see the results as badges on the report detail page and can filter searches by label. + +### Concepts + +- A **Label group** has a name and a **gate question**: a Yes/No question the LLM answers once per report before any label of the group is evaluated (e.g. "Is this a CT of the chest?"). A No answer skips the whole group for that report, which is the main lever for keeping LLM cost down +- A **Label** belongs to one group and has a name (unique across all groups), a **description** that defines the finding for the LLM, and an **active** flag. Inactive labels are neither classified nor offered in the search filter +- For each report and active label the LLM assigns one of five results: Present, Likely, Possible, Absent, or Unmentioned. Only the first three surface as badges and in the search filter -### Extraction Jobs +### Managing Labels -Administrators can monitor and manage extraction jobs where AI analyzes reports to extract structured data. +1. **Access Django Admin**: Navigate to **Labels** → **Label groups** and create a group with its gate question +2. Navigate to **Labels** → **Labels** and add labels to the group. Write the description as the definition you would give a radiologist; it is sent to the LLM verbatim +3. Run a backfill (see below) so that existing reports are labeled + +Saving a label (any field, not only the description) makes its existing results **stale**; saving a group (e.g. its gate question) makes the group's gate answers stale, but not the results of its labels. Stale results keep surfacing (with a greyed-out badge) until a backfill regenerates them; the periodic scan only picks up changed reports, not changed labels. Deactivating a label stops new classifications, but results already assigned remain visible on the reports. + +### Backfill and Periodic Scan + +Labeling runs as background jobs on the `llm` queue: + +- **Periodic scan**: Every night (`LABELING_SCAN_CRON`, default `0 2 * * *`) RADIS labels all reports that were created or updated since the last scan. The first scan after installation only records a checkpoint; reports that existed before are not labeled until you run a backfill +- **Backfill**: In Django Admin, navigate to **Labels** → **Labeling jobs** and click **Run backfill now**. This labels every report with a missing or stale result for any active label. The job is safe to cancel and restart. Only one labeling job can be active at a time; the scan skips its tick while a backfill runs + +The **Labeling jobs** page shows all jobs with their trigger (Periodic scan or Manual backfill) and status; a job can be canceled from its detail page. **Label results**, **Gate answers**, and **Labeling tasks** are available read-only for inspection. A task ends with status Warning when some of its reports could not be labeled and Failure when none could, which usually points to an LLM outage. + +To check the corpus-wide state (checkpoint, per-label and per-gate counts, stale counts), run the `labels_status` management command (see [Running Management Commands](#running-management-commands)). The checkpoint alone is also visible in Django Admin under **Labels** → **Labeling scan checkpoints**. + +Tuning variables in `.env`: `LABELING_TASK_BATCH_SIZE`, `LABELING_LLM_CONCURRENCY_LIMIT`, `LABELING_GATE_BATCH_SIZE`, `LABELING_JOB_PRIORITY`, and the optional prompt overrides `LABELING_SYSTEM_PROMPT` and `LABELING_GATE_SYSTEM_PROMPT`. Use `LLM_LABELING_MODEL` to run labeling on a different model than the other features. + +## Extraction Jobs + +Users create extraction jobs through a wizard (see the [User Guide](user-guide.md#6-extractions)). Administrators verify and monitor them. + +### Verifying Jobs + +Jobs created by regular users start in the **Unverified** state and are not processed until a staff user opens the job page and clicks **Verify Job**. Jobs created by staff users are queued immediately. Staff users see all users' jobs by appending `?all=1` to the job list URL. ### Managing Extractions -1. **Access Django Admin**: Navigate to **Extractions** → **Extraction Jobs** -2. **Monitor Status**: View jobs by status (Preparing, Pending, In Progress, Success, Failure, Canceled) -3. **View Details**: Click on a job to see: - - Owner and group - - Query and filters - - Output field - - Results data +1. **Access Django Admin**: Navigate to **Extractions** → **Extraction jobs** +2. **View Details**: Click on a job to see its owner and group, query and filters, status and message, and its output fields +3. **Extraction tasks** show the individual batches of a job together with the extracted data per report -### Granting Urgent Priority Permission +Each job may cover at most 25,000 reports; the wizard refuses larger result sets. -Users with urgent priority permission can skip the queue: +### Urgent Priority -1. Navigate to **Authentication and Authorization** → **Groups** -2. Edit the desired group -3. Add permission: `extractions | extraction job | Can process urgently` -4. Save +A job whose **urgent** flag is set is queued ahead of regular jobs. The flag can only be set in Django Admin, and only takes effect on jobs that are not queued yet (e.g. still Unverified), because the priority is fixed when the job is queued. The permission `Can analyze urgently` that exists on subscription jobs is not checked anywhere. ## Subscription Management -Administrators can view and manage all user subscriptions through the Django Admin interface. +Subscriptions are refreshed at the top of every hour. Each refresh looks at the reports created or updated since the previous refresh, applies the subscription's filters, and — if the subscription has filter questions or extraction fields — sends each report to the LLM on the `llm` queue. Users need the permission `subscriptions | subscription | Can add subscription` and an active group to create subscriptions. ### Managing Subscriptions 1. **Access Django Admin**: Navigate to **Subscriptions** → **Subscriptions** -2. **View All Subscriptions**: Click on a subscription entry to view its full details. -3. **Delete a Subscription**: Subscriptions can be removed if necessary using the Delete action +2. **View All Subscriptions**: Click on a subscription entry to view its full details, including its owner, group, and "last refreshed" timestamp. Changing "last refreshed" changes which reports the next refresh treats as new +3. **Delete a Subscription**: Subscriptions can be removed if necessary using the Delete action. This also deletes the inbox items collected so far + +The subscription list only shows a user's own subscriptions. Staff users can still open any user's inbox and its CSV export by URL (the subscription ID is shown in Django Admin). + +A refresh only considers reports of the owner's **current** active group, not the group the subscription was created in. A subscription silently stops collecting reports while its owner has another group active. + +## Background Workers and Crash Recovery -### Urgent Priority Permission +Extraction, subscription, and labeling jobs are processed by the `default_worker` and `llm_worker` services; embeddings by `embeddings_worker`. Check their logs (`docker compose logs llm_worker`) when jobs do not progress. -1. Navigate to **Authentication and Authorization** → **Groups** -2. Edit the desired group -3. Add permission: `subscriptions | subscription | Can process urgently` -4. Save +When a worker container is killed mid-task (crash, out-of-memory, redeploy), the affected tasks are repaired automatically: `default_worker` and `llm_worker` sweep stale tasks on startup, and a periodic sweep (`ANALYSIS_SWEEP_CRON`, default every minute) requeues tasks whose worker has been silent for longer than `ANALYSIS_STALLED_WORKER_GRACE_SECONDS` (default 30; never set it lower). Embedding subjobs are not analysis tasks; `embeddings_worker` runs no sweep, and interrupted subjobs are retried by the task queue itself. On every deploy the `init` service additionally runs `retry_stalled_jobs`, which re-enqueues queue jobs left behind by dead workers. No manual step is needed after a deploy; to run the sweep by hand, run the `sweep_stale_tasks` management command (see [Running Management Commands](#running-management-commands)). ## System Announcements @@ -130,7 +255,7 @@ System administrators can inform users about important updates, maintenance sche ### Announcement Display - Announcements appear prominently on the main/home page -- All logged-in users will see the announcement when they access RADIS +- Everyone who opens the home page sees the announcement, including visitors who are not logged in #### Example Announcements @@ -151,17 +276,18 @@ jobs with multiple output fields. Check out the user guide for more details. ## RADIS Client -RADIS Client is a Python library for programmatic access to RADIS features without using the web interface. +RADIS Client is a Python library to create, retrieve, update and delete reports without using the web interface. The reports API is restricted to staff users, so the token must belong to a staff account. ### Creating API Tokens To create an API token for programmatic access: -1. **Navigate** to **Token Authentication** by going to **"Profile"** --> **"Manage API Token"** -2. **Description** & **Expiry Time** : Add a description (optional) and expiry time for the token. +1. **Navigate** to **Token Authentication** by going to **"Profile"** --> **"Manage API Tokens"** +2. **Description** & **Expiry Time** : Add a description (optional) and choose an expiry time of 1, 7, 30 or 90 days. "Never" is only offered to users with the permission `token_authentication | token | Can generate never expiring token`. 3. **Click** on **"Generate Token"**. 4. This token will only be visible once, so make sure to copy it now and store it in a safe place. As you will not be able to see it again, you will have to generate a new token if you lose it. ### Revoking Tokens -- **Admins** can revoke tokens by navigating to **Django Admin** --> **Token Authentication** +- **Users** delete their own tokens on the same **Manage API Tokens** page +- **Admins** can revoke any token by navigating to **Django Admin** --> **Token Authentication** diff --git a/docs/user-docs/user-guide.md b/docs/user-docs/user-guide.md index 5ef43e173..d51a1cd6d 100644 --- a/docs/user-docs/user-guide.md +++ b/docs/user-docs/user-guide.md @@ -8,19 +8,20 @@ RADIS acts as a centralized repository for radiology reports, providing powerful 1. **Reports are stored** with structured metadata (patient info, study details, modalities, etc.) 2. **You search** using natural language queries with optional filters -3. **RADIS processes** your query using advanced search algorithms (BM25, semantic, or hybrid search) +3. **RADIS processes** your query with full-text search, combined with semantic (meaning-based) search when your administrator has configured it 4. **You receive** ranked results matching your criteria 5. **You organize** reports into collections, add notes, and subscribe to searches +6. **AI assists you** by labeling reports, extracting structured data, and answering questions about a report ## Functionalities Overview -When you log into RADIS, you'll see the main/homepage with several sections: +After logging in, the menu at the top of every page leads to these sections: - **Search**: Search for reports using text queries and filters - **Collections**: Organize reports into custom collections -- **Subscriptions**: Set up notifications for new matching reports +- **Subscriptions**: Get notified about new reports that match your criteria, with AI-based filtering and data extraction - **Notes**: View and manage notes you've added to reports -- **Extractions**: AI-powered analysis and filtering +- **Extractions**: Extract structured data from many reports at once using AI - **Chats**: Interactive chat interface with AI ## Main Features @@ -31,35 +32,62 @@ The search feature is the core of RADIS, allowing you to find relevant reports q 1. Navigate to the "Search" section 2. Enter your search query in the search box -3. Apply filters as needed (language, modality, date range, patient demographics) +3. Apply filters as needed on the right-hand side 4. Review the search results #### Search Syntax - **Case-insensitive**: Search terms are not case-sensitive -- **AND queries**: All terms must match (implicit AND between terms) -- **OR operator**: Use capital OR between terms (e.g., "fracture OR lesion") -- **Phrases**: Use quotes for exact matches (e.g., "lung cancer") -- **Exclusion**: Use minus sign to exclude terms (e.g., "-metastasis") +- **AND queries**: All terms must match (implicit AND between terms, or write `AND` explicitly) +- **OR operator**: Use capital `OR` between terms (e.g., `fracture OR lesion`) +- **NOT operator**: Use capital `NOT` to exclude a term (e.g., `pneumonia NOT metastasis`) +- **Phrases**: Use quotes for exact matches (e.g., `"lung cancer"`) +- **Grouping**: Use parentheses to combine operators (e.g., `(fracture OR lesion) NOT metastasis`) + +Operators must be written in capital letters; `or`, `and`, and `not` in lowercase are treated as ordinary search terms. A minus sign in front of a term does not exclude it, wildcards such as `pneumo*` are not supported (the `*` is dropped), and `field:value` syntax is not supported — the whole `field:value` token is removed; use the filters instead. A query is required (filters alone do not search) and may be at most 200 characters long. + +If your query contains mistakes such as unbalanced quotes or parentheses, unsupported characters, or stray operators, RADIS repairs it and shows a "Fixed invalid query" notice above the results with the query it actually ran. + +#### Semantic Search + +When your administrator has configured an embedding model, every search combines classic keyword matching with semantic search, which also finds reports that describe the same thing in different words. This happens automatically; there is no switch to turn it on or off. Each result shows the scores that determined its ranking. #### Available Filters -- **Language**: Filter by report language +- **Language**: Filter by report language. The default "All" searches reports in every language - **Modalities**: Filter by imaging modality (CT, MRI, X-ray, etc.) -- **Study Date**: Filter by date range -- **Study Description**: Filter by study description -- **Patient Sex**: Filter by patient sex (M/F/U) -- **Patient Age**: Filter by age range -- **Patient ID**: Search for specific patient ID +- **Labels**: Only show reports that carry at least one of the selected labels (see [Labels](#2-labels)). This filter only appears when labels have been set up +- **Study Date**: Filter by date range (from/till) +- **Study Description**: Filter by study description (partial match) +- **Patient Sex**: All, Male, or Female +- **Age Range**: Filter by patient age using the range slider (in steps of 10 years) + +Use "Reset filters" to clear all filters at once. + +#### Search Results + +Each result shows the patient age and sex, modalities, study description and the scores that determined its rank, followed by a highlighted excerpt of the report. Click "Show full report" to read the entire text. Results are paginated; the page size can be changed below the list. Each result panel has buttons to open the report details, add the report to a collection, add a note, start a chat about the report, and open the study in your PACS viewer. + +The report details page shows all metadata of a report (document ID, patient, study, accession number, modalities, additional metadata), its labels, and the full text. The arrows next to the Patient ID open a list of the same patient's earlier reports, later reports, or all reports. + +### 2. Labels + +Administrators can define labels (e.g. "Pulmonary nodule" or "Fracture") that RADIS assigns to reports automatically using AI. Each label is classified as Present, Likely, or Possible when the report supports it; reports where the label is absent or not mentioned carry no badge. + +- Labels appear as badges on the report detail page, grouped by their label group. Hover over a badge to see whether the finding is Present, Likely, or Possible +- A greyed-out badge means the report or the label definition changed since the label was assigned. A changed report is re-labeled by the next nightly run; a changed label definition is only refreshed when an administrator runs a backfill +- Use the **Labels** filter in the search to find all reports carrying a label. Selecting several labels finds reports that carry any of them -### 2. Collections +Labels cannot be edited by users. If a label seems wrong or missing, contact your administrator. + +### 3. Collections Collections allow you to organize reports for easy access: 1. Go to the "Collections" section to view your collections -2. Click "Create Collection" to create a new collection -3. Add reports to collections by clicking the collection button on report panels -4. View collection contents by clicking on the collection name +2. Click "Add new collection" to create a new collection +3. Add reports to collections by clicking the collection button on report panels. The same dialog removes a report from a collection again and preselects the collection you used last +4. View collection contents by clicking on the collection name. A collection page has buttons to rename, export (as an Excel spreadsheet with the report metadata and text), or delete the collection Collections are useful for: @@ -67,51 +95,76 @@ Collections are useful for: - Creating teaching file collections - Organizing reports by project or study -### 3. Notes +### 4. Notes Add personal notes to reports for additional context: -1. Open a report detail view -2. Click the notes button to add or edit notes -3. View all your notes in the "Notes" section +1. Click the note button on any report panel (in search results, collections, subscription inboxes, or the report details) to add or edit your note +2. Save a note with empty text to delete it +3. View and search all your notes in the "Notes" section 4. Notes are private to your user account -### 4. Subscriptions +### 5. Subscriptions -Set up subscriptions to be notified when new reports match your criteria: +Set up subscriptions to be notified when new reports match your criteria. RADIS checks for new reports every hour. Optionally, an AI model screens each new report with your questions and extracts data from it: 1. Navigate to the "Subscriptions" section -2. Click "Add Subscription" to set up a new subscription -3. Enter the Subscription Name and add filter questions. +2. Click "Add Subscription" +3. Enter a **Name** for the subscription +4. Narrow down the reports with the filters on the right: Patient ID, Language, Modalities, Study Description, Patient Sex, and Age Range +5. Optionally add up to three **Filter Questions**. Each is a yes/no question about the report (e.g. "Does the report describe a new pulmonary nodule?") together with the answer that should be accepted (Yes or No). A report only enters your inbox when the AI's answer to every question matches +6. Optionally add up to ten **Extraction Fields** to have data extracted from every matching report. Each field has a Name, a Description telling the AI what to extract, and an Output Type (Text, Numeric, Boolean, or Selection with a fixed list of options). Use the `[ ]` toggle next to the type to collect multiple values per report +7. Check "Notify me via mail of new reports" if you want an email whenever a refresh finds new reports +8. Save the subscription + +Only reports that arrive (or are updated) after the subscription was created are considered; existing reports are not searched retroactively. + +#### Inbox + +The subscription list shows how many reports each subscription has collected, with a badge for reports you have not looked at yet. Click the count to open the inbox: + +- Each matched report is shown with its header, a preview of the text, and — if you defined extraction fields — the extracted values +- Sort by arrival or study date, and narrow the list with the filters on the side (Patient ID, Study Description, Study Date, Modalities) +- Click "Download Extractions as CSV" to export the extracted values as a spreadsheet. Only reports with extracted values are included, so the file is empty for a subscription without extraction fields +- Opening the inbox marks its reports as seen + +Use the "Edit" and "Delete" buttons on the subscription page to change or remove a subscription. Changes apply to future refreshes only; reports already in the inbox stay there. + +### 6. Extractions + +Extraction jobs let you pull structured data out of many reports at once. Creating a job takes three steps: + +**Step 1 – Define fields.** Add one to five fields that you want extracted from each report. For each field, specify: + +- **Name**: A short column name for the results +- **Description**: What the AI should extract (e.g. "Largest diameter of the primary tumor in mm") +- **Output Type**: Text, Numeric, Boolean, or Selection. For Selection, enter the allowed options (up to seven) +- The **`[ ]` toggle** next to the type switches the field to an array, so multiple values of that type are returned per report (e.g. a list of all affected vertebrae) + +**Step 2 – Search query.** Enter a job title. Unless your administrator has turned this off, RADIS generates a search query from your field descriptions, which you can edit or replace. Apply filters (Language, Modalities, Study Date, Study Description, Patient Sex, Age Range) to narrow down the reports. A live counter shows how many reports the job will process; a job may cover at most 25,000 reports. Click "Preview Search Results" to open the matching reports in the regular search view in a new tab. - - Write each question and select Accept when answer is Yes or No.\ - - Add extraction fields by specifying the following for each field: - - Name - - Output Type (Boolean, Text, or Numeric) - - Description +**Step 3 – Review & submit.** Check the summary, optionally tick "Notify me via Email when job is finished", and click "Create Extraction Job". -4. Choose to receive email notifications -5. New matching reports will be tracked and you'll receive notifications +Jobs of regular users start in the "Unverified" state and are queued once an administrator has verified them. On the job page you can follow the progress of the individual tasks and, once results come in: -### 5. Extraction +- Click **View Results** to browse the extracted values in a table with one column per field +- Click **Download CSV** to export the table +- Use the control panel to cancel a running job, resume a canceled one, retry failed tasks, or delete the job. Administrators additionally see buttons to verify a job and to restart it entirely +- Click a task to see its reports and the extracted values per report -- Create extraction jobs to analyze multiple reports. -- Add extraction fields by specifying the following for each field: - - Name - - Output Type (Boolean, Text, or Numeric) - - Description -- In the next step, search query will be automatically generated based on these fields, which you can then review and refine. +The "Extractions" menu item opens the wizard directly; click "Previous Jobs" in step 2 of the wizard (or "Job List" on a job page) to see all your jobs. Administrators can lock the Extractions section temporarily, e.g. during maintenance; you then see a notice instead of the wizard. -### 6. Chats (AI Assistant) +### 7. Chats (AI Assistant) -If enabled, the Chats feature provides an interactive AI assistant: +The Chats feature provides an interactive AI assistant: -- Ask questions about reports in natural language -- Get contextual answers based on report content -- Useful for exploring report data interactively +- Start a chat from a report panel to ask questions about that report in natural language +- Click "New chat" in the "Chats" section for a general conversation that is not tied to a report +- The "Chats" section lists your previous chats with automatically generated titles; you can delete a single chat or clear all of them +- A message may be at most 1,000 characters long ## RADIS Client -RADIS Client is a Python library that provides programmatic access to RADIS features without using the web interface. +RADIS Client is a Python library to create, retrieve, update and delete reports without using the web interface. It does not search reports; use the web interface for that. -To use the RADIS Client, you must have an API token provided by an administrator. For instructions on generating an API token, refer to the [Admin Guide](admin-guide.md). +The reports API is restricted to staff users, so you need an API token of a staff account. For instructions on generating an API token, refer to the [Admin Guide](admin-guide.md#creating-api-tokens). diff --git a/mkdocs.yml b/mkdocs.yml index 35cdb990f..17abad1a8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -7,6 +7,10 @@ repo_name: openradx/radis repo_url: https://github.com/openradx/radis edit_uri: edit/main/docs/ +# Design notes and implementation plans are not part of the published docs. +exclude_docs: | + superpowers/ + theme: name: material palette: