Skip to content

Repository files navigation

osrm-quickstart

One command from a region name to a verified, running routing endpoint.

$ osrm-quickstart run --region europe/germany/berlin --profile car

That downloads the Geofabrik extract, checks it against the published MD5, warns you if the graph build will not fit in memory, runs the OSRM container pipeline, starts osrm-routed, waits until it genuinely answers a route request, and prints the distance and duration of a sample route. Re-running it skips everything that already completed.


The problem

Standing up a local routing engine is a well-documented pipeline that is nevertheless annoying to get right, and the failures are unhelpful:

  • The URL. Geofabrik paths are guessable until they are not (europe/great-britain/england/greater-london, europe/germany/nordrhein-westfalen), and a typo gives you a 404 or, worse, an HTML error page saved as .osm.pbf that fails three stages later inside a container.
  • The download. Extracts run from 2 MB to 14 GB. A transfer that dies at 95% should resume, not restart, and it should be checksummed before an hour of CPU gets spent on it.
  • The algorithm. OSRM has two preprocessing pipelines and they are not interchangeable. Run osrm-partition + osrm-customize and then start osrm-routed --algorithm ch and the server refuses to boot. Run osrm-contract on a country-sized extract with 16 GB of RAM and the container is OOM-killed; Docker reports exit code 137 and nothing else.
  • Health. OSRM binds its port before it has finished loading the graph, so "the container is up" is not "the endpoint works". The first real request fails, and it is not obvious why.

osrm-quickstart handles all four, for both OSRM and Valhalla, and shows you exactly what it is doing.

How it works

Five stages, each a separate module with a hard seam between it and the outside world:

  region string
       |
       v
  [ regions ]   europe/germany/berlin -> Geofabrik .osm.pbf + .osm.pbf.md5 URLs
       |                                 (pure string logic, no network)
       v
  [ download ]  resumable HTTP range request -> berlin-latest.osm.pbf.part
       |          -> MD5 verify -> atomic rename      (through Transport seam)
       v
  [ preflight ] extract size x engine/algorithm multiplier vs. free RAM & disk
       |          -> "use MLD instead" before anything expensive starts
       v
  [ plan ]      pure construction of the container pipeline:
       |          OSRM/MLD: extract -> partition -> customize -> serve
       |          OSRM/CH:  extract -> contract -> serve
       |          Valhalla: config -> tiles -> serve
       v
  [ runner ]    each command executed via the Docker CLI, logs streamed
       |          stage completion recorded in data_dir/state.json
       v
  [ verify ]    poll a real /route request until it succeeds, print km + min

Two design decisions carry most of the weight:

Docker is behind a Runner protocol. DockerRunner shells out; RecordingRunner records the commands and returns scripted exit codes. The entire plan-construction and orchestration path — stage ordering, mount paths, resume, --force, OOM handling, container failure — is covered by tests that never invoke Docker. --dry-run is a third consumer of the same rendering, which is why its output is guaranteed to be the command that would really run.

HTTP is behind a Transport protocol. Resume, checksum verification, retry and health-check timeout behaviour are all tested against a scripted fake. The test suite touches no network and takes under a second.

Install

This is not published to any package index. Clone and install from the checkout:

git clone https://github.com/geospatialrouting/osrm-quickstart.git
cd osrm-quickstart
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

That puts an osrm-quickstart console script on your PATH. You can equally run it as a module without activating anything:

.venv/bin/python -m osrm_quickstart run --region europe/monaco

Requirements: Python 3.11+, and Docker for anything other than --dry-run, regions and status.

Worked example

Pick a region:

$ osrm-quickstart regions --search berlin
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Region path           ┃ Name   ┃ Approx. PBF ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━┩
│ europe/germany/berlin │ Berlin │      75 MiB │
└───────────────────────┴────────┴─────────────┘
1 region(s). Sizes are approximate.

See exactly what would run, before running it:

$ osrm-quickstart run --region europe/germany/berlin --data-dir ./osrm-data --dry-run
docker run --rm -v /home/you/routing/osrm-data:/data ghcr.io/project-osrm/osrm-backend:latest osrm-extract -p /opt/car.lua /data/berlin-latest.osm.pbf
docker run --rm -v /home/you/routing/osrm-data:/data ghcr.io/project-osrm/osrm-backend:latest osrm-partition /data/berlin-latest.osrm
docker run --rm -v /home/you/routing/osrm-data:/data ghcr.io/project-osrm/osrm-backend:latest osrm-customize /data/berlin-latest.osrm
docker run -d --name osrm-quickstart -v /home/you/routing/osrm-data:/data -p 5000:5000 ghcr.io/project-osrm/osrm-backend:latest osrm-routed --algorithm mld --port 5000 /data/berlin-latest.osrm

Region     europe/germany/berlin
Engine     osrm
Data dir   /home/you/routing/osrm-data
Would run  extract, partition, customize, serve

Those lines are copy-pasteable, which is the point. Now run it for real:

$ osrm-quickstart run --region europe/germany/berlin --data-dir ./osrm-data
ok       Estimated peak memory 0.5 GiB and 0.7 GiB of disk; both fit comfortably.
run   extract: Extract the road network using the car profile
  [info] Parsing in progress..
  [info] input file /data/berlin-latest.osm.pbf
  [info] Using profile api version 4
  [info] RAM: peak bytes used: 512360448
run   partition: Partition the graph into multi-level cells (MLD)
run   customize: Customize cell weights (MLD)
run   serve: Serve the graph with osrm-routed (MLD)
note  endpoint listening on http://localhost:5000

Region        europe/germany/berlin
Engine        osrm
Data dir      /home/you/routing/osrm-data
Extract       74.8 MiB downloaded (md5 verified)
Ran           extract, partition, customize, serve
Endpoint      http://localhost:5000
Sample route  3.41 km in 6.2 min (osrm)

Run it again and the finished stages are skipped:

$ osrm-quickstart run --region europe/germany/berlin --data-dir ./osrm-data
skip  extract: already complete (use --force to rebuild)
skip  partition: already complete (use --force to rebuild)
skip  customize: already complete (use --force to rebuild)
run   serve: Serve the graph with osrm-routed (MLD)
note  endpoint listening on http://localhost:5000

Region        europe/germany/berlin
Extract       74.8 MiB reused (md5 verified)
Skipped       extract, partition, customize
Endpoint      http://localhost:5000
Sample route  3.41 km in 6.2 min (osrm)

Query it like any OSRM server:

$ curl -s 'http://localhost:5000/route/v1/car/13.388,52.517;13.397,52.529?overview=false' | jq '.routes[0] | {distance, duration}'
{
  "distance": 3413.8,
  "duration": 371.2
}

The preflight, which is the point

Ask for contraction hierarchies on an extract your machine cannot handle and the tool stops before the download rather than an hour into the build:

$ osrm-quickstart run --region europe/germany --algorithm ch
critical CH preprocessing needs roughly 56.1 GiB but this machine has 16.0 GiB. osrm-contract will very likely be OOM-killed (exit code 137).
         Use --algorithm mld instead (~28.0 GiB), or build on a larger machine.
error CH preprocessing needs roughly 56.1 GiB but this machine has 16.0 GiB. osrm-contract will very likely be OOM-killed (exit code 137).
Hint: Use --algorithm mld instead (~28.0 GiB), or build on a larger machine. Pass --skip-preflight to build anyway.

If a container does get OOM-killed anyway, the exit code is translated rather than passed through raw:

error Stage 'contract' was killed (exit 137) -- almost certainly out of memory.
Hint: Use --algorithm mld, choose a smaller region, or raise Docker's memory limit.

Valhalla

Same pipeline shape, different engine:

$ osrm-quickstart run --region europe/monaco --engine valhalla --profile auto --port 8002 --dry-run
docker run --rm -v /home/you/routing/osrm-data:/data ghcr.io/valhalla/valhalla:latest sh -c 'valhalla_build_config --mjolnir-tile-dir /data/valhalla_tiles --mjolnir-tile-extract /data/valhalla_tiles.tar --mjolnir-timezone /data/valhalla_tiles/timezones.sqlite --mjolnir-admin /data/valhalla_tiles/admins.sqlite > /data/valhalla.json'
docker run --rm -v /home/you/routing/osrm-data:/data ghcr.io/valhalla/valhalla:latest valhalla_build_tiles -c /data/valhalla.json /data/monaco-latest.osm.pbf
docker run -d --name osrm-quickstart -v /home/you/routing/osrm-data:/data -p 8002:8002 ghcr.io/valhalla/valhalla:latest valhalla_service /data/valhalla.json 1

Commands

Command What it does
run The whole pipeline: download, preflight, build, serve, verify.
download Fetch and MD5-verify the extract only. Resumes a partial transfer.
build Run the graph preprocessing stages against an already-downloaded extract.
serve Start the endpoint from an already-built graph.
verify Issue a sample route against an already-running endpoint.
regions Search or list the bundled Geofabrik region catalog.
status Show which stages are recorded complete in the data directory.
clean Delete build artefacts, optionally keeping the extract.

run, build, serve and download are the same code path with different stage toggles, so they cannot drift apart about paths, ports or images.

Options

Shared by every pipeline command

Flag Config key Default Meaning
--region, -r region europe/germany/berlin Geofabrik region path. A bare unambiguous name such as berlin also resolves.
--engine engine osrm osrm or valhalla.
--algorithm, -a algorithm mld OSRM preprocessing: mld or ch. Ignored by Valhalla.
--profile, -p profile car OSRM: car, bicycle, foot, or a .lua path. Valhalla: a costing name.
--data-dir, -d data_dir ./osrm-data Where the extract, graph and state.json live. Bind-mounted at /data.
--port port 5000 Host port to publish. Use 8002 for Valhalla.
--image image per engine Override the container image. Pin this for reproducibility.
--container-name container_name osrm-quickstart Name of the served container.
--memory-limit memory_limit unset Docker --memory value applied to every stage, e.g. 12g.
--allow-unknown-region allow_unknown_region false Accept a well-formed region path that is not in the bundled catalog.
--force force false Ignore resume state and rebuild from scratch; also re-downloads.
--dry-run dry_run false Print the docker commands instead of running them. Touches nothing.
--config nearest found Path to an osrm-quickstart.toml.

Verification options (run, serve, verify)

Flag Config key Default Meaning
--verify / --no-verify verify true Issue a sample route once the endpoint is up.
--from region centre Sample route origin as lon,lat.
--to region centre Sample route destination as lon,lat.
--health-timeout health_timeout 120.0 Seconds to wait for a healthy endpoint.
--health-interval health_interval 2.0 Seconds between health polls.

Command-specific options

Flag Applies to Meaning
--skip-preflight run, build Build even when the memory estimate says it will fail.
--search regions Filter the catalog by substring.
--limit regions Maximum rows to show (default 30).
--all regions List the entire bundled catalog.
--keep-extract clean Remove graph files and state but keep the .osm.pbf.
--yes, -y clean Skip the confirmation prompt.

Config keys with no matching flag: request_timeout (default 60.0), resume_download (default true), verify_checksum (default true).

Configuration file

Precedence is CLI flags > config file > defaults, and it is strict: an option you did not type on the command line can never override the file, even though every option has a default value.

The nearest osrm-quickstart.toml found walking upwards from the working directory is loaded automatically. A [quickstart] table is also accepted so the settings can live inside a file shared with other tooling. Unknown keys are an error rather than a silent no-op, because a typo in a config file is otherwise invisible. See examples/osrm-quickstart.toml for a documented template.

region = "europe/germany/nordrhein-westfalen"
algorithm = "mld"
profile = "bicycle"
data_dir = "./graphs"
port = 5001
memory_limit = "12g"

Resume and state.json

state.json in the data directory records which stages finished, keyed by a fingerprint of everything that affects the artefacts: region, engine, algorithm, profile and the extract's MD5. Change any of them and the records are discarded, which prevents the genuinely nasty failure of serving a bicycle graph you believe is a car graph, or a stale graph built from last month's extract.

serve is never resumable — a previous run's container is gone — so it always executes.

$ osrm-quickstart status --data-dir ./osrm-data
     europe/germany/berlin (osrm mld)
┏━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Stage     ┃ Duration ┃
┡━━━━━━━━━━━╇━━━━━━━━━━┩
│ customize │      11s │
│ extract   │      48s │
│ partition │      19s │
└───────────┴──────────┘

Custom profiles

Drop a .lua file into the data directory and pass its name. It is addressed through the same /data mount as the extract:

cp traffic-car.lua ./osrm-data/
osrm-quickstart run --region europe/germany/berlin --profile traffic-car.lua

Using it as a library

Everything the CLI does is available programmatically, with the same seams:

from osrm_quickstart import Settings, build_plan
from osrm_quickstart.regions import resolve

settings = Settings(region="europe/germany/berlin", profile="car").validate()
plan = build_plan(settings, resolve(settings.region))

print(plan.stages)          # ('extract', 'partition', 'customize', 'serve')
print(plan.shell_script())  # annotated, copy-pasteable bash

See examples/library_usage.py for plan inspection, a CH-vs-MLD preflight comparison, and a pipeline run that touches no Docker.

Limitations

  • Docker only. The runner shells out to the docker CLI. Podman works if you alias it, but it is not tested, and there is no rootless-specific handling.
  • The catalog is a curated subset, around 100 regions, not a mirror of Geofabrik's full index. Anything else works with --allow-unknown-region; you lose only the upfront size estimate and the automatic sample coordinates.
  • Size figures are approximate and drift as OSM grows. They drive order-of-magnitude preflight guidance only, and are replaced by the real file size as soon as the extract is on disk.
  • The memory multipliers are rules of thumb, tuned to be conservative. A dense urban extract costs more per compressed megabyte than a sparse rural one. Treat a warning as a prompt to think, not as a measurement.
  • No traffic updates, no clustering, no TLS. This builds and serves one local graph. Production deployment — reverse proxying, osrm-datastore shared memory, rolling graph updates — is deliberately out of scope.
  • Valhalla's config stage uses valhalla_build_config defaults. Multi-modal and transit setups want a hand-edited valhalla.json between the config and tiles stages; run those stages separately to do that.
  • --dry-run does not resolve the real extract size, so its preflight uses the catalog estimate rather than a measured one.

Development

pip install -e ".[dev]"
ruff check .
ruff format --check .
pytest -q

249 tests, no Docker, no network, under a second to run. That is deliberate — see CONTRIBUTING.md.

Further reading

Background on the pipeline this tool automates, and on the decisions it makes for you:

Related tools

License

MIT — see LICENSE.


Maintained by geospatialrouting.com.

About

One-command local routing stack — downloads an OSM extract, builds an OSRM or Valhalla graph with your chosen profile, and serves a verified routing endpoint.

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages