From d50560da0abb550366bb29667d6cd00d90f89cf5 Mon Sep 17 00:00:00 2001 From: Jilles van Gurp Date: Thu, 2 Apr 2026 18:46:26 +0200 Subject: [PATCH] Sync Open Location Hub docs into website --- content/open-location-hub/docs/_index.md | 37 +++- .../open-location-hub/docs/architecture.md | 25 ++- content/open-location-hub/docs/auth.md | 2 + .../open-location-hub/docs/configuration.md | 47 ++++- .../open-location-hub/docs/connectors-mqtt.md | 147 ++++++++++++++++ .../docs/connectors-websocket.md | 163 ++++++++++++++++++ content/open-location-hub/docs/connectors.md | 140 +++++++++++++++ .../open-location-hub/docs/getting-started.md | 54 ++++++ content/open-location-hub/docs/rpc.md | 2 + 9 files changed, 608 insertions(+), 9 deletions(-) create mode 100644 content/open-location-hub/docs/connectors-mqtt.md create mode 100644 content/open-location-hub/docs/connectors-websocket.md create mode 100644 content/open-location-hub/docs/connectors.md create mode 100644 content/open-location-hub/docs/getting-started.md diff --git a/content/open-location-hub/docs/_index.md b/content/open-location-hub/docs/_index.md index e6288f2..9fce9ef 100644 --- a/content/open-location-hub/docs/_index.md +++ b/content/open-location-hub/docs/_index.md @@ -1,9 +1,42 @@ --- title: "Software Documentation" -description: "Reference documentation for hub architecture, configuration, authentication, and RPC behavior." +description: "Reference documentation for local setup, hub architecture, configuration, authentication, RPC behavior, and connector development." draft: false generated: true generated_from: "docs/index.md" github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/index.md" --- -Reference documentation for hub architecture, configuration, authentication, and RPC behavior. +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + +Reference documentation for local setup, hub architecture, configuration, +authentication, RPC behavior, and connector development. + +Start here if you want the laptop-friendly local runtime: + +- [`docs/getting-started.md`](/open-location-hub/docs/getting-started/) + +Core hub docs: + +- [`docs/architecture.md`](/open-location-hub/docs/architecture/) +- [`docs/configuration.md`](/open-location-hub/docs/configuration/) +- [`docs/auth.md`](/open-location-hub/docs/auth/) +- [`docs/rpc.md`](/open-location-hub/docs/rpc/) + +Connector docs: + +- [`docs/connectors.md`](/open-location-hub/docs/connectors/) +- [`docs/connectors-websocket.md`](/open-location-hub/docs/connectors-websocket/) +- [`docs/connectors-mqtt.md`](/open-location-hub/docs/connectors-mqtt/) + +Connector demonstrators live outside the hub runtime under +[`connectors/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors). +Shared connector-agnostic utility scripts live under +[`scripts/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/scripts). +The shared local runtime is documented in +[`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md). +Connector examples currently include +[`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md) +and +[`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md), +plus +[`connectors/replay/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/replay/README.md). diff --git a/content/open-location-hub/docs/architecture.md b/content/open-location-hub/docs/architecture.md index 4aa65a8..3f0aa3b 100644 --- a/content/open-location-hub/docs/architecture.md +++ b/content/open-location-hub/docs/architecture.md @@ -1,11 +1,13 @@ --- title: "Architecture" -description: "System structure, processing flows, and trust boundaries." +description: "Generated documentation page for Architecture." draft: false generated: true generated_from: "docs/architecture.md" github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/architecture.md" --- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + ## Layers - `cmd/hub`: process bootstrap and wiring - `internal/config`: environment-driven configuration @@ -33,17 +35,32 @@ github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/ ## Event Fan-Out 1. REST, MQTT, or WebSocket ingest enters the shared hub service. -2. The hub validates, normalizes, deduplicates, updates in-memory transient state, and derives follow-on events. -3. The hub emits normalized internal events for locations, proximities, trackable motions, fence events, optional collision events, and metadata changes. -4. MQTT and WebSocket consume that same event stream and publish transport-specific payloads. +2. The hub validates, normalizes, deduplicates, and updates in-memory transient state on the ingest path. +3. A buffered native-publication stage emits native location and motion events without blocking ingest on downstream fan-out. +4. A second buffered decision stage is the insertion point for future filtered or smoothed track processing and currently drives alternate-CRS publication, geofence evaluation, and optional collision evaluation. +5. MQTT and WebSocket consume the resulting internal event stream and publish transport-specific payloads in batches. +6. When any non-critical queue fills, the hub drops newer work on that path rather than backpressuring raw ingest. Implications: - ingest logic is shared across REST, MQTT, and WebSocket - MQTT is no longer the only downstream publication path - the internal event seam decouples downstream publication from MQTT-specific topics +- location ingest latency is protected from slower transport fan-out, geofence work, or collision work +- the decision-stage queue is the intended insertion point for future filtered or smoothed track processing before fence/collision decisions +- WebSocket fan-out coalesces multiple internal events into fewer wrapper messages and drops outbound payloads for slow subscribers instead of tearing the connection down immediately - hub-issued UUIDs for REST-managed resources, derived fence/collision events, and RPC caller IDs now use UUIDv7 so emitted identifiers are time-sortable - internal hub events carry the persisted `origin_hub_id` so downstream transports preserve source provenance +## Observability Boundaries +- `internal/observability` owns OpenTelemetry resource setup, OTLP exporters, lifecycle management, and the small internal instrumentation API used by the rest of the runtime. +- OTLP export is collector-first: metrics, traces, and logs go to an OpenTelemetry collector such as SigNoz rather than to a hub-owned scrape endpoint. +- Transport handlers attach ingest transport context at entry so REST, MQTT, and WebSocket traffic create one shared root ingest span shape before entering `internal/hub`. +- The hot path records low-cardinality metrics for accepted, deduplicated, rejected, and failed ingest, queue depth and wait time, stage latency, event-bus fan-out, MQTT publish, WebSocket dispatch, RPC execution, and end-to-end processing time. +- Child spans isolate proximity resolution, native publication, decision processing, fence evaluation, collision evaluation, metadata reconcile, auth, and runtime dependency work so slow stages can be inspected directly. +- Asset-, provider-, zone-, and fence-centric identifiers belong on traces and structured logs only. They are intentionally excluded from normal metric labels so dashboards remain queryable under sustained ingest volume. +- Zap remains the application logging API. When OTLP logs are enabled, the logger tees into the OpenTelemetry bridge so the same structured events still appear locally while also being exported to the collector. +- Runtime gauges for queue occupancy, event-bus subscribers, and WebSocket connections are exposed from `internal/hub` through observable instruments so the e2e stack can dashboard degraded states without adding lock-heavy bookkeeping to the ingest path. + ## RPC Control Plane 1. A client calls `GET /v2/rpc/available` or `PUT /v2/rpc` over HTTP. 2. REST auth verifies the bearer token and route-level access. diff --git a/content/open-location-hub/docs/auth.md b/content/open-location-hub/docs/auth.md index 2c4495e..bdeec1c 100644 --- a/content/open-location-hub/docs/auth.md +++ b/content/open-location-hub/docs/auth.md @@ -6,6 +6,8 @@ generated: true generated_from: "docs/auth.md" github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/auth.md" --- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + This project supports standards-based JWT bearer authentication for the REST API and an authorization model built around JWT claims plus a server-side permissions file. The same token verifier is also used for the OMLOX WebSocket surface, but WebSocket authentication happens per message through `params.token` instead of the HTTP `Authorization` header. diff --git a/content/open-location-hub/docs/configuration.md b/content/open-location-hub/docs/configuration.md index 083cf76..38e2083 100644 --- a/content/open-location-hub/docs/configuration.md +++ b/content/open-location-hub/docs/configuration.md @@ -6,6 +6,8 @@ generated: true generated_from: "docs/configuration.md" github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/configuration.md" --- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + All runtime configuration is environment-driven. Runtime lifecycle behavior: @@ -23,7 +25,12 @@ Runtime lifecycle behavior: - `POSTGRES_URL` (default `postgres://postgres:postgres@localhost:5432/openrtls?sslmode=disable`) - `MQTT_BROKER_URL` (default `tcp://localhost:1883`) - `WEBSOCKET_WRITE_TIMEOUT` (duration, default `5s`) -- `WEBSOCKET_OUTBOUND_BUFFER` (default `32`) +- `WEBSOCKET_READ_TIMEOUT` (duration, default `1m`) +- `WEBSOCKET_PING_INTERVAL` (duration, default `30s`) +- `WEBSOCKET_OUTBOUND_BUFFER` (default `256`) +- `EVENT_BUS_SUBSCRIBER_BUFFER` (default `1024`) +- `NATIVE_LOCATION_BUFFER` (default `2048`) +- `DERIVED_LOCATION_BUFFER` (default `1024`) Hub metadata bootstrap behavior: - the hub persists one durable metadata row in Postgres containing the stable `hub_id` and operator-facing label @@ -55,9 +62,14 @@ Stateful ingest behavior: - latest provider-source location state, trackable latest-location state, proximity hysteresis state, fence membership state, and collision pair state are all kept in process memory with the configured expiry semantics - metadata is loaded from Postgres at startup, updated immediately after successful CRUD writes, and reconciled in the background every `METADATA_RECONCILE_INTERVAL` - durable hub metadata is also loaded from Postgres at startup before the service begins accepting traffic -- WebSocket delivery uses a per-connection outbound queue capped by `WEBSOCKET_OUTBOUND_BUFFER`; slow subscribers are disconnected instead of backpressuring the ingest path +- WebSocket delivery uses a per-connection outbound queue capped by `WEBSOCKET_OUTBOUND_BUFFER`; when that queue fills, outbound payloads are dropped instead of backpressuring ingest or disconnecting the subscriber +- WebSocket liveness uses server ping frames every `WEBSOCKET_PING_INTERVAL` and considers the connection stale when no inbound message or pong arrives before `WEBSOCKET_READ_TIMEOUT` +- internal event-bus subscribers such as MQTT and WebSocket consume behind `EVENT_BUS_SUBSCRIBER_BUFFER` +- native location publication is queued behind `NATIVE_LOCATION_BUFFER` so ingest can decouple from transport fan-out +- post-native decision work such as future filtering, alternate-CRS publication, geofence evaluation, and collision evaluation is queued behind `DERIVED_LOCATION_BUFFER` +- when the native, decision, event-bus, or outbound socket queues are full, the hub drops newer work on those non-critical paths instead of slowing raw location ingest - the `metadata_changes` WebSocket topic emits lightweight `{id,type,operation,timestamp}` notifications for zone, fence, trackable, and location-provider CRUD or reconcile drift -- when `COLLISIONS_ENABLED=true`, the hub evaluates trackable-versus-trackable collisions from the latest WGS84 motion state and keeps short-lived collision pair state in memory for `COLLISION_STATE_TTL` +- when `COLLISIONS_ENABLED=true`, the hub evaluates trackable-versus-trackable collisions from the latest active WGS84 motion state and keeps short-lived collision pair state in memory for `COLLISION_STATE_TTL` - `COLLISION_COLLIDING_DEBOUNCE` limits repeated `colliding` emissions for already-active pairs RPC behavior: @@ -98,6 +110,35 @@ Proximity resolution behavior: See [docs/auth.md](/open-location-hub/docs/auth/) for the full auth model, Dex setup, and permission file format. +## Telemetry +- `OTEL_ENABLED` (`true`/`false`, default `false`) +- `OTEL_METRICS_ENABLED` (`true`/`false`, default `true` when telemetry is enabled) +- `OTEL_TRACES_ENABLED` (`true`/`false`, default `true` when telemetry is enabled) +- `OTEL_LOGS_ENABLED` (`true`/`false`, default `true` when telemetry is enabled) +- `OTEL_EXPORTER_OTLP_ENDPOINT` (base OTLP HTTP endpoint such as `http://localhost:4318`) +- `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` (optional full traces endpoint override) +- `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` (optional full metrics endpoint override) +- `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` (optional full logs endpoint override) +- `OTEL_EXPORTER_OTLP_HEADERS` (comma-separated `key=value` headers for collector auth or routing) +- `OTEL_EXPORTER_OTLP_INSECURE` (`true`/`false`, default `false`) +- `OTEL_EXPORTER_OTLP_TIMEOUT` (duration, default `10s`) +- `OTEL_METRIC_EXPORT_INTERVAL` (duration, default `30s`) +- `OTEL_METRIC_EXPORT_TIMEOUT` (duration, default `10s`) +- `OTEL_TRACE_SAMPLE_RATIO` (number between `0` and `1`, default `1`) +- `OTEL_SERVICE_NAME` (default `open-rtls-hub`) +- `OTEL_SERVICE_VERSION` (optional override for release tagging) +- `OTEL_DEPLOYMENT_ENVIRONMENT` (optional deployment environment label such as `local-demo` or `production`) +- `OTEL_DEBUG_IDENTIFIERS` (`true`/`false`, default `false`) + +Telemetry behavior: +- the hub exports OTLP metrics, traces, and logs directly to a collector and does not expose a separate Prometheus scrape endpoint in this slice +- `service.name`, `service.version`, `deployment.environment`, and the persisted `hub.id` are attached to the OTel resource when available +- invalid telemetry configuration fails startup only when telemetry is enabled +- metrics are intentionally low-cardinality and are labeled only with bounded dimensions such as transport, signal type, stage, feature, and outcome +- entity identifiers such as `trackable_id`, `provider_id`, `zone_id`, `fence_id`, and collision pair identifiers are emitted on spans and structured logs for drill-down, not on normal metric series +- runtime metrics cover ingest acceptance and deduplication, end-to-end processing latency, queue occupancy and wait time, fence evaluation, collision evaluation, MQTT/WebSocket publication, metadata reconcile, auth, and RPC outcomes +- the local demo stack under [`local-hub/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub) enables all three OTLP signals by default and routes them to SigNoz + ## RPC Security Defaults For production deployments: diff --git a/content/open-location-hub/docs/connectors-mqtt.md b/content/open-location-hub/docs/connectors-mqtt.md new file mode 100644 index 0000000..a892e66 --- /dev/null +++ b/content/open-location-hub/docs/connectors-mqtt.md @@ -0,0 +1,147 @@ +--- +title: "Connector Guide: MQTT" +description: "Use MQTT when your deployment already centers on a broker, when trusted edge adapters naturally publish broker messages, or when broker-based routing fits better than a long-lived WebSocket client connection." +draft: false +generated: true +generated_from: "docs/connectors-mqtt.md" +github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/connectors-mqtt.md" +--- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + +Use MQTT when your deployment already centers on a broker, when trusted edge +adapters naturally publish broker messages, or when broker-based routing fits +better than a long-lived WebSocket client connection. + +This repository ships a concrete MQTT connector example in +[`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md). +The MQTT connector path is documented here against the hub's implemented OMLOX +MQTT surface. Transport details live in +[`specifications/omlox/mqtt.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/mqtt.md). + +## When To Choose MQTT + +Choose MQTT when: + +- your source system already publishes to a broker +- you are running trusted local or edge adapters near devices +- you want broker-native fan-out, retention, or topic-based routing +- WebSocket is not the natural operational fit for the connector process + +For user-facing applications and general connector examples, WebSocket is often +the simpler starting point. See +[docs/connectors-websocket.md](/open-location-hub/docs/connectors-websocket/). + +## Required Inputs + +An MQTT-based connector usually needs: + +- `HUB_HTTP_URL`: REST base URL for optional metadata bootstrap +- `MQTT_BROKER_URL`: broker URL such as `tcp://localhost:1883` +- MQTT client identity and auth settings that fit your deployment +- `HUB_TOKEN` only if the connector also uses REST against an auth-enabled hub + +The hub's runtime broker URL is documented in +[`docs/configuration.md`](/open-location-hub/docs/configuration/). + +## Minimal Runtime Flow + +The connector shape is almost the same as the WebSocket path: + +1. Load environment variables. +2. Optionally use REST to ensure the provider and other metadata exist. +3. Connect to the MQTT broker. +4. Map upstream data to OMLOX payloads. +5. Publish each payload to the correct OMLOX ingest topic. +6. Optionally subscribe to hub-generated output topics for validation. +7. Reconnect and retry when broker connectivity fails. + +The main difference is transport mapping. Instead of sending an OMLOX wrapper, +the connector publishes the payload body directly on the correct topic. + +## Inbound Versus Outbound Topics + +MQTT connector authors need to keep one boundary clear: + +- inbound ingest topics are written by the connector and consumed by the hub +- outbound topics are written by the hub after it processes or derives data + +Examples from the existing MQTT mapping: + +- connector writes location ingest to + `/omlox/json/location_updates/pub/{provider_id}` +- connector writes proximity ingest to + `/omlox/json/proximity_updates/{source}/{provider_id}` +- hub writes processed locations to + `/omlox/json/location_updates/local/{provider_id}` and + `/omlox/json/location_updates/epsg4326/{provider_id}` +- hub writes derived events to topic families such as `fence_events`, + `trackable_motions`, and `collision_events` + +Do not publish directly to the hub-generated output topics. Those exist so the +hub can publish normalized or derived results after it processes the inbound +payload. + +## Topic Selection + +The correct topic depends on the payload type: + +- publish `Location` ingest to + `/omlox/json/location_updates/pub/{provider_id}` +- publish `Proximity` ingest to + `/omlox/json/proximity_updates/{source}/{provider_id}` +- consume processed and derived hub output from the output topic families when + you need validation or downstream processing + +Use +[`specifications/omlox/mqtt.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/mqtt.md) +as the source of truth for topic families and payload expectations. + +## Example Ingest Shape + +For MQTT, the message body is the normalized OMLOX object itself. For example, +a connector publishing a location update would send a `Location` JSON payload to +`/omlox/json/location_updates/pub/{provider_id}`: + +```json +{ + "position": { "type": "Point", "coordinates": [8.5705, 50.0333] }, + "crs": "EPSG:4326", + "provider_id": "my-connector", + "provider_type": "virtual", + "source": "upstream:object-123" +} +``` + +## Reusing The Bundled Connector Pattern + +Even though the transport changes, the overall connector design can stay the +same as the bundled examples: + +- keep env-driven configuration +- keep a small REST helper for metadata bootstrap +- keep the upstream polling or subscription loop separate from the transport + client +- keep mapping code focused on OMLOX payload construction + +These repository examples are the best templates for that structure: + +- [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md): + includes both a WebSocket connector and an MQTT connector that publish the + same normalized GTFS-derived `Location` payloads +- [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md) + +The GTFS project is now the direct MQTT example. The OpenSky project remains a +useful example for shared project layout, env handling, and REST bootstrap. + +## Local Run Path + +For local development, use the shared demo stack: + +```bash +local-hub/start_demo.sh +``` + +That stack includes the hub, Postgres, Dex, and Mosquitto. See +[`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md) +for the local runtime details and the token helper if your connector also uses +REST against an auth-enabled hub. diff --git a/content/open-location-hub/docs/connectors-websocket.md b/content/open-location-hub/docs/connectors-websocket.md new file mode 100644 index 0000000..a9776aa --- /dev/null +++ b/content/open-location-hub/docs/connectors-websocket.md @@ -0,0 +1,163 @@ +--- +title: "Connector Guide: WebSocket" +description: "Use WebSocket when you want the simplest connector path into the hub. It maps well to the current bundled demos, it works cleanly with the hub's auth model, and it does not require you to design topic routing beyond the OMLOX WebSocket topic names." +draft: false +generated: true +generated_from: "docs/connectors-websocket.md" +github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/connectors-websocket.md" +--- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + +Use WebSocket when you want the simplest connector path into the hub. It maps +well to the current bundled demos, it works cleanly with the hub's auth model, +and it does not require you to design topic routing beyond the OMLOX WebSocket +topic names. + +The bundled connector demonstrators in this repository currently use this +approach: + +- [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md) +- [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md) + +Protocol details live in +[`specifications/omlox/websocket.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/websocket.md). + +## When To Choose WebSocket + +Choose WebSocket when: + +- your connector is an application-style process that already manages a direct + connection to the hub +- you want the same ingest path as the bundled examples +- you want per-message auth through `params.token` +- you want to subscribe to hub topics for validation, logging, or downstream + processing in the same transport family + +## Required Inputs + +Most WebSocket-based connectors need: + +- `HUB_HTTP_URL`: REST base URL for metadata bootstrap such as + `http://localhost:8080` +- `HUB_WS_URL`: WebSocket endpoint such as `ws://localhost:8080/v2/ws/socket` +- `HUB_TOKEN`: optional JWT access token; required when auth is enabled + +The bundled examples also keep upstream URLs, polling intervals, provider IDs, +and source-specific settings in environment variables. + +## Minimal Runtime Flow + +The common flow is: + +1. Load environment variables. +2. Create a REST client for provider or trackable upserts when needed. +3. Open a WebSocket connection to `GET /v2/ws/socket`. +4. Map upstream data to OMLOX payloads. +5. Send wrapper `message` events to the right topic. +6. Reconnect and retry when the socket breaks. + +The bundled helper modules also keep the connection alive with periodic ping +frames and reconnect on send failure. + +## Publish Shape + +For ingest, the connector sends OMLOX wrapper messages. A typical +`location_updates` publish looks like this: + +```json +{ + "event": "message", + "topic": "location_updates", + "payload": [ + { + "position": { "type": "Point", "coordinates": [8.5705, 50.0333] }, + "crs": "EPSG:4326", + "provider_id": "my-connector", + "provider_type": "virtual", + "source": "upstream:object-123" + } + ], + "params": { + "token": "..." + } +} +``` + +Notes: + +- `payload` is an array, even when you only send one item +- omit `params.token` only when hub auth is disabled +- publish to the topic that matches the payload type and intended behavior +- `location_updates` is the normal entry point for live location ingest + +## Subscribe Shape + +The same transport can be used to observe hub output topics. A minimal +subscription message looks like this: + +```json +{ + "event": "subscribe", + "topic": "fence_events", + "params": { + "token": "..." + } +} +``` + +This is useful when you want to: + +- validate what the hub emits after ingest +- capture NDJSON logs during local testing +- confirm that your metadata bootstrap and location mapping behave as expected + +## Metadata Bootstrap + +Most connectors need more than a socket sender. The bundled examples also use +REST for idempotent metadata setup: + +- create or update the `LocationProvider` +- upsert `Trackable` resources when the upstream source has stable asset IDs +- optionally create `Zone` and `Fence` resources before ingest starts + +That split is usually the easiest connector design: + +- REST for durable metadata and bootstrap +- WebSocket for live ingest and optional subscriptions + +## Local Run Path + +For local development: + +1. Start the shared demo runtime: + +```bash +local-hub/start_demo.sh +``` + +2. Fetch a token when auth is enabled: + +```bash +local-hub/fetch_demo_token.sh +``` + +3. Run your connector against the local hub with `HUB_HTTP_URL`, + `HUB_WS_URL`, and optionally `HUB_TOKEN`. + +See +[`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md) +for the reusable local stack. + +## Example Projects + +Use these as concrete WebSocket examples: + +- [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md): + REST bootstrap for provider, trackables, stations, and fences, then live + `location_updates` publish over WebSocket +- [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md): + REST bootstrap for provider, trackables, and airport fences, then live + `location_updates` publish over WebSocket + +If you want to build your own connector quickly, copying one of those patterns +is the shortest path. diff --git a/content/open-location-hub/docs/connectors.md b/content/open-location-hub/docs/connectors.md new file mode 100644 index 0000000..e30da18 --- /dev/null +++ b/content/open-location-hub/docs/connectors.md @@ -0,0 +1,140 @@ +--- +title: "Connectors" +description: "This repository keeps connectors outside the hub runtime. A connector is a small process that reads data from some upstream system, maps that data to the hub's OMLOX-facing model, and submits it through the hub's supported transport surfaces." +draft: false +generated: true +generated_from: "docs/connectors.md" +github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/connectors.md" +--- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + +This repository keeps connectors outside the hub runtime. A connector is a +small process that reads data from some upstream system, maps that data to the +hub's OMLOX-facing model, and submits it through the hub's supported transport +surfaces. + +The key point is simple: creating a connector is easy. You do not need to patch +the hub. In most cases you only need: + +1. a small runtime loop that reads upstream data +2. some mapping code that builds OMLOX `Location` or `Proximity` payloads +3. optional REST bootstrap calls for providers, trackables, zones, or fences +4. a transport publisher for WebSocket or MQTT + +## Bundled Connector Examples + +The repository already includes a few connector-oriented projects under +[`connectors/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors): + +- [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md): + GTFS-RT vehicle ingest plus station-zone and fence bootstrap +- [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md): + OpenSky aircraft ingest plus airport fence bootstrap +- [`connectors/replay/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/replay/README.md): + NDJSON trace replay back into the hub for diagnostic or demo use + +The bundled runtime connectors now cover both transport styles. The GTFS +project includes WebSocket and MQTT ingest variants, and the OpenSky project +shows the WebSocket path. Together they are useful examples for connector +structure, env handling, bootstrap logic, and local development flow. + +The recommended local runtime itself now lives outside `connectors/` under +[`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md) +because it is broader than connector bootstrapping. It is the normal starting +point when you want a local hub plus observability stack on your laptop. + +## Easy Path + +Most custom connectors follow the same shape: + +1. Pick a transport. + Use WebSocket when you want the simplest general-purpose hub-facing ingest + path. Use MQTT when your deployment already centers on a broker or trusted + edge adapters. +2. Prepare env-driven config. + Keep hub URLs, broker settings, auth tokens, upstream URLs, and poll + intervals in environment variables. +3. Create or reconcile hub metadata when needed. + Many connectors need a `LocationProvider` and sometimes `Trackable`, `Zone`, + or `Fence` resources before live ingest starts. +4. Map upstream records to OMLOX payloads. + Normalize each upstream event to a hub payload such as `Location` or + `Proximity`. +5. Publish to the hub. + Send those normalized payloads over WebSocket or MQTT using the transport's + expected wrapper or topic layout. +6. Run locally against the shared demo stack. + Use [`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md) + for a reproducible local hub, Postgres, Dex, Mosquitto, and optional SigNoz environment. + +## Shared Connector Shape + +The bundled connectors show a practical split that is easy to copy: + +- one runtime script that polls or subscribes to the upstream source +- one small client helper module for hub REST and transport calls +- optional bootstrap scripts for zones, fences, or other metadata +- simple env-file loading so the connector can run locally or in a container + +The shared flow looks like this: + +1. Load env-driven configuration. +2. Connect to the hub transport surface. +3. Ensure the provider exists through REST. +4. Optionally upsert trackables or bootstrap fences and zones. +5. Read upstream data. +6. Map it to OMLOX payloads. +7. Publish the payloads. +8. Retry or reconnect when the upstream source or hub transport fails. + +## Running A Connector Locally + +The recommended local development path is the shared demo runtime: + +1. Start the local stack: + +```bash +local-hub/start_demo.sh +``` + +2. If auth is enabled, fetch a token: + +```bash +local-hub/fetch_demo_token.sh +``` + +3. Create a connector-local env file with the hub URLs, optional token, and any + upstream-specific settings. +4. Run the connector process. + +The bundled demos use `HUB_HTTP_URL` for REST bootstrap work and `HUB_WS_URL` +for WebSocket ingest. MQTT-based connectors would typically use `HUB_HTTP_URL` +plus `MQTT_BROKER_URL` or equivalent broker settings. + +## Choosing A Transport + +- [docs/connectors-websocket.md](/open-location-hub/docs/connectors-websocket/) + is the best starting point if you want the same transport flow as the bundled + runtime connectors. +- [docs/connectors-mqtt.md](/open-location-hub/docs/connectors-mqtt/) + explains how to build the same kind of connector on top of the hub's OMLOX + MQTT surface. + +## Where To Copy From + +If you want a fast starting point: + +- copy the project structure and env handling style from + [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md) + or + [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md) +- use + [`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md) + as the local runtime guide +- use + [`specifications/omlox/websocket.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/websocket.md) + or + [`specifications/omlox/mqtt.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/mqtt.md) + as the transport source of truth + +That is usually enough to get a first connector running quickly. diff --git a/content/open-location-hub/docs/getting-started.md b/content/open-location-hub/docs/getting-started.md new file mode 100644 index 0000000..f088625 --- /dev/null +++ b/content/open-location-hub/docs/getting-started.md @@ -0,0 +1,54 @@ +--- +title: "Getting Started" +description: "If you want to try Open RTLS Hub on your laptop, start with the shared local runtime in [`local-hub/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub). That setup brings up the hub, Postgres, Mosquitto, Dex, and a ready-to-use observability stack so you can inspect what the hub is doing while you experiment." +draft: false +generated: true +generated_from: "docs/getting-started.md" +github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/getting-started.md" +--- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + +If you want to try Open RTLS Hub on your laptop, start with the shared local +runtime in [`local-hub/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub). +That setup brings up the hub, Postgres, Mosquitto, Dex, and a ready-to-use +observability stack so you can inspect what the hub is doing while you +experiment. + +## Fast Path + +1. Review [`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md). +2. Start the stack: + +```bash +local-hub/start_demo.sh +``` + +3. If you need an auth token for manual calls or connector demos: + +```bash +local-hub/fetch_demo_token.sh +``` + +4. Pick an example connector or continue with the docs below. + +## What This Setup Is For + +- local development and experimentation +- validating connector behavior against a real hub runtime +- inspecting OTLP telemetry during laptop demos and debugging + +This starter stack is not positioned as a production deployment recipe. + +## Notes + +- Dex is included because it is convenient for local OIDC and repeatable demo users, not because it is the recommended production IdP choice. +- SigNoz is included because it is easy to bootstrap and script for modern local observability workflows, but the hub does not depend on SigNoz specifically. +- Alternative OpenTelemetry-compatible collectors and observability stacks should work as well. + +## Good Next Steps + +- [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md) for GTFS transit vehicle ingest over WebSocket or MQTT +- [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md) for OpenSky aircraft ingest over WebSocket +- [`connectors/replay/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/replay/README.md) for replaying captured `location_updates` traffic +- [`docs/index.md`](/open-location-hub/docs/) for the full software documentation set +- [`docs/connectors.md`](/open-location-hub/docs/connectors/) for connector structure and transport guidance diff --git a/content/open-location-hub/docs/rpc.md b/content/open-location-hub/docs/rpc.md index 335c4a4..d9e4af1 100644 --- a/content/open-location-hub/docs/rpc.md +++ b/content/open-location-hub/docs/rpc.md @@ -6,6 +6,8 @@ generated: true generated_from: "docs/rpc.md" github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/rpc.md" --- +_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._ + ## What RPC is RPC is the hub's command and diagnostics interface.