Skip to content

feat(api): REST endpoints — latest, history, options/strip - #57

Merged
obchain merged 2 commits into
mainfrom
feat/23-rest-endpoints
May 26, 2026
Merged

obchain merged 2 commits into
mainfrom
feat/23-rest-endpoints

Conversation

@obchain

@obchain obchain commented May 26, 2026

Copy link
Copy Markdown
Owner

Summary

Three new endpoints per PRD §6 / issue #23, reading the hot cache + 1m OHLC rollup the engine populates every 60s.

Method Path Source
GET /v1/index/{bvol|evol}/latest Redis index:{id}:latest (hot cache)
GET /v1/index/{bvol|evol}/history?interval=…&limit=… ClickHouse volx.index_1m rollup
GET /v1/options/strip?index={bvol|evol} Redis index:{id}:last_strip (new)

Engine prerequisite (small)

The strip transparency endpoint reads a new Redis key the engine now writes on each publish.

  • engine/src/snapshot.rs::run_snapshot returns a SnapshotResult { value, near, next } — the two Strip instances travel out so the scheduler can persist them.
  • engine/src/sinks.rs::IndexSinks::publish(iv, near, next) widened. Writes SET index:{id}:last_strip <json> alongside the existing latest SET + stream PUBLISH. Best-effort; failure emits volx_engine_redis_errors_total{op=set_strip} and continues — matches the existing posture.

Strip envelope shape:

{
  "index_id": "BVOL",
  "ts": "2026-05-26T07:01:40.032Z",
  "near": {
    "forward": 76923.08,
    "k_zero":  76906.25,
    "time_to_expiry_y": 0.04,
    "quotes": [[65000.0, 119.25, 0.4523], [65031.25, 119.97, 0.4517], …]
  },
  "next": { … }
}

Tuple [K, Q(K), iv] saves ~30 % wire size over object-per-quote at the 801 dense-grid points.

/latest envelope (PRD §6 pinned)

{
  "index": "BVOL",
  "value": 36.94107234943254,
  "ts": "2026-05-26T07:01:40.032781Z",
  "confidence": 1,
  "source_strip_hash": "0x2812dc48ddafed8ab4b69c52be46af0e7870bb9705297836fd572571325214c0",
  "next_update_eta_seconds": 54
}
  • source_strip_hash prefixes 0x only when the stored value is a valid 64-char hex (engine wire format); otherwise pass-through so the consumer sees what storage actually contains.
  • next_update_eta_seconds = 60 − (now − ts), clamped at 0 when the engine has paused.

/history query knobs

Param Allowed Default
interval 1m, 5m, 1h, 1d (PRD-pinned) 1m
limit [1, 10000] 1000

Query path:

SELECT toStartOfInterval(bucket, INTERVAL ? MINUTE) AS ts,
       argMin(open, bucket) AS open,
       max(high) AS high, min(low) AS low,
       argMax(close, bucket) AS close,
       sum(count_v) AS cnt,
       avg(avg_conf_v) AS avg_conf
FROM (
    SELECT bucket,
           argMinMerge(open_state)  AS open,
           argMaxMerge(close_state) AS close,
           max(high) AS high, min(low) AS low,
           countMerge(count_state)  AS count_v,
           avgMerge(avg_conf_state) AS avg_conf_v
    FROM volx.index_1m
    WHERE index_id = ?
    GROUP BY index_id, bucket
)
GROUP BY ts ORDER BY ts DESC LIMIT ?

Result is reversed in-Go to ascending time so lightweight-charts.setData() accepts it directly. Defence-in-depth DB-name allowlist runs again in this path (^[A-Za-z_][A-Za-z0-9_]*$).

Error semantics

Condition HTTP
Unknown ticker (/v1/index/xxx/... or ?index=xxx) 400
Invalid interval / limit 400
Key missing in Redis (latest / last_strip before first engine tick) 404
Redis or ClickHouse unreachable 503
Stored value not parseable JSON 500

redis.Nil no longer escapes the storage boundary — wrapped in ErrNotFound.

Live smoke (Deribit → ingestion → engine → API)

$ curl -sS http://127.0.0.1:8080/v1/health
{"status":"ok","last_update_age_s":6.16,"version":"0.1.0"}

$ curl -sS http://127.0.0.1:8080/v1/index/bvol/latest
{"index":"BVOL","value":36.94107234943254,"ts":"2026-05-26T07:01:40.032781Z",
 "confidence":1,"source_strip_hash":"0x2812dc…","next_update_eta_seconds":54}

$ curl -sS http://127.0.0.1:8080/v1/index/evol/latest
{"index":"EVOL","value":49.59437947420183, …}

$ curl -sS "http://127.0.0.1:8080/v1/index/bvol/history?interval=1m&limit=5"
{"index":"BVOL","interval":"1m","bars":[
  {"ts":"2026-05-25T12:00:00Z","open":65.1,"high":65.42,"low":65.1,"close":65.3,"count":3,"avg_confidence":0.97},
  {"ts":"2026-05-25T18:08:00Z","open":36.57,…,"count":1,"avg_confidence":1},
  … ]}

$ curl -sS "http://127.0.0.1:8080/v1/options/strip?index=bvol" | head -c 200
{"index_id":"BVOL","near":{"forward":76923.08,"k_zero":76906.25,
 "quotes":[[65000.0,119.25,0.4523],[65031.25,119.97,0.4517], …]}, …}

$ curl -sS -i http://127.0.0.1:8080/v1/index/xxx/latest | head -1
HTTP/1.1 400 Bad Request

All four happy paths + bad-ticker confirmed end-to-end against the local docker stack.

Test plan

  • Engine: cargo test -p volx-engine (66 pass)
  • Workspace: cargo test --workspace (144 pass)
  • Engine: cargo clippy --workspace --all-targets clean (pedantic)
  • API: go build ./... && go vet ./... clean
  • Live smoke (above)

Fixes #23

Implements three new endpoints per PRD §6 / issue #23. Reads the
hot cache + 1m OHLC rollup the engine populates every 60s.

  GET /v1/index/{bvol|evol}/latest                 ← Redis hot cache
  GET /v1/index/{bvol|evol}/history?interval&limit ← ClickHouse index_1m
  GET /v1/options/strip?index={bvol|evol}          ← Redis last_strip

Engine wiring (prerequisite — small change):

- `engine/src/snapshot.rs::run_snapshot` now returns a `SnapshotResult`
  carrying the published `IndexValue` + the near / next `Strip`
  instances that produced it. Strips travel out to the scheduler so
  the new `index:{id}:last_strip` Redis key gets a JSON envelope on
  every publish.
- `engine/src/sinks.rs::IndexSinks::publish` signature widened to
  `(iv, near, next)`. Writes the strip envelope alongside the
  existing `latest` SET + `stream` PUBLISH. Best-effort same as the
  other Redis writes; failure emits `volx_engine_redis_errors_total
  {op=set_strip|encode_strip}` and continues.
- Strip envelope shape:
    { index_id, ts,
      near: { forward, k_zero, time_to_expiry_y, quotes: [[K,Q,iv], …] },
      next: { … } }
  Tuple-form `[K,Q,iv]` saves ~30 % wire size over object-per-quote
  at 801 dense-grid points.

API layer:

- `internal/storage/redis.go` — `GetLatest` + `GetLastStrip` helpers;
  `ErrNotFound` sentinel for 404 mapping. `redis.Nil` is no longer
  exposed past the storage boundary.
- `internal/storage/clickhouse.go` — `IndexHistory(tickerID, hi,
  limit)` queries `index_1m` via `argMin/argMaxMerge` to materialise
  per-bucket OHLC scalars, then re-buckets to the requested interval
  with `toStartOfInterval(bucket, INTERVAL N MINUTE)`. Returned in
  ascending time order (lightweight-charts requirement).
- `internal/handlers/index.go` — handlers + path-param ticker
  allowlist (`bvol → BVOL`, `evol → EVOL`). PRD-shape `latest`
  envelope:
    { index, value, ts, confidence,
      source_strip_hash, next_update_eta_seconds }
  `source_strip_hash` is the engine's hex form prefixed with `0x`
  per PRD §6. `next_update_eta_seconds` clamps at 0 when the engine
  has paused (`now − ts > 60s`).

History query knobs:
- interval ∈ {1m, 5m, 1h, 1d}, validated by allowlist (`ParseHistoryInterval`)
- limit ∈ [1, 10_000], default 1000 (matches the chart's default page)
- DB-name allowlist (`^[A-Za-z_][A-Za-z0-9_]*$`) runs again in the
  history path — defence-in-depth against a future refactor that
  mutates `c.DB` after `OpenClickHouse`.

Live smoke (Deribit → ingestion → engine → API), all four paths:

  /v1/health                            → status=ok, age 6.16 s
  /v1/index/bvol/latest                 → BVOL=36.94, 0x-prefixed hash, eta=54 s
  /v1/index/evol/latest                 → EVOL=49.59
  /v1/index/bvol/history?interval=1m   → 4 bars chronologically ordered,
                                          oldest from prior smoke seed
  /v1/options/strip?index=bvol         → forward=76923.08, k_zero=76906.25,
                                          801 [K,Q,iv] triples per leg
  /v1/index/xxx/latest                  → 400 Bad Request

All 66 engine tests + 144 workspace tests pass; clippy pedantic
clean; Go build + vet clean.
@obchain obchain added area:api Go fiber service area:engine crates/engine math lang:go Go lang:rust Rust priority:p0 Blocker, must ship this phase type:feature New functionality labels May 26, 2026
MED-1: history response now includes an explicit `order:
"oldest_first"` field so the pagination contract is unambiguous —
`limit=N` returns the **most recent N bars** in ascending time order.
Removes a guess-required contract that would surface as a frontend
bug when #27's `lightweight-charts.setData()` was wired against it.

MED-2: documented why `INTERVAL N MINUTE` with N=1440 produces the
correct UTC-midnight alignment for `interval=1d`. The alignment
falls out of `toStartOfInterval`'s epoch-anchored multiple-of-N
semantics combined with `1440 min × k` being an integer number of
UTC days. Non-day multiples like `90m` would NOT respect calendar
boundaries — the comment flags the constraint before a future
interval extension hits the trap.

LOW-1: `next_update_eta_seconds` now clamps at both bounds. The
downward clamp at 0 was already in place; the upward clamp at
`enginePublishIntervalSecs` (60) closes a clock-skew / future-ts
edge that would surface a > 60 s countdown for an index that
publishes every 60 s. The frontend countdown widget keys on this
field.

LOW-2 (strip envelope deserialization test): deliberately deferred
to #28 CI per reviewer's recommendation — the test belongs with
the cross-runtime contract suite, not in this PR.

Re-verified live: `/v1/index/bvol/history?interval=1m&limit=2` now
returns `{"index":"BVOL","interval":"1m","order":"oldest_first",
"bars":[…]}`.
@obchain
obchain merged commit 7bf2c20 into main May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:api Go fiber service area:engine crates/engine math lang:go Go lang:rust Rust priority:p0 Blocker, must ship this phase type:feature New functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Go API: REST endpoints (latest, history, options/strip)

1 participant