Skip to content

API.md

Latest commit

 

History

History
353 lines (262 loc) · 7.58 KB

API.md

File metadata and controls

353 lines (262 loc) · 7.58 KB
Raw

NewsLens API

NewsLens provides an artifact-backed recommendation API and a PostgreSQL-backed search API for articles delivered by the real-time ingestion path.

Current capabilities

The API provides:

  • service liveness reporting;
  • verified model-artifact loading during application startup;
  • model-version and ranking-configuration metadata;
  • candidate-based recommendation inference;
  • TF-IDF user-history recommendation;
  • popularity fallback for cold-start and zero-signal requests;
  • typed request and response validation;
  • automatic OpenAPI documentation; and
  • fail-fast startup for missing or corrupt configured artifacts.

The search route also provides deterministic category, entity, and freshness intent, bounded candidate retrieval, freshness-aware ranking, and score-component diagnostics. Recommendation readiness and search-store readiness are independent.

Artifact configuration

The API reads the model location from the NEWSLENS_ARTIFACT_PATH environment variable.

Example:

export NEWSLENS_ARTIFACT_PATH=artifacts/newslens-fallback-0.3.0

Generated artifacts are deliberately excluded from Git because they contain information derived from the licensed MIND dataset.

Only load artifacts generated by a trusted NewsLens training workflow. Joblib uses pickle-compatible deserialization. Checksums protect against accidental corruption but do not make an untrusted pickle safe.

The streamed-article database is configured separately:

export NEWSLENS_REALTIME_DATABASE_URL=postgresql://newslens:newslens@localhost:5432/newslens
export NEWSLENS_REALTIME_CANDIDATE_LIMIT=200

Running locally

Install NewsLens with development dependencies:

python -m pip install -e ".[dev]"

Start the API:

NEWSLENS_ARTIFACT_PATH=artifacts/newslens-fallback-0.3.0 \
python -m uvicorn newslens.api.app:app \
  --host 127.0.0.1 \
  --port 8000

Interactive documentation is available at:

http://127.0.0.1:8000/docs

GET /health

Reports service liveness independently of model readiness.

Example response:

{
  "status": "ok",
  "service": "newslens",
  "version": "0.3.0"
}

GET /model-info

Reports whether the model artifact was loaded successfully.

Example response:

{
  "model_name": "tfidf_content_with_popularity_fallback",
  "model_ready": true,
  "artifact_version": "0.3.0",
  "ranking_cutoff": 10
}

POST /recommend

Ranks a supplied candidate set using the user's article history.

Example request:

{
  "history_news_ids": ["N1", "N2"],
  "candidate_news_ids": ["N3", "N4", "N5"],
  "top_k": 3
}

Example response:

{
  "model_name": "tfidf_content_with_popularity_fallback",
  "artifact_version": "0.3.0",
  "requested_top_k": 3,
  "returned_count": 3,
  "recommendations": [
    {
      "news_id": "N3",
      "score": 0.421,
      "source": "content"
    }
  ]
}

The recommendation source is:

  • content when the user's history produces a usable TF-IDF profile; or
  • popularity when the user is cold-start or has no usable content signal.

GET /realtime/ready

Returns 200 only when the configured PostgreSQL article store answers a probe:

{
  "status": "ready",
  "realtime_store_ready": true
}

It returns 503 when the database is not configured or reachable. This does not depend on the recommendation model artifact.

GET /search

Searches the bounded streamed-article candidate set:

curl --get \
  --data-urlencode 'q=latest Apple AI chip' \
  --data-urlencode 'top_k=5' \
  http://127.0.0.1:8000/search

Example response:

{
  "request_id": "request-identifier",
  "intent": {
    "normalized_query": "latest Apple AI chip",
    "category": "technology",
    "entity": "Apple",
    "prefers_freshness": true
  },
  "candidate_count": 2,
  "returned_count": 2,
  "search_ms": 1.8,
  "results": [
    {
      "article_id": "article-123",
      "title": "Apple launches an AI chip today",
      "category": "technology",
      "published_at": "2026-08-23T12:00:00+00:00",
      "score": 0.91,
      "relevance_score": 0.90,
      "freshness_score": 0.98,
      "popularity_score": 0.60,
      "index_freshness_ms": 62.5
    }
  ]
}

index_freshness_ms is the event's stored produced_at to PostgreSQL indexed_at duration. search_ms measures candidate retrieval and ranking for this request. Query length is limited to 500 characters, top_k to 1–100, and database candidates to NEWSLENS_REALTIME_CANDIDATE_LIMIT.

Validation behavior

The API rejects:

  • empty candidate lists;
  • duplicate candidate identifiers;
  • empty article identifiers;
  • top_k values below 1;
  • top_k values above 100; and
  • unexpected request fields.

Invalid requests return HTTP 422.

Inference without a loaded model returns HTTP 503. Search without a ready real-time store also returns 503; either route can remain ready while the other is unavailable.

Testing

Run API tests:

python -m pytest tests/test_api.py -q

Run all quality checks:

python -m ruff check .
python -m pytest
git diff --check

Readiness and observability

NewsLens separates HTTP-service liveness from model-serving readiness.

Liveness

GET /health confirms that the HTTP service is running. It does not guarantee that a model artifact has been loaded.

curl -i http://127.0.0.1:8000/health

Example response:

{
  "status": "ok",
  "service": "newslens",
  "version": "0.3.0"
}

Model readiness

GET /ready reports whether the service can process recommendation requests.

curl -i http://127.0.0.1:8000/ready

When a verified model artifact is loaded, the endpoint returns 200 OK:

{
  "status": "ready",
  "model_ready": true,
  "artifact_version": "0.3.0"
}

When no artifact is loaded, it returns 503 Service Unavailable.

This separation allows deployment platforms to distinguish between:

  • a running HTTP process; and
  • an application that is ready to serve model-backed traffic.

Request identification

Every HTTP response includes an X-Request-ID header.

Clients may provide their own request identifier:

curl -i \
  -H "X-Request-ID: example-request-001" \
  http://127.0.0.1:8000/health

If no identifier is supplied, NewsLens generates one automatically.

The request identifier is included in request logs and recommendation responses so that one request can be traced through the service.

HTTP processing time

Every response includes:

X-Process-Time-Ms

This header reports the total server-side HTTP processing time in milliseconds.

Recommendation inference time

Successful POST /recommend responses include:

  • request_id;
  • inference_ms;
  • model name;
  • artifact version;
  • requested ranking cutoff;
  • returned recommendation count; and
  • recommendation routing source.

Example request:

curl -i \
  -X POST \
  http://127.0.0.1:8000/recommend \
  -H "Content-Type: application/json" \
  -H "X-Request-ID: recommendation-example-001" \
  -d '{
    "history_news_ids": ["N32211"],
    "candidate_news_ids": [
      "N47020",
      "N16616",
      "N34081"
    ],
    "top_k": 3
  }'

Service logs

NewsLens emits logs for:

  • completed HTTP requests;
  • failed HTTP requests; and
  • completed recommendation inference.

Recommendation logs include:

  • request ID;
  • artifact version;
  • history size;
  • candidate-set size;
  • requested ranking cutoff;
  • returned recommendation count;
  • routing source; and
  • inference latency.

These measurements provide local service observability. They do not yet represent a complete production monitoring, metrics-storage, or alerting system.