[jaineel132]

Summary

Adds observability to Mnema through a Prometheus /metrics endpoint and opt-in structured JSON logging. Instrumentation is implemented in the service layer, ensuring coverage across MCP tools, the SDK, CLI, and REST API.

Related issue

Closes #11

Changes

• pyproject.toml

  • Added a new [observability] extra:

    • prometheus-client>=0.21
    • python-json-logger>=3.2

• config.py

  • Added the MNEMA_LOG_FORMAT configuration field with:

    • text as the default format
    • json as the optional structured logging format

• diagnostics.py

  • Added a Prometheus metric registry containing:

    • mnema_memories_total counter with an operation label
    • mnema_embed_latency_seconds histogram
    • mnema_backend_latency_seconds histogram
    • mnema_memory_count gauge

  • Uses a private CollectorRegistry to prevent duplicate-timeseries errors when Mnema modules are reloaded during tests.

  • Added _NoOpMetric fallbacks when the observability extra is not installed.

  • Added JSON formatter support in configure_logging().

  • Falls back to text logging when python-json-logger is unavailable and prints an installation hint to stderr.

  • Added the shared metrics_payload() helper.

• service.py

  • Instrumented:

    • remember
    • recall
    • search
    • update
    • forget
    • forget_scope
    • stats

  • Added embedding and backend latency measurements.

  • Added operation counters and memory-count gauge updates.

  • Added structured logger.info() calls.

• api/app.py

  • Added the GET /metrics endpoint to the REST API.
  • Returns 501 with an installation hint when the Prometheus dependency is unavailable.

• server.py

  • Added the GET /metrics endpoint to the MCP streamable-HTTP transport using FastMCP.custom_route.

• tests/test_observability.py

  • Added 10 tests covering:

    • Metrics endpoint exposure on both HTTP transports
    • Counter and gauge updates
    • JSON formatter behavior and fallbacks
    • 501 behavior when prometheus_client is unavailable

• Documentation and repository configuration

  • Added an observability entry under Unreleased in CHANGELOG.md.
  • Added MNEMA_LOG_FORMAT to the configuration table in README.md.
  • Added the observability extra to the README extras list.
  • Added opencode.json to .gitignore.

Checklist

• ruff check . is clean
• pytest passes with the relevant extras installed
• New public functions and classes have docstrings
• Documentation updated where user-facing behavior changed
• Not applicable: this PR does not add a backend or embedding provider

Notes for reviewer

• Optional dependency discipline is preserved: importing mnema does not require the new observability dependencies. Metrics use _NoOpMetric fallbacks when prometheus-client is unavailable, and MNEMA_LOG_FORMAT=json falls back to text logging when python-json-logger is unavailable. Both cases provide an installation hint through stderr.

• Metrics use a module-owned private CollectorRegistry, preventing prometheus_client's duplicate-timeseries errors when Mnema modules are reloaded during tests.

• The /metrics endpoint is available on both HTTP surfaces:

  • REST API through mnema serve
  • MCP streamable-HTTP transport through FastMCP.custom_route

  The stdio transport has no HTTP endpoint, so mnema serve is the scrape path for stdio-based deployments.

• The python-json-logger>=3.2 requirement uses the pythonjsonlogger.json import path, avoiding the deprecated jsonlogger module warning introduced in version 4.x.

• mypy introduces no new errors compared with the baseline. Pre-existing errors in cli.py and server.py remain unchanged and were verified against main.

• Four backend-matrix skips are pre-existing because optional backend dependencies are not installed locally. The full suite passes with [dev,observability] and fastapi installed.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.