Skip to content

📊 Add basic metrics endpoint #4

@Gilfeather

Description

@Gilfeather

Description

Create a /metrics endpoint that exposes basic server metrics in a simple JSON format.

🔄 Current Status

Partially Implemented: The /model/info endpoint currently provides comprehensive statistics, but a dedicated /metrics endpoint would be more appropriate for monitoring systems.

Already Implemented

  • Metrics tracking in existing request handlers
  • Thread-safe metrics collection using Arc<Mutex<ModelStats>>
  • JSON format responses
  • Integration tests for metrics via /model/info

Current metrics available via /model/info:

  • Total inference count
  • Total inference time
  • Average inference time
  • Error count and success count
  • Memory usage estimation
  • Min/max inference times
  • Last inference timestamp

🔄 Remaining Work

  • Add dedicated GET /metrics endpoint to src/api.rs
  • Separate server-level metrics from model-specific metrics
  • Add total requests counter (not just inference requests)
  • Add average response time for all endpoints
  • Consider Prometheus-compatible format option

Implementation Guidance

  • Extract metrics logic from ModelStats to a separate ServerMetrics struct
  • Keep existing /model/info for model-specific statistics
  • New /metrics for server-wide metrics
  • Use atomic counters for high-performance metrics collection

Estimated Difficulty

Easy - 2-3 hours (reduced due to existing metrics infrastructure)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions