-
Notifications
You must be signed in to change notification settings - Fork 5
Open
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is needed
Description
Description
Create a /metrics endpoint that exposes basic server metrics in a simple JSON format.
🔄 Current Status
Partially Implemented: The /model/info endpoint currently provides comprehensive statistics, but a dedicated /metrics endpoint would be more appropriate for monitoring systems.
✅ Already Implemented
- Metrics tracking in existing request handlers
- Thread-safe metrics collection using
Arc<Mutex<ModelStats>> - JSON format responses
- Integration tests for metrics via
/model/info
Current metrics available via /model/info:
- Total inference count
- Total inference time
- Average inference time
- Error count and success count
- Memory usage estimation
- Min/max inference times
- Last inference timestamp
🔄 Remaining Work
- Add dedicated
GET /metricsendpoint tosrc/api.rs - Separate server-level metrics from model-specific metrics
- Add total requests counter (not just inference requests)
- Add average response time for all endpoints
- Consider Prometheus-compatible format option
Implementation Guidance
- Extract metrics logic from
ModelStatsto a separateServerMetricsstruct - Keep existing
/model/infofor model-specific statistics - New
/metricsfor server-wide metrics - Use atomic counters for high-performance metrics collection
Estimated Difficulty
Easy - 2-3 hours (reduced due to existing metrics infrastructure)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is needed