Support creating/updating prometheus rules#5228
Support creating/updating prometheus rules#5228lezzago wants to merge 5 commits intoopensearch-project:mainfrom
Conversation
Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
PR Reviewer Guide 🔍(Review updated until commit b0ab3b5)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to b0ab3b5 Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit b5480e9
Suggestions up to commit c047583
Suggestions up to commit 3a17d23
Suggestions up to commit 98d6a33
|
Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
|
Persistent review updated to latest commit 3a17d23 |
Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
|
Persistent review updated to latest commit c047583 |
Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
|
Persistent review updated to latest commit b5480e9 |
Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
|
Persistent review updated to latest commit b0ab3b5 |
Description
Adds CRUD support for the Prometheus/Cortex/AMP Ruler API and Alertmanager enhancements through OpenSearch's direct query resources framework.
Prometheus Rules — enables management of recording and alerting rule groups via the Cortex/Thanos Ruler API, compatible with Amazon Managed Prometheus (AMP), Cortex, Thanos, and Grafana Mimir.
Alertmanager — adds support for fetching Alertmanager status/configuration and expiring silences.
Vanilla Prometheus does not support the Ruler API (it manages rules via config files). Write/delete operations return clear errors when used against vanilla Prometheus. The existing
GET /api/v1/rules(all rules) continues to work against all backends.New API Endpoints
Rules — under
/_plugins/_directquery/_resources/{dataSource}/api/v1/rules/:GET/{namespace}POST/{namespace}DELETE/{namespace}DELETE/{namespace}/{groupName}Alertmanager — under
/_plugins/_directquery/_resources/{dataSource}/alertmanager/api/v2/:GET/statusDELETE/silence/{silenceID}Response Normalization
Different backends return rules in different formats — Prometheus/AMP return JSON, Cortex/Thanos return YAML. A
normalizeRulesResponse()method converts all formats into a consistent{"groups":[...]}JSON structure with afilefield on each group indicating the namespace. The existinggetRules()return type remainsJSONObject(no breaking change).Configurable Ruler URI
A new optional
prometheus.ruler.uridatasource property provides a separate base URL for Ruler CRUD operations. This is needed because Cortex serves the Ruler API at a different base path than the Prometheus-compatible query API. When not set, defaults toprometheus.uri(works for AMP out of the box). Fully backward compatible — existing datasources work without changes.Example Cortex configuration:
{ "prometheus.uri": "http://cortex:9009/api/prom", "prometheus.ruler.uri": "http://cortex:9009" }Design Decisions
writeDirectQueryResourcestransport action using adeleteflag onWriteDirectQueryResourcesRequest, avoiding a separate delete transport layer.Content-Type: application/yaml.WriteDirectQueryResourcesRequestextended withgroupNameanddeletefields usingreadOptionalString/writeOptionalBooleanfor serialization safety.Other Changes
readRulerResponsehelper to consolidate duplicated HTTP error handling across ruler methodsResponseobjects in ruler methods to prevent connection leaksALERTMANAGER_STATUSadded toDirectQueryResourceTypeenumRelated Issues
N/A
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.