feat(observability): service-state OTel metrics, PrometheusRule, Grafana dashboard

[QuentinBisson]

Closes #401

What changed

Go: service lifecycle metrics (internal/orchestrator/metrics.go)

Two new OTel instruments emitted from the orchestrator's state-change hook:

Metric	Type	Labels
muster.service.state_transitions_total	Counter	service_name, service_type, from_state, to_state
muster.service.up	Observable Gauge	service_name, service_type

muster.service.up is 1 when a service is in running or connected state, 0 otherwise — the standard "is it up?" signal for alerting.

Both use otel.Meter(observability.TracerName), the same scope as the existing muster.tool_calls counter in internal/aggregator/metrics.go. The global MeterProvider is already initialised in cmd/serve.go via mcp-toolkit, so no additional bootstrap is needed.

Helm: PrometheusRule (helm/muster/templates/prometheusrule.yaml)

Three alerting rules (guarded by prometheusExporterEnabled, same as ServiceMonitor):

• MusterServiceDown — fires after 5 min with muster_service_up == 0
• MusterServiceFlapping — fires immediately on > 4 transitions in 10 min
• MusterHighToolErrorRate — fires after 5 min when tool error ratio > 10 %

Supports observability.giantswarm.io/tenant label for multi-tenant Mimir.

Enable with:

muster:
  observability:
    metrics:
      exporter: prometheus
      prometheus:
        prometheusRule:
          enabled: true
          tenant: giantswarm   # optional

Helm: Grafana dashboard (helm/muster/templates/grafana-dashboard.yaml)

ConfigMap carrying a ready-to-use dashboard, picked up by the standard grafana-sidecar. Panels:

• Service status stat (up/down per service)
• State transitions timeseries
• Tool call rate by tool + outcome
• Tool call latency p50/p99
• Tool error rate per tool

Enable with:

muster:
  observability:
    metrics:
      grafanaDashboard:
        enabled: true

What was already present (not changed)

• muster.tool_calls counter + muster.tool_call.duration histogram in internal/aggregator/metrics.go
• mcp-go OTel tracing hooks (internal/aggregator/server_options.go)
• cmd/serve.go tracing + metrics init via mcp-toolkit
• ServiceMonitor Helm template and deployment OTel env vars
• values.yaml observability section