Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,7 @@ and only needed for the project-specific scenarios noted.
| [Kiali](https://kiali.io) | `kiali` | 16 |
| [Kubernetes](https://kubernetes.io) | - | 32 |
| [KubeVirt](https://kubevirt.io) | `kubevirt`, `tekton` | 19 |
| [LVMS](https://github.com/openshift/lvm-operator) | `lvms` | 11 |
| [Tekton](https://tekton.dev) | `tekton` | 9 |

<!-- VALIDATED-PROJECTS-END -->
Expand All @@ -292,6 +293,7 @@ The following sets of tools are available (toolsets marked with ✓ in the Defau
| kcp | Manage kcp workspaces and multi-tenancy features | |
| kiali | Most common tools for managing Kiali, check the [Kiali documentation](https://github.com/containers/kubernetes-mcp-server/blob/main/docs/KIALI.md) for more details. | |
| kubevirt | KubeVirt virtual machine management tools, check the [KubeVirt documentation](https://github.com/containers/kubernetes-mcp-server/blob/main/docs/kubevirt.md) for more details. | |
| lvms | LVMS (Logical Volume Manager Storage) troubleshooting prompts for diagnosing storage issues | |
| netobserv | Network observability tools backed by the NetObserv console plugin API (flows, metrics, export). Check the [NetObserv documentation](https://github.com/containers/kubernetes-mcp-server/blob/main/docs/NETOBSERV.md) for more details. | |
| tekton | Tekton pipeline management tools for Pipelines, PipelineRuns, Tasks, and TaskRuns. | |

Expand Down Expand Up @@ -498,6 +500,19 @@ In case multi-cluster support is enabled (default) and you have access to multip

<details>

<summary>lvms</summary>

- **lvms-troubleshoot** - Diagnose LVMS storage issues by gathering LVMCluster status, volume group health, and node-level LVM data with interpretation of domain-specific fields
- `namespace` (`string`) - The LVMS namespace (default: openshift-lvm-storage)
- `node` (`string`) - Specific node to inspect (default: all nodes with LVMS)

- **lvms-capacity** - Check LVMS storage capacity across all nodes including available space per node, PVC usage, and capacity warnings
- `namespace` (`string`) - The LVMS namespace (default: openshift-lvm-storage)

</details>

<details>

<summary>netobserv</summary>

- **netobserv_list_flows** - Lists NetObserv network flow records from Loki. Use when investigating traffic between workloads, IPs, ports, or protocols in a namespace or time window.
Expand Down
1 change: 1 addition & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,7 @@ Toolsets group related tools together. Enable only the toolsets you need to redu
| kcp | Manage kcp workspaces and multi-tenancy features | |
| kiali | Most common tools for managing Kiali, check the [Kiali documentation](https://github.com/containers/kubernetes-mcp-server/blob/main/docs/KIALI.md) for more details. | |
| kubevirt | KubeVirt virtual machine management tools, check the [KubeVirt documentation](https://github.com/containers/kubernetes-mcp-server/blob/main/docs/kubevirt.md) for more details. | |
| lvms | LVMS (Logical Volume Manager Storage) troubleshooting prompts for diagnosing storage issues | |
| netobserv | Network observability tools backed by the NetObserv console plugin API (flows, metrics, export). Check the [NetObserv documentation](https://github.com/containers/kubernetes-mcp-server/blob/main/docs/NETOBSERV.md) for more details. | |
| tekton | Tekton pipeline management tools for Pipelines, PipelineRuns, Tasks, and TaskRuns. | |

Expand Down
11 changes: 11 additions & 0 deletions evals/claude-code/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -84,3 +84,14 @@ config:
toolPattern: "netobserv_.*"
minToolCalls: 1
maxToolCalls: 10
# LVMS tasks - requires LVMS operator on cluster
# LVMS is a prompts-only toolset; uses core tools for K8s queries
- glob: ../tasks/lvms/*/*.yaml
labelSelector:
suite: lvms
assertions:
toolsUsed:
- server: kubernetes
toolPattern: ".*"
minToolCalls: 1
maxToolCalls: 20
Comment thread
coderabbitai[bot] marked this conversation as resolved.
11 changes: 11 additions & 0 deletions evals/openai-agent/eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -82,3 +82,14 @@ config:
toolPattern: "netobserv_.*"
minToolCalls: 1
maxToolCalls: 10
# LVMS tasks - requires LVMS operator on cluster
# LVMS is a prompts-only toolset; uses core tools for K8s queries
- glob: ../tasks/lvms/*/*.yaml
labelSelector:
suite: lvms
assertions:
toolsUsed:
- server: kubernetes
toolPattern: ".*"
minToolCalls: 1
maxToolCalls: 20
57 changes: 57 additions & 0 deletions evals/tasks/lvms/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# LVMS Task Library

Evaluation tasks for the LVMS (Logical Volume Manager Storage) toolset. These tasks exercise storage troubleshooting workflows on OpenShift clusters using LVMS.

## Prerequisites

- LVMS operator installed on the cluster
- LVMCluster CR configured (typically in `openshift-lvm-storage` namespace)
- Available block devices on nodes for volume group creation

## Toolset Design

LVMS is a **prompts-only toolset** — it provides no dedicated tools. This design follows the toolset-design principle that CRUD operations on Kubernetes resources should use the core toolset's generic tools (`resources_list`, `resources_get`, etc.).

### What the LVMS toolset provides:

- **`lvms-troubleshoot` prompt** — Pre-fetches and presents:
- LVMCluster status and device class configuration
- LVMVolumeGroupNodeStatus per node
- vg-manager pod health and logs
- **Node-level LVM data** (vgs, lvs, pvs output via debug pods)
- Events in the LVMS namespace
- Domain knowledge (vg_attr/lv_attr field interpretation)

### What eval tasks use:

| Tool | Source | Purpose |
|------|--------|---------|
| `resources_list` / `resources_get` | core | Query LVMS CRDs (LVMCluster, LVMVolumeGroup, etc.) |
| `nodes_debug_exec` | cluster-diagnostics | Run LVM commands on nodes (vgs, lvs, pvs, lsblk) |
| `lvms-troubleshoot` prompt | lvms | Guided diagnostics with pre-fetched data |

## Task Categories

| Task | Difficulty | Description |
|------|------------|-------------|
| check-lvmcluster-status | Easy | Verify LVMCluster health and device class configuration |
| check-capacity | Medium | Analyze thin pool utilization and storage capacity |
| check-thin-pool-utilization | Medium | Check thin pool data% and metadata% thresholds |
| list-node-block-devices | Easy | List available block devices on nodes |
| show-volume-groups | Easy | Display volume group status |
| show-lvm-physical-volumes | Easy | Display physical volume status |
| diagnose-no-space-left | Hard | Diagnose thin pool exhaustion issues |
| diagnose-disk-not-used | Hard | Investigate why a disk isn't being used by LVMS |
| troubleshoot-stuck-pvc | Hard | Diagnose why a PVC is stuck in Pending state |
| troubleshoot-prompt | Medium | Use the lvms-troubleshoot prompt for guided diagnosis |
| forced-cleanup | Hard | Perform forced cleanup of stuck LVMS resources |

## Running Tasks

```bash
# Run all LVMS tasks
mcpchecker check evals/claude-code/eval.yaml --label-selector suite=lvms

# Run specific difficulty level
mcpchecker check evals/claude-code/eval.yaml --label-selector suite=lvms,difficulty=hard
```
68 changes: 68 additions & 0 deletions evals/tasks/lvms/check-capacity/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
kind: Task
metadata:
labels:
suite: lvms
project: lvms
requires: lvms
annotations:
project-name: "LVMS"
project-url: "https://github.com/openshift/lvm-operator"
name: "check-lvms-capacity"
difficulty: medium
description: "Check LVMS storage capacity and thin pool utilization across nodes"
steps:
setup:
inline: |-
#!/usr/bin/env bash
NS="openshift-lvm-storage"

# Verify LVMS is installed
if ! kubectl get crd lvmclusters.lvm.topolvm.io > /dev/null 2>&1; then
echo "LVMS CRD not found - LVMS must be installed"
exit 1
fi

# Check if an LVMCluster exists and is ready
CLUSTER=$(kubectl get lvmclusters -n "$NS" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
if [ -z "$CLUSTER" ]; then
echo "No LVMCluster found - LVMS must be configured"
exit 1
fi

STATE=$(kubectl get lvmcluster "$CLUSTER" -n "$NS" -o jsonpath='{.status.state}' 2>/dev/null)
if [ "$STATE" != "Ready" ]; then
echo "LVMCluster '$CLUSTER' is not ready (state: $STATE)"
exit 1
fi

echo "LVMS is configured with LVMCluster: $CLUSTER"
verify:
inline: |-
#!/usr/bin/env bash
NS="openshift-lvm-storage"

echo "=== Verification: Checking agent capacity analysis ==="

# List node statuses
kubectl get lvmvolumegroupnodestatuses -n "$NS" -o custom-columns='NODE:.metadata.name,STATUS:.status' 2>/dev/null || true

echo ""
echo "=== Capacity Eval Complete ==="
echo "The agent should have:"
echo " 1. Queried LVMS volume group status across nodes"
echo " 2. Reported per-node storage capacity (total, allocatable)"
echo " 3. Identified any nodes with low available space"
echo " 4. Provided capacity planning recommendations if applicable"
echo ""

exit 0
cleanup:
inline: |-
#!/usr/bin/env bash
# No cleanup needed - read-only operation
exit 0
prompt:
inline: |-
We're planning to deploy several new applications that will need persistent storage.
How much LVMS storage capacity do we have available across the cluster, and are there
any nodes running low on space?
79 changes: 79 additions & 0 deletions evals/tasks/lvms/check-lvmcluster-status/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
kind: Task
metadata:
labels:
suite: lvms
project: lvms
requires: lvms
annotations:
project-name: "LVMS"
project-url: "https://github.com/openshift/lvm-operator"
name: "check-lvms-cluster-status"
difficulty: easy
description: "Check the status of the LVMCluster and verify it is healthy"
steps:
setup:
inline: |-
#!/usr/bin/env bash
NS="openshift-lvm-storage"

# Verify LVMS is installed
if ! kubectl get crd lvmclusters.lvm.topolvm.io > /dev/null 2>&1; then
echo "LVMS CRD not found - LVMS must be installed"
exit 1
fi

# Check if an LVMCluster exists
if [ -z "$(kubectl get lvmclusters -n "$NS" -o name 2>/dev/null)" ]; then
echo "No LVMCluster found in $NS - creating test cluster"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
cat <<EOF | kubectl apply -f -
apiVersion: lvm.topolvm.io/v1alpha1
kind: LVMCluster
metadata:
name: eval-lvmcluster
namespace: $NS
spec:
storage:
deviceClasses:
- name: vg1
default: true
fstype: xfs
thinPoolConfig:
name: thin-pool-1
sizePercent: 90
overprovisionRatio: 10
EOF
echo "Created eval-lvmcluster - waiting for reconciliation"
sleep 30
fi
verify:
inline: |-
#!/usr/bin/env bash
NS="openshift-lvm-storage"

echo "=== Verification: Checking agent findings ==="

# The agent should have used list_lvmclusters or describe_lvmcluster
# and reported the cluster status

CLUSTER=$(kubectl get lvmclusters -n "$NS" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
if [ -n "$CLUSTER" ]; then
STATE=$(kubectl get lvmcluster "$CLUSTER" -n "$NS" -o jsonpath='{.status.state}' 2>/dev/null || echo "Unknown")
READY=$(kubectl get lvmcluster "$CLUSTER" -n "$NS" -o jsonpath='{.status.ready}' 2>/dev/null || echo "Unknown")
echo "LVMCluster '$CLUSTER' state: $STATE, ready: $READY"
fi

echo ""
echo "=== Eval Complete ==="
echo "The agent should have:"
echo " 1. Queried the LVMCluster resource"
echo " 2. Reported the cluster name, state, and ready status"
echo " 3. Listed any device classes configured"

exit 0
cleanup:
inline: |-
#!/usr/bin/env bash
# Only clean up the eval cluster if we created it
kubectl delete lvmcluster eval-lvmcluster -n openshift-lvm-storage --ignore-not-found
prompt:
inline: Is LVMS storage configured and working properly on this cluster? What storage options are available?
52 changes: 52 additions & 0 deletions evals/tasks/lvms/check-thin-pool-utilization/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
kind: Task
metadata:
labels:
suite: lvms
project: lvms
requires: lvms
annotations:
project-name: "LVMS"
project-url: "https://github.com/openshift/lvm-operator"
name: "check-thin-pool-utilization"
difficulty: medium
description: "Check actual thin pool data and metadata utilization on nodes"
steps:
setup:
inline: |-
#!/usr/bin/env bash
NS="openshift-lvm-storage"

# Verify LVMS is installed and ready
if ! kubectl get crd lvmclusters.lvm.topolvm.io > /dev/null 2>&1; then
echo "LVMS CRD not found - LVMS must be installed"
exit 1
fi

CLUSTER=$(kubectl get lvmclusters -n "$NS" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
if [ -z "$CLUSTER" ]; then
echo "No LVMCluster found - LVMS must be configured"
exit 1
fi

echo "LVMS is configured. This task requires checking LIVE thin pool utilization."
verify:
inline: |-
#!/usr/bin/env bash
echo "=== Verification ==="
echo "The agent should have:"
echo " 1. Run 'lvs' command on the node to get thin pool data"
echo " 2. Reported data_percent and metadata_percent for thin pools"
echo " 3. This CANNOT be done with resources_list/resources_get"
echo ""
echo "If the agent only queried LVMVolumeGroupNodeStatus CRD,"
echo "it did NOT get live thin pool utilization - that requires lvs on node."
exit 0
cleanup:
inline: |-
#!/usr/bin/env bash
exit 0
prompt:
inline: |-
What is the current thin pool utilization on each node? I need the actual
data_percent and metadata_percent values from the LVM thin pools, not just
the Kubernetes CRD status.
54 changes: 54 additions & 0 deletions evals/tasks/lvms/diagnose-disk-not-used/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
kind: Task
metadata:
labels:
suite: lvms
project: lvms
requires: lvms
annotations:
project-name: "LVMS"
project-url: "https://github.com/openshift/lvm-operator"
name: "diagnose-disk-not-used"
difficulty: hard
description: "Diagnose why a specific disk is not being used by LVMS"
steps:
setup:
inline: |-
#!/usr/bin/env bash
NS="openshift-lvm-storage"

# Verify LVMS is installed
if ! kubectl get crd lvmclusters.lvm.topolvm.io > /dev/null 2>&1; then
echo "LVMS CRD not found - LVMS must be installed"
exit 1
fi

echo "LVMS installed. This task tests disk filtering knowledge."
echo ""
echo "The agent must know LVMS device filtering rules:"
echo "- Read-only devices are excluded"
echo "- Devices with children (partitions) are excluded"
echo "- Devices with reserved partition labels are excluded"
echo "- LVM2_member fstype devices are excluded unless explicitly listed"
echo "- Loop devices in use by Kubernetes are excluded"
verify:
inline: |-
#!/usr/bin/env bash
echo "=== Verification ==="
echo "The agent should have:"
echo " 1. Identified the node where the disk exists"
echo " 2. Run lsblk to inspect disk properties"
echo " 3. Checked LVMCluster deviceSelector configuration"
echo " 4. Identified which LVMS filtering rule excludes the disk"
echo " 5. Suggested how to make the disk available (if possible)"
exit 0
cleanup:
inline: |-
#!/usr/bin/env bash
exit 0
prompt:
inline: |-
A user reports that /dev/sdb on node worker-1 is not being used by LVMS
even though they expected it to be available for storage.

Investigate why this disk is not being used by LVMS and explain
the reason to the user.
Loading