diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/RIGHTS.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/RIGHTS.md new file mode 100644 index 0000000..65b0ee8 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/RIGHTS.md @@ -0,0 +1,16 @@ +# Submission rights declaration + +Project: `MCP Doctor` +Submission slug: `shahadattest-mcp-doctor` +Submitter: `shahadattest` +Date: `2026-09-04` + +The submitter confirms that they own, or have sufficient authorization for, the source code, dependencies, service, data, branding, and other materials submitted in this pull request. + +Subject to the official program terms, the submitter authorizes X-Agent to retain, reproduce, audit, test, archive, and publish the submitted program artifact for judging, fraud prevention, dispute handling, ecosystem submission, and post-award accountability. Closing the pull request, deleting a fork, or deleting an external repository does not revoke the official archive rights attached to an accepted and rewarded entry. + +Third-party components and their licenses: FastAPI (MIT), uvicorn (BSD), Pydantic v2 (MIT), SQLAlchemy (MIT), httpx (BSD), PyYAML (MIT), jsonschema (MIT), pytest (MIT), nginx (BSD) — see `source/` manifests. + +Exceptions or restrictions: `none` + +This template is an operational declaration, not a substitute for event terms reviewed by qualified counsel. diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/SUBMISSION.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/SUBMISSION.md new file mode 100644 index 0000000..553e922 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/SUBMISSION.md @@ -0,0 +1,47 @@ +# MCP Doctor + +## Capability + +- **One-line description:** Turn any OpenAPI-described API into a tested, normalized, agent-ready tool with a before/after readiness score. +- **Who it helps:** Developers and AI agents that need strict schemas, predictable outputs, and standardized failures from third-party APIs. +- **Capability boundary:** Does API discovery, live safe-method testing, deterministic quality scoring, declarative response normalization, proxying, and tool-JSON generation. Does NOT modify upstream APIs, does NOT do security auditing, penetration testing, or risk scoring. + +## Live API + +- **API base URL:** `https://belts-raymond-advertisements-radical.trycloudflare.com/api` (local verified: `http://localhost:8000/api`) +- **Health-check URL:** `https://belts-raymond-advertisements-radical.trycloudflare.com/health` +- **Authentication:** none +- **Rate limits / known limits:** No auth limits; upstream calls timeout 10s, max 2 retries, 1MB response cap. Auto-test covers GET/HEAD/OPTIONS only. +- **API contract:** `source/examples/broken-demo-api/openapi-demo.json` + live OpenAPI at `/openapi.json`; endpoint docs in `source/docs/api.md`. + +## Source and reproducibility + +- **Source repository:** `https://github.com/ShahadatTest/mcp-doctor` +- **Review commit:** `3ce175fb14a0e176a9b29a2c213d78d4fab91bad` +- **Source submitted in this PR:** `source/` +- **Run tests:** `cd source/backend && pip install -r requirements.txt && python -m pytest tests/ -q` (8 passed) +- **Run locally:** `cd source && docker-compose up --build` (frontend :3000, backend :8000, demo :8001) +- **Deploy:** build `source/backend/Dockerfile`, set `GIT_COMMIT=` and `PROJECT_SLUG=shahadattest-mcp-doctor` +- **Version binding:** `/health` returns `{"status":"ok","commit":""}` and `/.well-known/xagent-verification.json` returns `{"schemaVersion":1,"slug":"shahadattest-mcp-doctor","commit":""}` + +## Verification + +The reproducible call instructions and redacted example responses are in `verification/README.md`. + +- **Health-check result:** `{"status":"ok","service":"mcp-doctor","version":"0.1.0","commit":"3ce175fb14a0e176a9b29a2c213d78d4fab91bad"}` +- **Capability call:** `POST /api/projects` with `{"name":"Demo Weather API","openapi_json":{...}}` → project created, 4 endpoints discovered, readiness 62/100, repair rules generated, proxy normalizes `{"tmp":"31 C","desc":"sun"}` → `{"temperature_celsius":31,"condition":"sunny"}` +- **Expected error behavior:** invalid spec → 400; unsafe method auto-test → `skipped`; unreachable upstream → `UPSTREAM_TIMEOUT` with `retryable:true`; bad proxy args → `INVALID_ARGUMENT` + +## Security and data handling + +- **Data collected:** Project specs and test metadata the reviewer submits; no end-user data. +- **Purpose and retention:** Review/demo only, stored in local SQLite file. +- **Third parties / outbound network calls:** Only the upstream API under test (reviewer-supplied URL), via httpx with SSRF guard. +- **Secrets:** No secrets are committed. Review access is supplied only through an approved private channel when required. +- **Known risks / restrictions:** Set `ALLOW_PRIVATE_NETWORK=true` only for local demo against localhost; keep `false` in production. + +## Support + +- **Team / builder:** shahadattest (solo) +- **Contact:** via GitHub `shahadattest` +- **License / rights:** MIT (see `source/LICENSE`); submitter authorizes review and archival per RIGHTS.md. diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/.env.example b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/.env.example new file mode 100644 index 0000000..a334359 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/.env.example @@ -0,0 +1,10 @@ +PORT=8000 +DATABASE_URL=sqlite:///./mcp_doctor.db +ALLOW_PRIVATE_NETWORK=false +REQUEST_TIMEOUT=10 +MAX_RETRIES=2 +GIT_COMMIT=dev-local +PROJECT_SLUG=team-mcp-doctor +LLM_BASE_URL= +LLM_API_KEY= +LLM_MODEL= diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/LICENSE b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/LICENSE new file mode 100644 index 0000000..5fb1fbd --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/LICENSE @@ -0,0 +1 @@ +MIT License — MCP Doctor (hackathon MVP). diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/README.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/README.md new file mode 100644 index 0000000..f49edd1 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/README.md @@ -0,0 +1,56 @@ +# MCP Doctor + +**MCP Doctor turns messy or unreliable APIs into consistent, validated, agent-ready tools.** + +Give MCP Doctor an API. It tests the API, finds what makes it unreliable for AI agents, creates a normalized compatibility layer, validates the repaired interface, and generates an agent-ready tool. + +> Postman tests APIs. MCP Doctor prepares them for AI agents. + +## How it works + +Import → Test → Diagnose → Repair → Agentize → Retest → Generate tool. Before/after readiness score is the hero moment (e.g. 43 → 96). + +## Quick start (local) + +```bash +cd mcp-doctor/backend +pip install -r requirements.txt +python -m uvicorn app.main:app --port 8000 +# demo api +cd ../examples/broken-demo-api +python -m uvicorn main:app --port 8001 +# open ../frontend/index.html (set backend http://localhost:8000) +``` + +Set `ALLOW_PRIVATE_NETWORK=true` for local demo testing against localhost. + +## Docker + +```bash +cd mcp-doctor +docker-compose up --build +# frontend http://localhost:3000 backend http://localhost:8000/health demo http://localhost:8001/health +``` + +## API examples + +```bash +curl http://localhost:8000/health +curl -X POST http://localhost:8000/api/projects -H 'Content-Type: application/json' -d '{"name":"Demo","openapi_url":"http://demo-api:8001/openapi.json"}' +``` + +## Project structure + +See `docs/architecture.md`. Backend `backend/app/services/*`, demo `examples/broken-demo-api`, dashboard `frontend/`. + +## Security + +SSRF guard, timeout/retry, size limits, safe-method-only auto-test, no codegen execution, secrets via env. This is a compatibility tool, not a vulnerability scanner. + +## Limitations / Future + +Static dashboard (Next.js port later), repair simulation scoring, single-table SQLite, no auth/billing yet. Roadmap: drift detection, monitoring, self-healing adapters, MCP server export, pay-per-call. + +## License + +MIT (see LICENSE). diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/Dockerfile b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/Dockerfile new file mode 100644 index 0000000..a62a795 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/Dockerfile @@ -0,0 +1,9 @@ +FROM python:3.12-slim +WORKDIR /code +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt +COPY app ./app +COPY tests ./tests +ENV PORT=8000 +EXPOSE 8000 +CMD ["sh","-c","python -m uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-8000}"] diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/__init__.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/api/__init__.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/api/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/api/projects.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/api/projects.py new file mode 100644 index 0000000..835f896 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/api/projects.py @@ -0,0 +1,316 @@ +import json +import time +import uuid +import httpx +from fastapi import APIRouter, HTTPException, UploadFile, File, Form +from sqlalchemy.orm import Session + +from app.core.config import MAX_UPLOAD_BYTES +from app.core.security import validate_url_for_fetch +from app.models.db import ProjectRow, engine +from app.schemas.api import ProjectCreate, ProxyRequest +from app.services import importer as imp +from app.services.analyzer import analyze_endpoints +from app.services.scoring import compute_score +from app.services.tester import test_endpoint +from app.services.repair import apply_repairs, auto_suggest_repairs +from app.services.tools import generate_tools + +router = APIRouter() + + +def _db() -> Session: + return Session(engine, expire_on_commit=False) + + +def _row_to_dict(r: ProjectRow) -> dict: + return {"id": r.id, "name": r.name, "openapi_url": r.openapi_url, + "spec": json.loads(r.spec_json or "{}")} + + +def _save(r: ProjectRow, db: Session): + db.add(r) + db.commit() + + +@router.post("/projects") +async def create_project(body: ProjectCreate): + spec = None + if body.openapi_json: + spec = imp.parse_spec_content(body.openapi_json) + elif body.openapi_url: + validate_url_for_fetch(body.openapi_url) + try: + async with httpx.AsyncClient(timeout=15, follow_redirects=True, max_redirects=3) as c: + resp = await c.get(body.openapi_url) + resp.raise_for_status() + try: + spec = imp.parse_spec_content(resp.json()) + except Exception: + spec = imp.parse_spec_content(resp.text) + except ValueError as e: + raise HTTPException(400, str(e)) + except Exception as e: + raise HTTPException(400, f"Failed to fetch OpenAPI: {str(e)[:300]}") + else: + raise HTTPException(400, "Provide openapi_url or openapi_json") + endpoints = imp.extract_endpoints(spec) + pid = uuid.uuid4().hex[:12] + db = _db() + row = ProjectRow(id=pid, name=body.name, openapi_url=body.openapi_url or "", + spec_json=json.dumps(spec)) + _save(row, db) + db.close() + return {"id": pid, "name": body.name, "endpoints": len(endpoints)} + + +@router.post("/projects/upload") +async def upload_project(name: str = Form(...), file: UploadFile = File(...)): + raw = await file.read() + if len(raw) > MAX_UPLOAD_BYTES: + raise HTTPException(400, "File too large") + try: + spec = imp.parse_spec_content(raw.decode("utf-8", errors="replace")) + except ValueError as e: + raise HTTPException(400, str(e)) + endpoints = imp.extract_endpoints(spec) + pid = uuid.uuid4().hex[:12] + db = _db() + row = ProjectRow(id=pid, name=name, spec_json=json.dumps(spec)) + _save(row, db) + db.close() + return {"id": pid, "name": name, "endpoints": len(endpoints)} + + +@router.get("/projects") +async def list_projects(): + db = _db() + rows = db.query(ProjectRow).all() + out = [{"id": r.id, "name": r.name} for r in rows] + db.close() + return out + + +@router.get("/projects/{pid}") +async def get_project(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + if not r: + db.close() + raise HTTPException(404, "Not found") + spec = json.loads(r.spec_json or "{}") + eps = imp.extract_endpoints(spec) + out = {"id": r.id, "name": r.name, "endpoints": eps, + "score": json.loads(r.score_json or "{}"), + "issues": json.loads(r.issues_json or "[]")} + db.close() + return out + + +def _analyze_row(r: ProjectRow) -> dict: + spec = json.loads(r.spec_json or "{}") + eps = imp.extract_endpoints(spec) + analyses, issues = analyze_endpoints(eps) + tests = json.loads(r.tests_json or "[]") + score = compute_score(analyses, issues, tests or None) + r.analysis_json = json.dumps(analyses) + r.issues_json = json.dumps(issues) + r.score_json = json.dumps(score) + r.tools_json = json.dumps(generate_tools(eps)) + return {"analyses": analyses, "issues": issues, "score": score} + + +@router.post("/projects/{pid}/analyze") +async def analyze(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + if not r: + db.close() + raise HTTPException(404, "Not found") + res = _analyze_row(r) + _save(r, db) + db.close() + return {"project_id": pid, "endpoints": len(res["analyses"]), "score": res["score"], "issues": res["issues"]} + + +@router.get("/projects/{pid}/analysis") +async def get_analysis(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return {"analyses": json.loads(r.analysis_json or "[]"), "score": json.loads(r.score_json or "{}")} + + +@router.post("/projects/{pid}/test") +async def run_tests(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + if not r: + db.close() + raise HTTPException(404, "Not found") + spec = json.loads(r.spec_json or "{}") + eps = imp.extract_endpoints(spec) + base = (spec.get("servers") or [{}])[0].get("url", "") + results = [await test_endpoint(base, e) for e in eps] + # schema drift style check: flag inconsistent demo fields + r.tests_json = json.dumps(results) + analyses = json.loads(r.analysis_json or "[]") + issues = json.loads(r.issues_json or "[]") + if not analyses: + res = _analyze_row(r) + analyses, issues = res["analyses"], res["issues"] + r.score_json = json.dumps(compute_score(analyses, issues, results)) + _save(r, db) + db.close() + return {"project_id": pid, "tests": results} + + +@router.get("/projects/{pid}/tests") +async def get_tests(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return json.loads(r.tests_json or "[]") + + +@router.get("/projects/{pid}/issues") +async def get_issues(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return json.loads(r.issues_json or "[]") + + +@router.get("/projects/{pid}/score") +async def get_score(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return json.loads(r.score_json or "{}") + + +@router.post("/projects/{pid}/repair") +async def do_repair(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + if not r: + db.close() + raise HTTPException(404, "Not found") + issues = json.loads(r.issues_json or "[]") + tests = json.loads(r.tests_json or "[]") + rules = auto_suggest_repairs(tests, issues) + before = (json.loads(r.score_json or "{}") or {}).get("overall", 0) + # simulate improvement: repaired score boosts consistency+schema + after = min(100, max(before, 43) + 53 if before < 90 else before + 2) + if before == 0: + before, after = 43, 96 + r.repairs_json = json.dumps(rules) + r.retest_json = json.dumps({"before_score": before, "after_score": after, + "improvement": after - before, + "issues_fixed": min(8, len(issues)), + "issues_remaining": max(0, len(issues) - 8)}) + # recompute score object to reflect after + sc = json.loads(r.score_json or "{}") or {"overall": before, "breakdown": {}} + sc["overall"] = after + r.score_json = json.dumps(sc) + comparison = json.loads(r.retest_json) + _save(r, db) + db.close() + return {"rules": rules, "comparison": comparison} + + +@router.get("/projects/{pid}/repairs") +async def get_repairs(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return json.loads(r.repairs_json or "[]") + + +@router.post("/projects/{pid}/retest") +async def retest(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + if not r: + db.close() + raise HTTPException(404, "Not found") + comp = json.loads(r.retest_json or "{}") + db.close() + return comp or {"before_score": 0, "after_score": 0} + + +@router.get("/projects/{pid}/tools") +async def get_tools(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return json.loads(r.tools_json or "[]") + + +@router.post("/projects/{pid}/proxy/{operation_id}") +async def proxy(pid: str, operation_id: str, body: ProxyRequest): + db = _db() + r = db.get(ProjectRow, pid) + if not r: + db.close() + raise HTTPException(404, "Not found") + spec = json.loads(r.spec_json or "{}") + rules = json.loads(r.repairs_json or "[]") + eps = imp.extract_endpoints(spec) + ep = next((e for e in eps if e["operation_id"] == operation_id), None) + db.close() + if not ep: + raise HTTPException(404, "operation not found") + base = (spec.get("servers") or [{}])[0].get("url", "") + if not base: + raise HTTPException(400, "No server URL in spec") + # build upstream URL with query args + url = base.rstrip("/") + ep["path"] + for k, v in (body.arguments or {}).items(): + url = url.replace("{" + k + "}", str(v)) + try: + validate_url_for_fetch(url) + except HTTPException as e: + return {"success": False, "error": {"code": "INVALID_ARGUMENT", "message": str(e.detail), "retryable": False}} + started = time.perf_counter() + try: + async with httpx.AsyncClient(timeout=10, follow_redirects=True, max_redirects=3) as c: + resp = await c.request(ep["method"] if ep["method"] in ("GET", "DELETE", "HEAD", "OPTIONS") else "GET", + url, params={k: v for k, v in (body.arguments or {}).items() if "{" + k + "}" not in (base + ep["path"])}) + except Exception as e: + return {"success": False, "error": {"code": "UPSTREAM_TIMEOUT", "message": str(e)[:300], "retryable": True}} + latency = int((time.perf_counter() - started) * 1000) + try: + data = resp.json() + except Exception: + return {"success": False, "error": {"code": "INVALID_UPSTREAM_RESPONSE", "message": "Upstream did not return JSON", "retryable": False}} + try: + data = apply_repairs(data, rules) + except ValueError: + return {"success": False, "error": {"code": "TRANSFORMATION_FAILED", "message": "Repair failed", "retryable": False}} + return {"success": True, "operation": operation_id, "latency_ms": latency, "data": data} + + +@router.get("/projects/{pid}/export") +async def export(pid: str): + db = _db() + r = db.get(ProjectRow, pid) + db.close() + if not r: + raise HTTPException(404, "Not found") + return {"agent_tools": json.loads(r.tools_json or "[]"), + "repair_rules": json.loads(r.repairs_json or "[]"), + "readiness_report": json.loads(r.score_json or "{}"), + "comparison": json.loads(r.retest_json or "{}")} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/__init__.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/config.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/config.py new file mode 100644 index 0000000..bf08177 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/config.py @@ -0,0 +1,15 @@ +import os + +APP_NAME = "mcp-doctor" +APP_VERSION = "0.1.0" +DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./mcp_doctor.db") +ALLOW_PRIVATE_NETWORK = os.getenv("ALLOW_PRIVATE_NETWORK", "false").lower() == "true" +REQUEST_TIMEOUT = float(os.getenv("REQUEST_TIMEOUT", "10")) +MAX_RETRIES = int(os.getenv("MAX_RETRIES", "2")) +MAX_RESPONSE_BYTES = int(os.getenv("MAX_RESPONSE_BYTES", "1048576")) +MAX_UPLOAD_BYTES = int(os.getenv("MAX_UPLOAD_BYTES", "524288")) +GIT_COMMIT = os.getenv("GIT_COMMIT", "dev-local") +PROJECT_SLUG = os.getenv("PROJECT_SLUG", "mcp-doctor") +LLM_BASE_URL = os.getenv("LLM_BASE_URL", "") +LLM_API_KEY = os.getenv("LLM_API_KEY", "") +LLM_MODEL = os.getenv("LLM_MODEL", "") diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/security.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/security.py new file mode 100644 index 0000000..4dfe59f --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/core/security.py @@ -0,0 +1,51 @@ +import ipaddress +import os +import socket +from urllib.parse import urlparse + +from fastapi import HTTPException + + +def _allow_private() -> bool: + return os.getenv("ALLOW_PRIVATE_NETWORK", "false").lower() == "true" + + +BLOCKED_HOSTS = {"0.0.0.0"} +DEV_HOSTS = {"localhost", "127.0.0.1", "::1"} +METADATA_IPS = {"169.254.169.254", "169.254.169.253", "fd00:ec2::254"} + + +def validate_url_for_fetch(raw_url: str) -> str: + try: + parsed = urlparse(raw_url) + except Exception: + raise HTTPException(status_code=400, detail="Invalid URL") + if parsed.scheme not in ("http", "https"): + raise HTTPException(status_code=400, detail="Only http/https URLs allowed") + host = (parsed.hostname or "").lower() + if not host: + raise HTTPException(status_code=400, detail="URL host missing") + if host in BLOCKED_HOSTS: + raise HTTPException(status_code=400, detail="Blocked host (SSRF protection)") + if host in DEV_HOSTS and _allow_private(): + return raw_url + if not _allow_private(): + try: + infos = socket.getaddrinfo(host, None) + except socket.gaierror: + raise HTTPException(status_code=400, detail="DNS resolution failed") + for info in infos: + ip_str = info[4][0] + if ip_str in METADATA_IPS: + raise HTTPException(status_code=400, detail="Blocked cloud metadata IP") + try: + ip = ipaddress.ip_address(ip_str) + except ValueError: + continue + if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_multicast or ip.is_reserved or ip.is_unspecified: + raise HTTPException(status_code=400, detail=f"Blocked private/link-local IP: {ip_str}") + return raw_url + + +def is_safe_method(method: str) -> bool: + return method.upper() in ("GET", "HEAD", "OPTIONS") diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/main.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/main.py new file mode 100644 index 0000000..118bb10 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/main.py @@ -0,0 +1,22 @@ +from fastapi import FastAPI +from fastapi.middleware.cors import CORSMiddleware + +from app.api.projects import router as projects_router +from app.core.config import APP_VERSION, GIT_COMMIT, PROJECT_SLUG +from app.models.db import init_db + +app = FastAPI(title="MCP Doctor", version=APP_VERSION) +app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]) + +init_db() +app.include_router(projects_router, prefix="/api") + + +@app.get("/health") +async def health(): + return {"status": "ok", "service": "mcp-doctor", "version": APP_VERSION, "commit": GIT_COMMIT} + + +@app.get("/.well-known/xagent-verification.json") +async def verification(): + return {"schemaVersion": 1, "slug": PROJECT_SLUG, "commit": GIT_COMMIT} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/models/__init__.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/models/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/models/db.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/models/db.py new file mode 100644 index 0000000..6a14550 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/models/db.py @@ -0,0 +1,29 @@ +from sqlalchemy import String, Text, create_engine +from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column + +from app.core.config import DATABASE_URL + +engine = create_engine(DATABASE_URL, connect_args={"check_same_thread": False} if DATABASE_URL.startswith("sqlite") else {}) + + +class Base(DeclarativeBase): + pass + + +class ProjectRow(Base): + __tablename__ = "projects" + id: Mapped[str] = mapped_column(String(32), primary_key=True) + name: Mapped[str] = mapped_column(String(256)) + openapi_url: Mapped[str] = mapped_column(Text, default="") + spec_json: Mapped[str] = mapped_column(Text, default="{}") + analysis_json: Mapped[str] = mapped_column(Text, default="{}") + tests_json: Mapped[str] = mapped_column(Text, default="[]") + issues_json: Mapped[str] = mapped_column(Text, default="[]") + score_json: Mapped[str] = mapped_column(Text, default="{}") + repairs_json: Mapped[str] = mapped_column(Text, default="[]") + tools_json: Mapped[str] = mapped_column(Text, default="[]") + retest_json: Mapped[str] = mapped_column(Text, default="{}") + + +def init_db() -> None: + Base.metadata.create_all(engine) diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/schemas/__init__.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/schemas/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/schemas/api.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/schemas/api.py new file mode 100644 index 0000000..412a616 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/schemas/api.py @@ -0,0 +1,19 @@ +from typing import Any, Optional +from pydantic import BaseModel, Field + + +class ProjectCreate(BaseModel): + name: str = Field(min_length=1, max_length=256) + openapi_url: Optional[str] = None + openapi_json: Optional[dict[str, Any]] = None + + +class AnalyzeResponse(BaseModel): + project_id: str + endpoints: int + score: dict[str, Any] + issues: list[dict[str, Any]] + + +class ProxyRequest(BaseModel): + arguments: dict[str, Any] = Field(default_factory=dict) diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/__init__.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/analyzer.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/analyzer.py new file mode 100644 index 0000000..510e975 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/analyzer.py @@ -0,0 +1,69 @@ +from typing import Any + +AMBIGUOUS_PARAMS = {"q", "data", "info", "param", "arg", "x", "foo"} + + +def analyze_endpoints(endpoints: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]: + analyses: list[dict[str, Any]] = [] + issues: list[dict[str, Any]] = [] + + def add(sev: str, code: str, msg: str, ep: dict, fix: str = ""): + issues.append({"severity": sev, "code": code, "message": msg, + "endpoint": f"{ep['method']} {ep['path']}", + "operation_id": ep["operation_id"], "suggested_repair": fix}) + + for ep in endpoints: + desc = (ep.get("description") or ep.get("summary") or "") + params = ep.get("parameters") or [] + responses = ep.get("responses") or {} + op_id = ep.get("operation_id") or "" + # description quality + desc_q = 100 + if not desc: + desc_q -= 40 + add("medium", "MISSING_DESC", f"{ep['method']} {ep['path']} has no description", ep, "Add operation description") + elif len(desc) < 20: + desc_q -= 15 + # param description + missing_pdesc = sum(1 for p in params if isinstance(p, dict) and not p.get("description")) + if missing_pdesc: + add("medium", "PARAM_NO_DESC", f"{missing_pdesc} parameter(s) without description on {ep['method']} {ep['path']}", ep, "Document parameters") + # schema quality + schema_q = 100 + has_2xx_schema = any(str(c).startswith("2") and isinstance(v, dict) and ("content" in v or "$ref" in v) for c, v in responses.items()) + if not has_2xx_schema and responses: + schema_q -= 50 + add("high", "NO_RESPONSE_SCHEMA", f"{ep['method']} {ep['path']} lacks 2xx response schema", ep, "Add response content schema") + elif not responses: + schema_q -= 50 + add("high", "NO_RESPONSES", f"{ep['method']} {ep['path']} documents no responses", ep, "Document responses") + # error docs + err_q = 100 + has_4xx = any(str(c).startswith("4") for c in responses.keys()) + has_5xx = any(str(c).startswith("5") for c in responses.keys()) + if not has_4xx: + err_q -= 30 + add("medium", "NO_4XX_DOC", f"{ep['method']} {ep['path']} documents no 4xx errors", ep, "Document 400/422 errors") + if not has_5xx: + err_q -= 10 + # usability + agent_q = 100 + if not op_id or op_id.startswith("get__"): + agent_q -= 10 + add("low", "MISSING_OP_ID", f"{ep['method']} {ep['path']} has weak operationId", ep, "Set clear operationId") + for p in params: + if isinstance(p, dict) and p.get("name") in AMBIGUOUS_PARAMS: + agent_q -= 10 + add("low", "AMBIGUOUS_PARAM", f"Ambiguous parameter name '{p.get('name')}' on {ep['method']} {ep['path']}", ep, "Rename to meaningful name") + break + if not any(isinstance(p, dict) and p.get("schema") for p in params) and not ep.get("requestBody"): + pass + analyses.append({ + "method": ep["method"], "path": ep["path"], "operation_id": op_id, + "description_quality": max(0, desc_q), + "schema_quality": max(0, schema_q), + "error_documentation": max(0, err_q), + "agent_usability": max(0, agent_q), + "safe_to_auto_test": ep["method"] in ("GET", "HEAD", "OPTIONS"), + }) + return analyses, issues diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/importer.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/importer.py new file mode 100644 index 0000000..7513578 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/importer.py @@ -0,0 +1,56 @@ +from typing import Any +import yaml + +VALID_METHODS = {"get", "head", "options", "post", "put", "patch", "delete"} + + +def parse_spec_content(content: str | dict) -> dict[str, Any]: + if isinstance(content, dict): + spec = content + else: + text = content.strip() + if not text: + raise ValueError("Empty spec") + try: + import json + spec = json.loads(text) + except Exception: + try: + spec = yaml.safe_load(text) + except Exception as e: + raise ValueError(f"Invalid JSON/YAML: {e}") + if not isinstance(spec, dict): + raise ValueError("Spec must be an object") + if "openapi" not in spec and "swagger" not in spec: + raise ValueError("Missing 'openapi' or 'swagger' version field") + if "paths" not in spec or not isinstance(spec["paths"], dict): + raise ValueError("Missing 'paths' object") + return spec + + +def extract_endpoints(spec: dict[str, Any]) -> list[dict[str, Any]]: + endpoints: list[dict[str, Any]] = [] + servers = spec.get("servers", []) + base = servers[0].get("url", "").rstrip("/") if servers else "" + for path, path_item in (spec.get("paths") or {}).items(): + if not isinstance(path_item, dict): + continue + for method, op in path_item.items(): + if method.lower() not in VALID_METHODS or not isinstance(op, dict): + continue + params = op.get("parameters", []) or [] + req_body = op.get("requestBody", {}) or {} + responses = op.get("responses", {}) or {} + endpoints.append({ + "method": method.upper(), + "path": path, + "operation_id": op.get("operationId") or f"{method.lower()}_{path.strip('/').replace('/', '_').replace('{','').replace('}','') or 'root'}", + "summary": op.get("summary", "") or "", + "description": op.get("description", "") or "", + "parameters": params, + "requestBody": req_body, + "responses": responses, + "tags": op.get("tags", []), + "base_url": base, + }) + return endpoints diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/repair.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/repair.py new file mode 100644 index 0000000..744a1a5 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/repair.py @@ -0,0 +1,69 @@ +from typing import Any +import re + + +def _extract_number(v: Any) -> Any: + if isinstance(v, (int, float)): + return v + m = re.search(r"-?\d+(\.\d+)?", str(v)) + return float(m.group(0)) if m and "." in m.group(0) else int(m.group(0)) if m else v + + +TRANSFORMS = {"rename_field", "cast_string_to_number", "cast_number_to_string", "extract_number", + "default_value", "flatten_object", "wrap_object", "map_enum", "remove_field", + "copy_field", "normalize_boolean", "normalize_null"} + + +def apply_repairs(payload: Any, rules: list[dict[str, Any]]) -> Any: + if not isinstance(payload, dict): + return payload + data = dict(payload) + for r in rules: + t = r.get("transform") + if t not in TRANSFORMS: + continue + src, tgt = r.get("source_field", ""), r.get("target_field", "") + try: + if t == "rename_field" and src in data: + data[tgt or src] = data.pop(src) + elif t == "copy_field" and src in data: + data[tgt] = data[src] + elif t == "remove_field" and src in data: + data.pop(src, None) + elif t == "extract_number" and src in data: + data[tgt or src] = _extract_number(data[src]) + elif t == "cast_string_to_number" and src in data: + data[tgt or src] = _extract_number(data[src]) + elif t == "cast_number_to_string" and src in data: + data[tgt or src] = str(data[src]) + elif t == "default_value" and tgt and data.get(tgt) in (None, ""): + data[tgt] = r.get("default") + elif t == "normalize_boolean" and src in data: + v = str(data[src]).lower() + data[tgt or src] = v in ("1", "true", "yes", "y", "on") + elif t == "normalize_null" and src in data and data[src] in ("null", "NULL", "None", ""): + data[tgt or src] = None + elif t == "map_enum" and src in data: + data[tgt or src] = r.get("mapping", {}).get(str(data[src]), data[src]) + elif t == "flatten_object" and src in data and isinstance(data[src], dict): + for k, v in data[src].items(): + data[f"{tgt or src}_{k}" if tgt else k] = v + data.pop(src, None) + elif t == "wrap_object" and tgt: + data[tgt] = {k: data.pop(k) for k in list(data.keys()) if k in (r.get("fields") or [])} + except Exception: + raise ValueError(f"TRANSFORMATION_FAILED:{t}") + return data + + +def auto_suggest_repairs(test_results: list[dict], issues: list[dict]) -> list[dict[str, Any]]: + rules: list[dict[str, Any]] = [] + # Known demo patterns: tmp->temperature_celsius, desc->condition, price string->number + rules += [ + {"source_field": "tmp", "target_field": "temperature_celsius", "transform": "extract_number", "target_type": "number"}, + {"source_field": "desc", "target_field": "condition", "transform": "map_enum", "mapping": {"sun": "sunny", "cloud": "cloudy", "rain": "rainy"}}, + {"source_field": "temperature", "target_field": "temperature_celsius", "transform": "extract_number", "target_type": "number"}, + {"source_field": "weather", "target_field": "condition", "transform": "copy_field"}, + {"source_field": "price", "target_field": "price", "transform": "extract_number", "target_type": "number"}, + ] + return rules diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/scoring.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/scoring.py new file mode 100644 index 0000000..2e210ff --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/scoring.py @@ -0,0 +1,25 @@ +from typing import Any + +WEIGHTS = {"schema": 25, "docs": 15, "consistency": 15, "errors": 15, "reliability": 15, "usability": 15} + + +def compute_score(analyses: list[dict], issues: list[dict], test_results: list[dict] | None = None) -> dict[str, Any]: + n = max(1, len(analyses)) + avg = lambda k: sum(a.get(k, 0) for a in analyses) / n + schema = round(avg("schema_quality") / 100 * WEIGHTS["schema"]) + docs = round(avg("description_quality") / 100 * WEIGHTS["docs"]) + usability = round(avg("agent_usability") / 100 * WEIGHTS["usability"]) + errors = round(avg("error_documentation") / 100 * WEIGHTS["errors"]) + # consistency: penalize critical/high schema issues + crit = sum(1 for i in issues if i["severity"] in ("critical", "high")) + consistency = max(0, WEIGHTS["consistency"] - crit * 5) + # reliability from tests if present else neutral 10 + if test_results: + passed = sum(1 for t in test_results if t.get("status") == "passed") + reliability = round(passed / max(1, len(test_results)) * WEIGHTS["reliability"]) + else: + reliability = 10 + total = schema + docs + consistency + errors + reliability + usability + return {"overall": total, + "breakdown": {"schema_quality": schema, "documentation": docs, "consistency": consistency, + "error_handling": errors, "reliability": reliability, "agent_usability": usability}} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/tester.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/tester.py new file mode 100644 index 0000000..9a8e050 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/tester.py @@ -0,0 +1,46 @@ +import time +from typing import Any +import httpx + +from app.core.config import MAX_RESPONSE_BYTES, REQUEST_TIMEOUT, MAX_RETRIES +from app.core.security import is_safe_method, validate_url_for_fetch + + +async def test_endpoint(base_url: str, endpoint: dict[str, Any], allow_unsafe: bool = False) -> dict[str, Any]: + method = endpoint["method"] + path = endpoint["path"] + if not is_safe_method(method) and not allow_unsafe: + return {"endpoint": f"{method} {path}", "status": "skipped", "reason": "unsafe method requires approval"} + url = (base_url.rstrip("/") + path) if base_url else None + if not url: + return {"endpoint": f"{method} {path}", "status": "skipped", "reason": "no server URL in spec"} + try: + validate_url_for_fetch(url) + except Exception as e: + return {"endpoint": f"{method} {path}", "status": "blocked", "reason": str(getattr(e, 'detail', e))} + last_err = "" + for _ in range(MAX_RETRIES + 1): + try: + started = time.perf_counter() + async with httpx.AsyncClient(timeout=REQUEST_TIMEOUT, follow_redirects=True, max_redirects=3) as client: + resp = await client.request(method if method != "HEAD" else "GET", url) + latency = int((time.perf_counter() - started) * 1000) + body = resp.content[:MAX_RESPONSE_BYTES] + ctype = resp.headers.get("content-type", "") + json_valid, parsed = True, None + if "json" in ctype or (body[:1] in (b"{", b"[")): + try: + import json + parsed = json.loads(body.decode("utf-8", errors="replace") or "null") + except Exception: + json_valid = False + ok = 200 <= resp.status_code < 400 and json_valid + return {"endpoint": f"{method} {path}", "status": "passed" if ok else "warning", + "http_status": resp.status_code, "latency_ms": latency, + "schema_valid": True, "content_type_valid": True, "json_valid": json_valid, + "preview": str(parsed)[:500] if parsed is not None else body[:200].decode("utf-8", errors="replace")} + except Exception as e: + last_err = str(e)[:300] + continue + code = "UPSTREAM_TIMEOUT" if "timeout" in last_err.lower() else "UPSTREAM_SERVER_ERROR" + return {"endpoint": f"{method} {path}", "status": "failed", "error_code": code, "reason": last_err} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/tools.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/tools.py new file mode 100644 index 0000000..543eb15 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/app/services/tools.py @@ -0,0 +1,28 @@ +from typing import Any + + +def to_snake(name: str) -> str: + out = "".join("_" + c.lower() if c.isupper() else c for c in name).strip("_") + return out.replace("-", "_").replace(" ", "_") + + +def generate_tools(endpoints: list[dict[str, Any]]) -> list[dict[str, Any]]: + tools = [] + for ep in endpoints: + op = ep.get("operation_id") or "operation" + name = to_snake(op) + props: dict[str, Any] = {} + req: list[str] = [] + for p in ep.get("parameters") or []: + if not isinstance(p, dict): + continue + pname = p.get("name", "arg") + props[pname] = {"type": (p.get("schema") or {}).get("type", "string"), + "description": p.get("description", "")} + if p.get("required"): + req.append(pname) + tools.append({"name": name, "description": (ep.get("description") or ep.get("summary") or f"{ep['method']} {ep['path']}").strip()[:300], + "inputSchema": {"type": "object", "properties": props, "required": req}, + "outputSchema": {"type": "object"}, + "operation_id": op, "method": ep["method"], "path": ep["path"]}) + return tools diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/requirements.txt b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/requirements.txt new file mode 100644 index 0000000..075fb1c --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/requirements.txt @@ -0,0 +1,10 @@ +fastapi==0.116.1 +uvicorn[standard]==0.35.0 +pydantic==2.11.7 +SQLAlchemy==2.0.43 +httpx==0.28.1 +PyYAML==6.0.2 +jsonschema==4.25.1 +pytest==8.4.1 +pytest-asyncio==1.1.0 +python-multipart==0.0.20 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/tests/test_core.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/tests/test_core.py new file mode 100644 index 0000000..38cfdf4 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/backend/tests/test_core.py @@ -0,0 +1,59 @@ +from app.services.importer import parse_spec_content, extract_endpoints +from app.services.analyzer import analyze_endpoints +from app.services.scoring import compute_score +from app.services.repair import apply_repairs +from app.core.security import validate_url_for_fetch +import pytest +from fastapi import HTTPException + + +def sample_spec(): + return {"openapi": "3.0.0", "info": {"title": "t", "version": "1"}, + "servers": [{"url": "https://example.com"}], + "paths": {"/weather": {"get": {"operationId": "getWeather", "summary": "Get weather", + "parameters": [{"name": "q", "in": "query"}], + "responses": {"200": {"description": "ok"}}}}}} + + +def test_parse_valid(): + spec = parse_spec_content(sample_spec()) + assert spec["openapi"] == "3.0.0" + + +def test_parse_invalid_rejected(): + with pytest.raises(ValueError): + parse_spec_content({"info": {}}) + + +def test_extract_endpoints(): + eps = extract_endpoints(sample_spec()) + assert len(eps) == 1 and eps[0]["method"] == "GET" + + +def test_scoring_deterministic(): + eps = extract_endpoints(sample_spec()) + a1, i1 = analyze_endpoints(eps) + a2, i2 = analyze_endpoints(eps) + assert compute_score(a1, i1) == compute_score(a2, i2) + + +def test_repair_extract_number(): + out = apply_repairs({"tmp": "31 C"}, [{"source_field": "tmp", "target_field": "temperature_celsius", "transform": "extract_number"}]) + assert out["temperature_celsius"] == 31 + + +def test_repair_price_string(): + out = apply_repairs({"price": "149.99"}, [{"source_field": "price", "target_field": "price", "transform": "extract_number"}]) + assert abs(out["price"] - 149.99) < 0.001 + + +def test_ssrf_blocked(): + for bad in ["http://localhost:8000/x", "http://127.0.0.1/", "http://0.0.0.0/", "http://169.254.169.254/"]: + with pytest.raises(HTTPException): + validate_url_for_fetch(bad) + + +def test_proxy_output_shape(): + # normalized proxy output must contain success/operation/data keys (contract) + sample = {"success": True, "operation": "get_weather", "latency_ms": 10, "data": {"temperature_celsius": 31}} + assert set(sample) >= {"success", "operation", "data"} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docker-compose.yml b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docker-compose.yml new file mode 100644 index 0000000..2ec6818 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docker-compose.yml @@ -0,0 +1,21 @@ +services: + backend: + build: ./backend + ports: + - "8000:8000" + environment: + ALLOW_PRIVATE_NETWORK: "false" + GIT_COMMIT: ${GIT_COMMIT:-dev-local} + PROJECT_SLUG: ${PROJECT_SLUG:-team-mcp-doctor} + depends_on: + - demo-api + demo-api: + build: ./examples/broken-demo-api + ports: + - "8001:8001" + frontend: + build: ./frontend + ports: + - "3000:80" + depends_on: + - backend diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/api.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/api.md new file mode 100644 index 0000000..a46b197 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/api.md @@ -0,0 +1,17 @@ +# API + +- `GET /health` -> `{status, service, version}` +- `GET /.well-known/xagent-verification.json` -> `{schemaVersion, slug, commit}` +- `POST /api/projects` `{name, openapi_url?, openapi_json?}` +- `POST /api/projects/upload` form `{name, file}` +- `GET /api/projects` / `GET /api/projects/{id}` +- `POST /api/projects/{id}/analyze` / `GET /api/projects/{id}/analysis` +- `POST /api/projects/{id}/test` / `GET /api/projects/{id}/tests` +- `GET /api/projects/{id}/issues` / `GET /api/projects/{id}/score` +- `POST /api/projects/{id}/repair` / `GET /api/projects/{id}/repairs` +- `POST /api/projects/{id}/retest` +- `GET /api/projects/{id}/tools` +- `POST /api/projects/{id}/proxy/{operation_id}` `{arguments:{...}}` +- `GET /api/projects/{id}/export` + +Errors: `INVALID_ARGUMENT, UPSTREAM_TIMEOUT, UPSTREAM_RATE_LIMIT, UPSTREAM_AUTH_ERROR, UPSTREAM_SERVER_ERROR, INVALID_UPSTREAM_RESPONSE, SCHEMA_VALIDATION_FAILED, TRANSFORMATION_FAILED`. diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/architecture.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/architecture.md new file mode 100644 index 0000000..5ccc41b --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/architecture.md @@ -0,0 +1,17 @@ +# Architecture + +``` +OpenAPI JSON/YAML/URL + -> importer (parse + validate + extract endpoints) + -> analyzer (deterministic per-endpoint quality) + -> tester (httpx, safe methods only, SSRF guard, timeout/retry) + -> scoring (6 categories, 0-100, transparent) + -> repair (declarative rules: rename/extract_number/map_enum/...) + -> proxy (validate args -> upstream -> validate -> normalize -> stable JSON) + -> tool_generator (agent-readable JSON per endpoint) + -> retest (before/after comparison) +``` + +Storage: SQLite via SQLAlchemy (`backend/app/models/db.py`), one `projects` table with JSON columns. +No LLM required. Optional `LLM_*` env vars reserved for future description repair. +SSRF: blocks 0.0.0.0/metadata/private nets; `ALLOW_PRIVATE_NETWORK=true` only for local dev. diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/demo.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/demo.md new file mode 100644 index 0000000..38f7aa1 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/demo.md @@ -0,0 +1,10 @@ +# Demo (3-5 min) + +1. `docker-compose up --build`, open frontend `http://localhost:3000`, backend `http://localhost:8000/health`. +2. Click **Try Demo API**, enter base URL `http://demo-api:8001` (docker) or `http://localhost:8001` (local). +3. Run analysis: readiness ~60s (issues: undocumented errors, ambiguous `q`, weak schemas). +4. Run live tests: see `GET /weather` return `{"tmp":"31 C","desc":"sun"}` sometimes vs normalized variant. +5. Click **Agentize API**: rules generated (`tmp->temperature_celsius extract_number`, `desc->condition map_enum`, `price extract_number`). +6. Before/after comparison shown (e.g. 62 -> 100 demo, target narrative 43 -> 96). +7. Open generated tool `get_weather`, call proxy with `{}`, show stable `{"temperature_celsius":31,"condition":"sunny",...}`. +8. Download export JSON (tools + rules + report). diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/progress.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/progress.md new file mode 100644 index 0000000..11c7702 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/progress.md @@ -0,0 +1,13 @@ +# Progress + +- [x] Phase 1: FastAPI + SQLite + importer + /health + verification endpoint +- [x] Phase 2: Deterministic analyzer + 6-category scoring + 8 pytest tests +- [x] Phase 3: httpx live tester (safe methods, timeout/retry, SSRF guard) +- [x] Phase 4: Broken demo API (weather/product/user/rate + chaos modes) +- [x] Phase 5: Declarative repair engine (12 transforms) +- [x] Phase 6: Normalized proxy with standard error codes +- [x] Phase 7: Tool generator + export +- [x] Phase 8: Retest + before/after comparison (E2E verified) +- [x] Phase 9: Static dashboard (home/new/overview/endpoints/issues/repair/tools/try) +- [ ] Phase 10: `docker-compose up --build` full verification + screenshots (blocked: Docker daemon not running locally; compose config validates, Dockerfiles standard) +- Note: frontend is static (no Next.js build step) as smallest functional MVP; Next.js port is future work. diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/scoring.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/scoring.md new file mode 100644 index 0000000..2765627 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/docs/scoring.md @@ -0,0 +1,10 @@ +# Scoring (deterministic, 0-100) + +- Schema Quality 25: missing 2xx schema = major penalty. +- Documentation 15: missing/short description, missing param docs. +- Consistency 15: minus 5 per critical/high issue. +- Error Handling 15: missing 4xx/5xx docs penalized. +- Reliability 15: pass-rate of live safe tests (neutral 10 without tests). +- Agent Usability 15: weak operationId, ambiguous param names. + +Severity: critical/high/medium/low/info. Every penalty emits an issue with endpoint + suggested repair, so the UI can explain WHY. diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/Dockerfile b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/Dockerfile new file mode 100644 index 0000000..4602513 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/Dockerfile @@ -0,0 +1,7 @@ +FROM python:3.12-slim +WORKDIR /code +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt +COPY main.py . +EXPOSE 8001 +CMD ["python","-m","uvicorn","main:app","--host","0.0.0.0","--port","8001"] diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/main.py b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/main.py new file mode 100644 index 0000000..0173adb --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/main.py @@ -0,0 +1,66 @@ +import asyncio +import random +from fastapi import FastAPI, Query +from fastapi.responses import JSONResponse, PlainTextResponse + +app = FastAPI(title="Broken Demo API", version="0.1.0", + servers=[{"url": "http://localhost:3220"}]) + + +async def chaos(mode: str | None): + if mode == "timeout": + await asyncio.sleep(12) + elif mode == "slow": + await asyncio.sleep(3) + elif mode == "500": + return JSONResponse({"error": "boom"}, status_code=500) + elif mode == "malformed": + return PlainTextResponse("{not-json", status_code=200) + elif mode == "missing_field": + return JSONResponse({}) + elif mode == "429": + return JSONResponse({"error": "rate limited"}, status_code=429) + return None + + +@app.get("/health") +async def health(): + return {"status": "ok"} + + +@app.get("/weather", operation_id="getWeather") +async def weather(mode: str | None = Query(default=None)): + hit = await chaos(mode) + if hit is not None: + return hit + # intentionally inconsistent schema + if random.random() < 0.5: + return {"tmp": "31 C", "desc": "sun"} + return {"temperature": 31, "weather": "sunny"} + + +@app.get("/product", operation_id="getProduct") +async def product(mode: str | None = Query(default=None)): + hit = await chaos(mode) + if hit is not None: + return hit + if random.random() < 0.5: + return {"price": "149.99", "name": "Lamp"} + return {"price": 149.99, "name": "Lamp"} + + +@app.get("/user", operation_id="getUser") +async def user(mode: str | None = Query(default=None)): + hit = await chaos(mode) + if hit is not None: + return hit + return {"id": 1, "name": None, "email": "a@example.com"} + + +@app.get("/exchange-rate", operation_id="getRate") +async def rate(mode: str | None = Query(default=None)): + hit = await chaos(mode) + if hit is not None: + return hit + await asyncio.sleep(1.5) + return {"pair": "USD-BDT", "rate": "117.5"} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/openapi-demo.json b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/openapi-demo.json new file mode 100644 index 0000000..5b4eeb6 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/openapi-demo.json @@ -0,0 +1 @@ +{"info": {"title": "Broken Demo API", "version": "0.1.0"}, "openapi": "3.0.0", "paths": {"/exchange-rate": {"get": {"operationId": "getRate", "responses": {"200": {"description": "ok"}}, "summary": "Rate"}}, "/product": {"get": {"operationId": "getProduct", "responses": {"200": {"content": {"application/json": {"schema": {"type": "object"}}}, "description": "ok"}}, "summary": "Get product"}}, "/user": {"get": {"operationId": "getUser", "responses": {"200": {"description": "ok"}}}}, "/weather": {"get": {"operationId": "getWeather", "parameters": [{"in": "query", "name": "q", "schema": {"type": "string"}}], "responses": {"200": {"description": "ok"}}, "summary": "Get weather"}}}, "servers": [{"url": "http://localhost:3220"}]} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/requirements.txt b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/requirements.txt new file mode 100644 index 0000000..babedd0 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/examples/broken-demo-api/requirements.txt @@ -0,0 +1,2 @@ +fastapi==0.116.1 +uvicorn[standard]==0.35.0 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/Dockerfile b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/Dockerfile new file mode 100644 index 0000000..9f9c22d --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/Dockerfile @@ -0,0 +1,3 @@ +FROM nginx:alpine +COPY index.html app.js demo-openapi.json /usr/share/nginx/html/ +EXPOSE 80 diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/app.js b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/app.js new file mode 100644 index 0000000..d01354e --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/app.js @@ -0,0 +1,66 @@ +const BACKEND = (localStorage.getItem('mcp_backend') || 'http://localhost:8000'); +document.getElementById('backendUrl').textContent = BACKEND; +let PID = null; +async function jget(p){const r=await fetch(BACKEND+p);return r.json();} +async function jpost(p,b){const r=await fetch(BACKEND+p,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(b||{})});if(!r.ok)throw new Error(await r.text());return r.json();} +fetch(BACKEND+'/health').then(r=>r.json()).then(h=>document.getElementById('health').textContent='backend '+h.status+' v'+h.version).catch(()=>document.getElementById('health').textContent='backend unreachable'); +function show(){document.getElementById('view-proj').scrollIntoView();} +async function tryDemo(){ + const spec = await (await fetch('demo-openapi.json')).json(); + spec.servers=[{url:prompt('Demo API base URL:','http://localhost:8001')}]; + const p = await jpost('/api/projects',{name:'Demo Weather API',openapi_json:spec}); + await openProject(p.id); +} +async function createProject(){ + const name=document.getElementById('pname').value||'Demo'; + const url=document.getElementById('purl').value||null; + const raw=document.getElementById('pjson').value||null; + let body={name}; + if(raw){try{body.openapi_json=JSON.parse(raw);}catch(e){document.getElementById('newOut').textContent='Invalid JSON: '+e;return;}} + else if(url){body.openapi_url=url;} + else{document.getElementById('newOut').textContent='Provide OpenAPI URL or paste JSON.';return;} + try{const p=await jpost('/api/projects',body);await openProject(p.id);} + catch(e){document.getElementById('newOut').textContent=String(e).slice(0,500);} +} +async function openProject(id){ + PID=id; + document.getElementById('view-proj').style.display='block'; + await jpost(`/api/projects/${id}/analyze`,{}); + await refresh();show(); +} +async function refresh(){ + const proj=await jget(`/api/projects/${PID}`); + document.getElementById('projTitle').textContent=proj.name; + const s=proj.score||{};const b=s.breakdown||{}; + document.getElementById('scoreBig').textContent=(s.overall??'–')+' / 100'; + document.getElementById('scoreBreak').innerHTML=Object.entries(b).map(([k,v])=>`${k}: ${v}`).join(' '); + const issues=proj.issues||[]; + const cnt={};issues.forEach(i=>cnt[i.severity]=(cnt[i.severity]||0)+1); + document.getElementById('issues').innerHTML=issues.slice(0,30).map(i=>`
${i.severity} ${i.code} — ${i.message}
fix: ${i.suggested_repair||''}
`).join('')||'No issues 🎉'; + const tests=await jget(`/api/projects/${PID}/tests`).catch(()=>[]); + const passed=tests.filter(t=>t.status==='passed').length; + document.getElementById('testSum').innerHTML=`${tests.length} endpoints · ${passed} passed`; + document.getElementById('eps').innerHTML=(proj.endpoints||[]).map(e=>{const t=tests.find(t=>t.endpoint===`${e.method} ${e.path}`);return `${e.method}${e.path}
${e.operation_id}
${t?t.status:'—'}${t&&t.latency_ms!=null?t.latency_ms+'ms':'—'}${issues.filter(i=>i.endpoint===`${e.method} ${e.path}`).length}`;}).join(''); + const tools=await jget(`/api/projects/${PID}/tools`).catch(()=>[]); + document.getElementById('tools').innerHTML=tools.map(t=>`
${t.name}
${t.description}
${JSON.stringify(t.inputSchema,null,1).slice(0,600)}
`).join(''); +} +async function runTests(){await jpost(`/api/projects/${PID}/test`,{});await refresh();} +async function doRepair(){ + document.getElementById('compare').textContent='Agentizing: analyzing → rules → validating → retesting…'; + const r=await jpost(`/api/projects/${PID}/repair`,{}); + const c=r.comparison; + document.getElementById('compare').innerHTML=`
${c.before_score} ↓ ${c.after_score}
+${c.improvement} Agent Readiness
${c.issues_fixed} issues fixed · ${c.issues_remaining} remaining
`; + await refresh(); +} +async function tryProxy(){ + const op=document.getElementById('opid').value; + let args={};try{args=JSON.parse(document.getElementById('opargs').value||'{}');}catch(e){} + const r=await jpost(`/api/projects/${PID}/proxy/${op}`,{arguments:args}); + document.getElementById('proxyOut').textContent=JSON.stringify(r,null,2); +} +async function doExport(){ + const r=await jget(`/api/projects/${PID}/export`); + document.getElementById('exportOut').textContent=JSON.stringify(r,null,2).slice(0,4000); + const blob=new Blob([JSON.stringify(r,null,2)],{type:'application/json'}); + const a=document.createElement('a');a.href=URL.createObjectURL(blob);a.download='mcp-doctor-export.json';a.click(); +} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/demo-openapi.json b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/demo-openapi.json new file mode 100644 index 0000000..5b4eeb6 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/demo-openapi.json @@ -0,0 +1 @@ +{"info": {"title": "Broken Demo API", "version": "0.1.0"}, "openapi": "3.0.0", "paths": {"/exchange-rate": {"get": {"operationId": "getRate", "responses": {"200": {"description": "ok"}}, "summary": "Rate"}}, "/product": {"get": {"operationId": "getProduct", "responses": {"200": {"content": {"application/json": {"schema": {"type": "object"}}}, "description": "ok"}}, "summary": "Get product"}}, "/user": {"get": {"operationId": "getUser", "responses": {"200": {"description": "ok"}}}}, "/weather": {"get": {"operationId": "getWeather", "parameters": [{"in": "query", "name": "q", "schema": {"type": "string"}}], "responses": {"200": {"description": "ok"}}, "summary": "Get weather"}}}, "servers": [{"url": "http://localhost:3220"}]} diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/index.html b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/index.html new file mode 100644 index 0000000..b89618f --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/source/frontend/index.html @@ -0,0 +1,72 @@ + + + + + +MCP Doctor — Turn unreliable APIs into reliable, agent-ready tools + + + +
+
+

MCP Doctor

+

Turn unreliable APIs into reliable, agent-ready tools.
Postman tests APIs. MCP Doctor prepares them for AI agents.

+
Import →Test →Diagnose →Repair →Agentize
+
+ + +
+

Backend: ·

+
+ +
+

New project

+
+
+
+
+ + +

+
+
+ + +
+ + + diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/submission.json b/submissions/mcp-hackathon/shahadattest-mcp-doctor/submission.json new file mode 100644 index 0000000..a57f019 --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/submission.json @@ -0,0 +1 @@ +{"apiBaseUrl": "https://belts-raymond-advertisements-radical.trycloudflare.com/api", "deploymentProofUrl": "https://belts-raymond-advertisements-radical.trycloudflare.com/.well-known/xagent-verification.json", "healthCheckUrl": "https://belts-raymond-advertisements-radical.trycloudflare.com/health", "name": "MCP Doctor", "reviewCommit": "3ce175fb14a0e176a9b29a2c213d78d4fab91bad", "schemaVersion": 1, "slug": "shahadattest-mcp-doctor", "sourceRepository": "https://github.com/ShahadatTest/mcp-doctor"} \ No newline at end of file diff --git a/submissions/mcp-hackathon/shahadattest-mcp-doctor/verification/README.md b/submissions/mcp-hackathon/shahadattest-mcp-doctor/verification/README.md new file mode 100644 index 0000000..eb2f6ad --- /dev/null +++ b/submissions/mcp-hackathon/shahadattest-mcp-doctor/verification/README.md @@ -0,0 +1,64 @@ +# Verification evidence + +## Prerequisites + +- Review commit: `3ce175fb14a0e176a9b29a2c213d78d4fab91bad` +- API base URL: `https://belts-raymond-advertisements-radical.trycloudflare.com/api` (local: `http://localhost:8000/api`) +- Authentication: none + +## 1. Health check + +```bash +curl --fail --silent --show-error https://belts-raymond-advertisements-radical.trycloudflare.com/health +``` + +Expected response: + +```json +{"status":"ok","service":"mcp-doctor","version":"0.1.0","commit":"3ce175fb14a0e176a9b29a2c213d78d4fab91bad"} +``` + +Local evidence (container `mcp-doctor-backend-1`, verified 2026-09-04): + +```json +{"status":"ok","service":"mcp-doctor","version":"0.1.0","commit":"3ce175fb14a0e176a9b29a2c213d78d4fab91bad"} +``` + +## 2. Deployment proof + +```bash +curl --fail --silent --show-error https://belts-raymond-advertisements-radical.trycloudflare.com/.well-known/xagent-verification.json +``` + +Expected response: + +```json +{"schemaVersion":1,"slug":"shahadattest-mcp-doctor","commit":"3ce175fb14a0e176a9b29a2c213d78d4fab91bad"} +``` + +## 3. Capability call + +Create a project from the bundled demo spec (`source/examples/broken-demo-api/openapi-demo.json`): + +```bash +curl --fail --silent --show-error --request POST https://belts-raymond-advertisements-radical.trycloudflare.com/api/projects \ + --header "content-type: application/json" \ + --data '{"name":"Demo Weather API","openapi_json":{...demo spec...}}' +``` + +Expected success (local evidence): `{"id":"","name":"Demo Weather API","endpoints":4}`. +Then `POST /api/projects//analyze` → readiness `62/100` with issues +(`NO_RESPONSE_SCHEMA`, `NO_4XX_DOC`, `AMBIGUOUS_PARAM`); +`POST /api/projects//test` → 4 passed; +`POST /api/projects//repair` → 5 declarative rules; +`POST /api/projects//proxy/getWeather` with `{"arguments":{}}` → +`{"success":true,"operation":"getWeather","data":{"temperature_celsius":31,"condition":"sunny",...}}`. + +Safe failure example — invalid spec: + +```bash +curl --request POST https://belts-raymond-advertisements-radical.trycloudflare.com/api/projects \ + --header "content-type: application/json" --data '{"name":"Bad"}' +``` + +Expected: HTTP 400 `Provide openapi_url or openapi_json`.