[ FRONTIER MODEL EVALUATIONS ]
▲
│
┌────────────────────┴────────────────────┐
│ │
[ CONTAINER ISOLATION ] [ CLOUD AUTOMATION ]
• Docker cgroups (Strict 1GB RAM) • GitHub Actions Matrix Sharding
• Non-root runtime isolation • GCP Compute Engine & Cloud Run
• Deterministic OOM signal trap • Automated healthcheck rollbacks
│ │
└────────────────────┬────────────────────┘
│
▼
[ SYSTEMIC FAILURE TAXONOMY ]
(Wildcards, RBAC Drift)
Primary Languages & Systems Engineering
DevOps, Cloud, Sandboxing & Testing
AI, ML & Interpretability Frameworks
At Ethara.ai, I design deterministic evaluation harnesses to rigorously test frontier LLMs on enterprise cloud infrastructure tasks. Rather than relying on simple string-matching, our benchmarks run in isolated Linux containers with multi-axis rubric verification:
- 🎯 Deterministic Benchmarking & Task Authoring: Engineering reproducible task environments across Docker, Kubernetes, Helm, Terraform, and CI/CD with unit tests and golden solutions.
- 🐳 Hermetic Sandbox Execution: Isolating model-generated scripts inside ephemeral Docker runners with restricted network egress, non-root execution, and strict memory cgroups.
- 🚨 Systemic AI Mistake Taxonomy: Cataloging recurring model flaws—such as unpinned base image digests, excessive
*permissions in Kubernetes roles, and hardcoded cloud secrets. - ☁️ Cloud CI/CD & Infrastructure Automation: Architecting production deployment workflows with GitHub Actions, Google Cloud Platform (Compute Engine, GCR, Cloud Storage), and automated health-check rollback gates.
- 🧠 Explainable AI Research (DepreXAI): Co-authored clinical NLP research utilizing transformer ensembles (PyTorch, Hugging Face) paired with SHAP/LIME feature attribution (94.2% accuracy, 0.91 macro F1).
Pass rates and deterministic failure profiles observed across frontier models in our sandboxed test suites:
| Benchmark Domain | Target Models | Core Verification Standard | Caught Failure Modes | Pass Rate |
|---|---|---|---|---|
| Multi-Stage Dockerfiles | Claude 3.5 Sonnet, Codex, GLM-4 | Non-root user, minimal base image, multi-stage build | Default root user, cache invalidation, bloated layers | 78.4% |
| Kubernetes RBAC & NetPol | Claude 3.5 Sonnet, GLM-4, DeepSeek | Namespace isolation, explicit verbs, no cluster-admin | Wildcard * verbs, missing egress boundaries |
64.2% |
| Terraform GCP Infrastructure | Claude 3.5 Sonnet, Codex | Resource naming conventions, IAM least privilege | Hardcoded secrets, unencrypted bucket configs | 71.8% |
| Production Helm Charts | Codex, GLM-4, DeepSeek | Values schema validation, probe health configurations | Missing readiness probes, template linting errors | 62.0% |
| CI/CD Pipeline Security | Claude 3.5 Sonnet, Codex | GitHub Actions token permissions, step timeouts | Unpinned third-party actions, excessive GITHUB_TOKEN | 81.5% |
| Repository | Focus & Domain | Architecture & Stack | Verification Method |
|---|---|---|---|
devops-llm-benchmark |
DevOps LLM Benchmark Suite | Docker • Kubernetes • Terraform • CI/CD | Docker Sandboxing + Pytest |
cloud-cicd-gcp-pipeline |
Cloud CI/CD Automation | GitHub Actions • Docker • GCP | GCP Deployment + Live Healthchecks |
playwright-qa-automation |
Enterprise E2E Test Suite | Playwright • Python • Allure Reports | Parallel Matrix Shards (4x Speed) |
deprexai |
Explainable Clinical NLP | PyTorch • Transformers • SHAP • LIME | Cross-Validation (94.2% Acc, 0.91 F1) |
Automation-Testing-tool |
QA Test Automation Framework | Python • Playwright • AstraQ | Deterministic Selector Assertions |
taskflow-rbac-app |
Full-Stack Enterprise RBAC | React • Express • Prisma • SQLite | Automated Route Bypass & Audit Tests |
Verified milestones awarded for open-source contributions, technical discussions, and code reviews.
| Badge | Distinction | Verification Metric | Status |
|---|---|---|---|
| 🧠 Galaxy Brain | Bronze (x1) | 2+ accepted answers to technical Q&A discussions | UNLOCKED |
| 🪂 YOLO | Unlocked | Merged pull request without blocking review requirements | UNLOCKED |
| 🦈 Pull Shark | Silver (x2) | 16+ merged pull requests into default branches | UNLOCKED |
| ⚡ Quickdraw | Unlocked | Issue/PR closed within 5 minutes of opening | UNLOCKED |
| 👥 Pair Extraordinaire | Bronze (x1) | Merged co-authored commits via pull requests | UNLOCKED |
role: "DevOps Engineer & AI Evaluation Specialist"
current_company: "Ethara.ai"
domains:
- "Frontier Model Benchmarking (Claude 3.5 Sonnet, Codex, GLM-4, DeepSeek)"
- "Deterministic Cloud Sandboxing & Ephemeral Test Containers"
- "Automated CI/CD Pipelines & Kubernetes Security Verification"
contact:
email: "peeyushrajput321@gmail.com"
linkedin: "https://linkedin.com/in/piyush-chauhan-8a02a0251"
github: "https://github.com/pxyushch"
status: "Available for high-impact AI Evaluation & DevOps opportunities"


