security-benchmark

Adversarial security benchmark for agent authorization: does a compromised agent's policy-violating proposal become an unauthorized external effect? 73 trials, nine families, an independent oracle, per-mechanism ablation, confidence intervals. 0 unauthorized effects in 61 attack trials (95% CI [0.0%, 5.9%]). Reproduction is partial.

Open-source, 100% reproducible AI Agent Runtime Security Benchmark & Sandbox Environment (RFC-010 Draft Protocol).

Open, privacy-bounded assurance for AI agents: containment provenance, identity passports, authorization twins, OTel evidence, CI gates, and OSCAL.

Open-source benchmark for adversarial evidence attacks on LLM-based cybersecurity auditors, targeting ACM AsiaCCS 2027.

Open deterministic security tests for unsafe multi-agent handoffs and authority escalation.

Deterministic security benchmark for tool-using AI agents

Local-first workbench to run, inspect, compare, report, and gate OpenAI Codex Security scans.

Vendor-neutral benchmark measuring how MCP security proxies/gateways DEFEND against 22+ attack vectors — crosswalked to NIST AI RMF & OWASP LLM/Agentic Top 10. CI-gated, reproducible, DOI-cited. Submit your tool to the leaderboard.

Internal PyPI SCA precision and recall benchmark corpus

FreightSkillBench is a reproducible benchmark for evaluating document-to-transaction integrity, prompt-injection risk, and security controls in AI-enabled shipping and logistics workflows.

Production-grade microservices security benchmark featuring OWASP Top 10 logic exploits, automated remediation, custom Semgrep SAST rules, and CI/CD DevSecOps gates.

21 passive security evaluation inputs: conventional appsec and exploratory hardware, firmware, PLC, RTOS, database and policy domains. No scans started.

Open AI-for-security validation benchmark: non-LLM scorer + a SOTA-validation loop. Labeled positive corpus withheld pending coordinated disclosure.

Security evaluation input | Database-engine source, SQL parser, storage, transaction and extension boundaries | production-control-no-vulnerability-claim

The public adversarial evaluation suite for Signetry: measures attack success rate (ASR) and utility-under-defense for coding-agent prompt injection, skill/MCP poisoning, and memory-injection threats, governed by signetry-core.

Internal Go Modules SCA precision and recall benchmark corpus

Commit-pinned cal.diy source corpus; Vybscan ground-truth oracle available in benchmark-results

Security evaluation input | OpenMP C/C++/Fortran data races and race-free controls | labeled-positive-negative

Security evaluation input | Labeled Python injection, deserialization, crypto and trust-boundary cases | labeled-positive-negative

Security evaluation input | GraphQL authorization, queries, mutations, subscriptions, injection and query complexity and resource-exhaustion scenarios | intentional-vulnerabilities