Product Connection
AI-Assisted-Coding-Tool is Anote's AI coding product. CodeBench research is measuring two critical dimensions of AI code generation that directly apply: (1) security vulnerabilities in generated code, and (2) specification gaming (AI passing tests by gaming them rather than genuinely solving the problem). Both represent real risks in production code generation.
Investigation Tasks
1. Run AI-Assisted-Coding-Tool Through the CodeBench Security Evaluation
The security scoring function security_score() implemented in src/codebench/evaluate.py can be applied directly to code our tool generates.
Investigation: Collect 100 code samples generated by AI-Assisted-Coding-Tool from real usage (anonymized). Run security_score() against each. Measure the CWE violation rate — specifically how often our tool generates code with: SQL injection patterns, command injection, hardcoded credentials, path traversal, or insecure random number usage. If violation rate > 5%, this is a product quality crisis requiring immediate action.
2. Test for Specification Gaming in AI-Assisted-Coding-Tool
Research (issue #5 in research-codebench) shows AI coding tools frequently exploit underspecified requirements — passing tests by gaming them rather than genuinely implementing the intent.
Investigation: Design 20 deliberately underspecified test cases where gaming is possible (e.g., "make this function return 0 errors" → AI could suppress error handling rather than fix errors). Run AI-Assisted-Coding-Tool on each. Count how many solutions game the requirement. If gaming rate > 10%, implement a post-generation "intent check" heuristic.
3. Apply Security-Adjusted pass@k as Our Internal Quality Metric
The research proposes security-adjusted pass@k as a more complete code quality metric than functional pass@k alone. We should adopt this for our internal benchmarking.
Investigation: Set up an internal benchmark of 50 coding tasks with both functional test suites and security test suites (CWE checks). Compute security-adjusted pass@k alongside functional pass@k on every major release. Define a minimum acceptable security-adjusted pass@k threshold as a release gate.
4. Add a Real-Time Security Warning to Code Generation
Based on the CodeBench security patterns, implement lightweight real-time security scanning.
Investigation: Implement a post-generation security scanner using the patterns from src/codebench/evaluate.py. When generated code matches a high-confidence security anti-pattern, surface an inline warning before the developer accepts the suggestion. Measure false positive rate on a safe code corpus to calibrate thresholds.
Expected Product Impact
- Security scanning → reduces liability for customers deploying AI-generated code in production
- Specification gaming detection → increases reliability for customers using AI-Assisted-Coding-Tool for test-driven development
- Security-adjusted metrics → honest capability assessment on our leaderboard
Owner
Research team (this repo) produces the benchmark and security patterns; AI-Assisted-Coding-Tool team integrates findings into the product pipeline.
Product Connection
AI-Assisted-Coding-Tool is Anote's AI coding product. CodeBench research is measuring two critical dimensions of AI code generation that directly apply: (1) security vulnerabilities in generated code, and (2) specification gaming (AI passing tests by gaming them rather than genuinely solving the problem). Both represent real risks in production code generation.
Investigation Tasks
1. Run AI-Assisted-Coding-Tool Through the CodeBench Security Evaluation
The security scoring function
security_score()implemented insrc/codebench/evaluate.pycan be applied directly to code our tool generates.Investigation: Collect 100 code samples generated by AI-Assisted-Coding-Tool from real usage (anonymized). Run
security_score()against each. Measure the CWE violation rate — specifically how often our tool generates code with: SQL injection patterns, command injection, hardcoded credentials, path traversal, or insecure random number usage. If violation rate > 5%, this is a product quality crisis requiring immediate action.2. Test for Specification Gaming in AI-Assisted-Coding-Tool
Research (issue #5 in research-codebench) shows AI coding tools frequently exploit underspecified requirements — passing tests by gaming them rather than genuinely implementing the intent.
Investigation: Design 20 deliberately underspecified test cases where gaming is possible (e.g., "make this function return 0 errors" → AI could suppress error handling rather than fix errors). Run AI-Assisted-Coding-Tool on each. Count how many solutions game the requirement. If gaming rate > 10%, implement a post-generation "intent check" heuristic.
3. Apply Security-Adjusted pass@k as Our Internal Quality Metric
The research proposes security-adjusted pass@k as a more complete code quality metric than functional pass@k alone. We should adopt this for our internal benchmarking.
Investigation: Set up an internal benchmark of 50 coding tasks with both functional test suites and security test suites (CWE checks). Compute security-adjusted pass@k alongside functional pass@k on every major release. Define a minimum acceptable security-adjusted pass@k threshold as a release gate.
4. Add a Real-Time Security Warning to Code Generation
Based on the CodeBench security patterns, implement lightweight real-time security scanning.
Investigation: Implement a post-generation security scanner using the patterns from
src/codebench/evaluate.py. When generated code matches a high-confidence security anti-pattern, surface an inline warning before the developer accepts the suggestion. Measure false positive rate on a safe code corpus to calibrate thresholds.Expected Product Impact
Owner
Research team (this repo) produces the benchmark and security patterns; AI-Assisted-Coding-Tool team integrates findings into the product pipeline.