Spec2Checker

Spec2Checker translates natural-language coding specifications into a unified intermediate representation (UIR), generates rules for Semgrep, CodeQL, and PMD, validates the generated rules, and scans source projects.

This is a code-only publication snapshot. Research datasets, generated rules, test corpora, experiment outputs, statistics, and figures are still being curated and are intentionally not included in this version.

Source layout

Path	Purpose
LLM_to_IR/	Natural-language specification to structured UIR conversion
Adapter/	UIR adapters for Semgrep, CodeQL, and PMD
LLM_to_XML/	PMD XML generation utilities
LLM_to_cases/	Test-case generation utilities
scan_project.py	Multi-analyser project scanner
validate_tool_syntax.py	Generated-rule validation
run_experiment.py	General experiment pipeline orchestration
run_iteration.py	Iterative rule-improvement orchestration
Spec2CheckerEngineeringProject/	Standard-library backend, frontend, and packaging code

Environment

• Python 3.10 or later
• Git
• Optional external analysers: Semgrep, CodeQL CLI, and PMD 6.55.0

Create an isolated environment and install Python dependencies:

python -m venv .venv

Windows PowerShell:

.\.venv\Scripts\Activate.ps1
python -m pip install -r requirements.txt

Linux or macOS:

source .venv/bin/activate
python -m pip install -r requirements.txt

Install Semgrep, CodeQL, and PMD separately and make their executables available on PATH. PMD may also be configured with PMD_HOME.

Model API configuration

Scripts that call an OpenAI-compatible model read credentials from environment variables. Never commit an API key.

Windows PowerShell:

$env:SPEC2CHECKER_API_KEY = "your-key"
$env:SPEC2CHECKER_MODEL = "your-model"
# Optional for a compatible API provider:
$env:SPEC2CHECKER_BASE_URL = "https://your-provider.example/v1"

Linux or macOS:

export SPEC2CHECKER_API_KEY="your-key"
export SPEC2CHECKER_MODEL="your-model"
# export SPEC2CHECKER_BASE_URL="https://your-provider.example/v1"

See .env.example for optional output-directory settings.

Usage

The main experiment and iteration entry points document their arguments:

python run_experiment.py --help
python run_iteration.py --help

Running the full pipeline requires separately prepared specification data, test projects, a configured model API, and the selected external analysers. The dataset and exact paper-evaluation package will be published in a later version after curation is complete.

The lightweight Web prototype can be started with:

cd Spec2CheckerEngineeringProject
python backend/server.py

Without the separately supplied data and generated-rule directories, the Web prototype starts as a code demonstration but has no paper dataset to load.

Security

• Configure credentials only through environment variables.
• The original research workspace is not part of this repository's history.
• Local databases, caches, logs, build packages, datasets, and generated reports are excluded by .gitignore.

Citation

For now, cite the accompanying paper and the versioned GitHub release. A DOI-backed archive can be created after the dataset and experiment artifacts have been finalized.

Licence

No open-source licence has been selected for this snapshot. The repository is publicly viewable, but reuse rights are not granted unless the authors add a licence.