Skip to content

Repository files navigation

KQL Detection Library

License: MIT Rules: 45 Hunting: 13 Tools: 12 GitHub Actions Release v1.0.0

Azure Sentinel / Microsoft 365 Detection Rules Library — curated KQL queries for threat detection, threat hunting, and security monitoring in Microsoft cloud environments.

Overview

Metric Value
Total Rules 45 detection rules + 13 hunting queries (current) — Roadmap complete ✅
Tactics Covered 12 of 14
Techniques Covered 45 of ~200
Tools 12 zero-dep CLI tools + GitHub Actions CI
Target Platform Microsoft Sentinel (Log Analytics / KQL)
Rule Format Standardized KQL with MITRE ATT&CK frontmatter

Quick Start

// Copy any rule directly into Sentinel Log Analytics Workspace
// or deploy as Scheduled Query Rule via ARM / CLI.

Prerequisites

  • Microsoft Sentinel workspace (or Azure Log Analytics)
  • Required log tables per rule (see each rule's Data Source header)
  • Minimum permissions: Microsoft.OperationalInsights/workspaces/query/read

Rule Index

Credential Access

Rule Technique Severity Data Source
Brute Force RDP T1110 High SigninLogs, AADNonInteractiveUserSignInLogs
Kerberoasting Detection T1558.003 High SecurityEvent, IdentityLogonEvents
DCSync Detection T1003.006 Critical DirectoryServiceAccess, BehaviorAnalytics
LSASS Memory Access T1003.001 Critical DeviceEvents, SecurityEvent (4663/4656)

Execution

Rule Technique Severity Data Source
PowerShell Obfuscation T1059.001 Medium DeviceProcessEvents (MDE)

Defense Evasion

Rule Technique Severity Data Source
AMSI Bypass T1562.001 High DeviceProcessEvents, DeviceRegistryEvents
Process Hollowing T1055.012 High DeviceProcessEvents (MDE)
Log Clearing T1070.001 High SecurityEvent (1102), DeviceProcessEvents
DLL Search Order Hijacking T1574.001 Medium/High DeviceImageLoadEvents (MDE)

Persistence

Rule Technique Severity Data Source
Scheduled Task Creation T1053.005 Medium SecurityEvent, DeviceEvents
Registry Run Key T1547.001 Medium/High DeviceRegistryEvents (MDE)
WMI Event Subscription T1546.003 Medium/High SecurityEvent (5861), DeviceEvents
Startup Folder T1547.001 Medium/High DeviceFileEvents (MDE)
BITS Jobs T1197 Medium/High DeviceProcessEvents (MDE)
Service Installation T1543.003 Medium/High SecurityEvent (7045/4697), DeviceProcessEvents

Lateral Movement

Rule Technique Severity Data Source
Pass-the-Hash T1550.002 High SecurityEvent (4624), DeviceLogonEvents
WMI Execution T1047 Medium/High DeviceProcessEvents (MDE)
PsExec / SMB Execution T1570 High SecurityEvent (7045/5145), DeviceProcessEvents
RDP Lateral Movement T1021.001 Medium SecurityEvent (4624), DeviceLogonEvents
Remote Service Creation T1543.003 High SecurityEvent (4697/7045), DeviceProcessEvents
DCOM Lateral Movement T1021.003 High DeviceProcessEvents (MDE)
WinRM / PowerShell Remoting T1021.006 High DeviceProcessEvents (MDE)
SMB Named Pipe Impersonation T1550.003 High SecurityEvent (5145)

Discovery

Rule Technique Severity Data Source
Network Share Discovery T1135 Low/Medium DeviceProcessEvents (MDE)
Active Directory Discovery T1087.002 Medium/High DeviceProcessEvents (MDE)

Collection

Rule Technique Severity Data Source
Data Staging T1074 Medium/High DeviceProcessEvents (MDE)
Email Forwarding Exfiltration T1114.003 High OfficeActivity (Exchange)

Command and Control

Rule Technique Severity Data Source
DNS Tunneling T1572 High DnsEvents
Beaconing Pattern T1071.001 High DeviceNetworkEvents (MDE)
C2 over WebSocket T1071.001 High DeviceNetworkEvents (MDE)
C2 JA3/JA3S Fingerprint T1071.001 High DeviceNetworkEvents (TLS/JA3)
ICMP / Protocol Tunneling T1571 Medium/High DeviceNetworkEvents (MDE)

Exfiltration

Rule Technique Severity Data Source
Unusual Outbound Traffic T1048 High DeviceNetworkEvents (MDE)
Large File Upload T1030 Medium DeviceNetworkEvents (MDE)
Cloud Storage Exfiltration T1567.002 Medium/High DeviceNetworkEvents (MDE)

Cloud & Kubernetes

Rule Technique Severity Data Source
Azure AD MFA Bypass / Legacy Auth T1078.004 High SigninLogs
OAuth Consent Grant T1528 High AuditLogs (Azure AD)
Service Principal Abuse T1098 High AuditLogs, AADServicePrincipalSignInLogs
Kubernetes Container Escape T1611 High AzureDiagnostics (kube-audit)
Key Vault Access Anomaly T1552.005 High AzureDiagnostics (Key Vault)
Kubernetes RBAC Abuse T1087.004 High AzureDiagnostics (kube-audit)
Azure VM Run Command Abuse T1059.009 High AzureActivity
Blob Storage Public Access T1530 High StorageBlobLogs
Azure Resource Deletion T1485 High AzureActivity
Logic App / Automation Abuse T1053.006 Medium/High AzureActivity

Hunting Queries

Exploratory, analyst-driven queries in hunting-queries/. Unlike detection rules (which alert), these support proactive threat hunting and use a hunt-specific header format (Hunt Hypothesis, Investigation Steps, Pivots).

Query Technique Focus Data Source
User Behavior Baseline T1078 Per-user deviation (hours/geo/devices) SigninLogs
Unmanaged Device Access T1078.004 Non-compliant device → sensitive apps SigninLogs
Anomalous Logon Hour T1078 First-ever logon at odd hour SigninLogs, SecurityEvent
Cross-Tenant Access Anomalies T1078.004 B2B guest abuse SigninLogs, AuditLogs
Service Account Interactive Logon T1078 Svc accounts w/ interactive logon SecurityEvent (4624)
Pass-the-Hash Hunting T1550.002 NTLM type-3 fan-out SecurityEvent (4624)
Golden Ticket Hunting T1558.001 TGS without TGT, RC4 tickets SecurityEvent (4768/4769)
Mailbox Access Anomalies T1114 Non-owner mailbox access OfficeActivity
Identity Protection Insights T1078 Aggregated AAD risk detections AADUserRiskEvents
Data Staging → Exfil Correlation T1074 / T1048 Archive-then-egress chain DeviceProcessEvents, DeviceNetworkEvents

Rule Format

Every rule follows this structure:

// === MITRE ATT&CK ===
// Technique: TXXXX.XXX — Description
// Tactic: Tactic Name
// Severity: High / Medium / Low / Critical
// Data Source: TableName (Source)
// False Positives: Known noise sources
// Recommended Response: Step-by-step IR guidance
// === QUERY ===
<kql_query>

Testing

See mapping/test-cases/ for sample data + expected detection logic:

python tools/rule-validator.py azure-sentinel/
python tools/rule-validator.py hunting-queries/

Each test case simulates the relevant table schema with realistic event data and validates query syntax. The validator supports two file types: detection rules (6 MITRE headers) and hunting queries (6 hunt headers: Hunt Hypothesis / Investigation Steps / Pivots). The test — not a live Sentinel deployment — catches structural errors and schema mismatches before deployment.

MITRE ATT&CK Coverage

See ATTACK_MATRIX.md for full tactic/technique mapping. This file is auto-generated — run python tools/coverage-report.py --write after adding rules.

Tooling

All tools are zero-dependency (Python stdlib only) and live in tools/.

Tool Purpose
rule-validator.py Validate rules & hunting queries (headers, syntax, naming). CI gate.
rule-scaffold.py Generate a new rule or hunt from template + matching test case.
coverage-report.py Auto-generate ATTACK_MATRIX.md from rule headers (--write / --check).
rule-dependency-checker.py Verify every rule has a test case, yaml entry, and matrix reference.
fp-report.py False-positive hardening score per rule (FP section + response + benign test row).
arm-exporter.py Export all rules as a Sentinel ARM template (Microsoft.SecurityInsights/alertRules).
terraform-exporter.py Export all rules as Terraform (azurerm_sentinel_alert_rule_scheduled).
workbook-generator.py Generate Sentinel Workbook JSON (rule inventory + pie chart).
navigator-export.py Export rules as ATT&CK Navigator heatmap layer JSON.
kql-test-framework.py Test case validation + expected-result checking.
sigma-to-kql.py Convert Sigma YAML rules to KQL.
test-simulator.py Generate synthetic datatable test fixtures from rule headers.
# Add a new rule end-to-end
python tools/rule-scaffold.py rule --name my-rule --category execution \
    --technique T1059 --tactic Execution --severity High --data-source "DeviceProcessEvents (MDE)"
# ... fill in the TODOs ...
python tools/rule-validator.py azure-sentinel/
python tools/rule-dependency-checker.py
python tools/coverage-report.py --write          # refresh the ATT&CK matrix

# Deploy to Sentinel
python tools/arm-exporter.py --out deploy/sentinel-rules.json
python tools/terraform-exporter.py --out deploy/sentinel.tf

CI (.github/workflows/validate.yml) runs the validator, dependency checker, and coverage --check on every push and PR.

Roadmap

v0.1 — MVP (Current)

  • Brute Force RDP Detection (T1110)
  • Kerberoasting Detection (T1558.003)
  • DCSync Detection (T1003.006)
  • PowerShell Obfuscation Detection (T1059.001)
  • Scheduled Task Persistence (T1053.005)
  • Rule validator tool
  • Test harness per rule
  • MITRE ATT&CK mapping

v0.2 — Execution & Defense Evasion (+10 rules) ✅ COMPLETE

  • LSASS Access / Mimikatz Detection (T1003.001) — ProcessAccess via lsass.exe, Event 4663
  • AMSI Bypass Detection (T1562.001) — AmsiScanBuffer patching, registry disable
  • Process Hollowing Detection (T1055.012) — Unbacked memory + suspended thread in trusted binaries
  • Log Clearing Detection (T1070.001) — Event 1102 (Security log cleared), wevtutil /auditpol
  • Registry Run Key Persistence (T1547.001) — CurrentVersion\Run modifications (Event 4657)
  • WMI Event Subscription Persistence (T1546.003) — __EventFilter / __FilterToConsumerBinding (Event 5861)
  • Startup Folder Persistence (T1547.001) — LNK creation in StartUp folders
  • DLL Search Order Hijacking (T1574.001) — DLL loads from user-writable paths
  • BITS Jobs Persistence (T1197) — Background Intelligent Transfer Service job creation
  • Service Installation Detection (T1543.003) — New service creation (Event 4697) from non-admin

v0.3 — Lateral Movement & Discovery (+10 rules) ✅ COMPLETE

  • Pass-the-Hash Detection (T1550.002) — NTLM logon type 3 anomalies
  • Remote Service Creation (T1543.003) — sc.exe / PowerShell New-Service from remote
  • WMI Execution Detection (T1047) — wmic process call create, Invoke-CimMethod
  • PsExec / SMB Execution (T1570) — PSEXESVC service + named pipe \psexecsvc
  • Network Share Discovery (T1135) — net view / net share enumeration bursts
  • DCOM Lateral Movement (T1021.003) — MMC20.Application, ShellWindows, Excel DCOM
  • WinRM / PowerShell Remoting (T1021.006) — WinRM service access, PS session creation
  • RDP Lateral Movement (T1021.001) — Successful RDP logon from non-admin workstation
  • SMB Named Pipe Impersonation (T1550.003) — \pipe\* access after SMB session
  • Active Directory Discovery (T1087.002) — BloodHound / AD enumeration tools (adfind, sharphound)

v0.4 — Exfiltration & C2 (+10 rules) ✅ COMPLETE

  • Unusual Outbound Traffic (T1048) — Egress volume anomaly from non-web servers
  • DNS Tunneling Anomaly (T1572) — High-entropy subdomains, TXT query bursts
  • Beaconing Pattern Detection (T1071.001) — Periodic HTTP/HTTPS connection interval analysis
  • Large File Upload Anomaly (T1030) — File uploads > 100MB via web / cloud storage
  • Data Staging Detection (T1074) — Archive creation (zip/rar/7z) on sensitive shares
  • C2 Over WebSocket (T1071.001) — Long-lived WebSocket connections to rare domains
  • C2 Over HTTPS (JA3/S) — JA3 hash clustering for known C2 frameworks
  • Cloud Storage Exfiltration (T1567.002) — Files uploaded to personal cloud (Dropbox, GDrive)
  • Email Forwarding Exfiltration (T1114.003) — Auto-forwarding rule creation to external domain
  • ICMP / Custom Protocol Tunneling (T1571) — Non-DNS/HTTP outbound protocol anomalies

v0.5 — Cloud-Specific & Kubernetes (+10 rules) ✅ COMPLETE

  • Azure AD MFA Bypass / Legacy Auth (T1078.004) — Legacy protocol auth, no MFA conditional access
  • Suspicious OAuth Consent Grant (T1528) — Third-party app consent with high permissions
  • Anomalous Service Principal Usage (T1098) — New SPN / client secret outside business hours
  • Container Escape Detection (T1611) — privileged flag, host mount, hostPID/hostNetwork
  • Azure Key Vault Access Anomaly (T1552.005) — Bulk secret access / unexpected identity
  • Kubernetes RBAC Abuse (T1087.004) — ClusterRole escalation, secret enumeration via API
  • Azure VM Run Command Abuse (T1059.009) — RunCommand / custom script extension RCE
  • Blob Storage Public Access (T1530) — Anonymous blob enumeration, storage account misconfig
  • Azure Resource Deletion (T1485) — Bulk resource group / NSG deletion within minutes
  • Azure Logic App / Automation Account Abuse (T1053.006) — Suspicious runbook/job creation

v0.6 — Hunting & Baseline (+10 queries) ✅ COMPLETE

  • Threat Hunting Queries Module — hunting-queries/ directory (hunt-format + validator support)
  • User Behavior Baseline KQL — Historical logon hours, geolocation, device count
  • Unmanaged Device Access Detection — Device compliance check + Conditional Access bypass
  • Anomalous Logon Hour Analysis — First-time logon at 3 AM for a given user
  • Cross-Tenant Access Anomalies — B2B guest account enumeration
  • Service Account Interactive Logon — Service accounts should never have interactive sessions
  • Pass-the-Hash Hunting — RC4 NTLM logon across non-DC hosts
  • Golden Ticket Hunting — Kerberos ticket anomalies (Event 4768/4769, TGS without TGT, RC4)
  • Mailbox Access Anomalies — Non-owner mailbox access via EWS/Graph API
  • Identity Protection Insights — Azure AD Identity Protection risk detections aggregated
  • Data Staging → Exfil Correlation — archive creation followed by large egress (bonus)

v0.7 — Tooling & CI ✅ COMPLETE

  • rule-scaffold.py — Generate new rule/hunt from template with auto-frontmatter
  • coverage-report.py — Auto-generate ATT&CK matrix markdown from rules directory (--write / --check)
  • GitHub Actions CI — Run rule-validator.py + dependency + coverage check on every push + PR
  • ARM Template Exporter (arm-exporter.py) — Deploy rules as Sentinel Analytic Rules via ARM
  • Sentinel-as-code (Terraform) (terraform-exporter.py) — azurerm_sentinel_alert_rule_scheduled
  • Rule Dependency Checker (rule-dependency-checker.py) — Cross-ref rules ↔ test cases ↔ yaml ↔ matrix
  • False-Positive Coverage Report (fp-report.py) — FP-hardening score per rule

v0.8 — Advanced Features ✅ COMPLETE

  • Sentinel Workbook Generator (workbook-generator.py) — Auto-create Azure Workbook JSON from ruleset
  • Tier-2: Correlation Rules — Multi-event chain detection (LSASS → RDP → Scheduled Task chain)
  • Tier-3: ML / Fusion Anomalies — Z-score time-series anomaly detection (sign-in failure spike)
  • Cross-Workspace Hunting — KQL queries using workspace() across multiple Sentinel workspaces
  • Watchlist Integration — Dynamic allowlisting via Sentinel watchlists (docs + examples)
  • KQL Unit Test Framework (kql-test-framework.py) — Automated test-case validation + expected-result checking
  • ATT&CK Navigator Layer (navigator-export.py) — Export rules as ATT&CK Navigator heatmap JSON
  • Sigma → KQL Converter (sigma-to-kql.py) — Basic Sigma detection rule → KQL translation
  • Test Simulator (test-simulator.py) — Synthetic event generator (deferred from v0.7)

Deployment

Automated (ARM / Terraform)

# Generate ARM template
make deploy-arm   # → deploy/sentinel-rules.json
# Generate Terraform config
make deploy-tf    # → deploy/sentinel.tf

# Deploy with Azure CLI:
az deployment group create -g rg-sentinel --template-file deploy/sentinel-rules.json
# or: terraform init && terraform apply -var 'log_analytics_workspace_id=<id>'

Manual

  1. Open Azure Portal → Sentinel → Analytics → Create → Scheduled query rule
  2. Copy the KQL from the desired rule file
  3. Set query frequency, period, threshold, entity mappings

Prerequisites: Data Connectors

See docs/data-connectors.md for the exact Sentinel connector each rule depends on. Without the correct connector enabled, a rule produces zero results.

Contributing

See CONTRIBUTING.md for the full workflow.

Quick start:

git clone https://github.com/rizko-d/kql-detection-library.git
cd kql-detection-library
make check    # run validator + depcheck + coverage

License

MIT — use freely, attribution appreciated.

Docs

Document Description
CONTRIBUTING.md How to add rules, hunts, and submit PRs
data-connectors.md Sentinel connector prerequisites per rule
watchlist-integration.kql Dynamic allowlisting integration guide
fp-coverage.md False-positive hardening coverage report
ATTACK_MATRIX.md Auto-generated MITRE ATT&CK coverage matrix

Releases

Packages

Contributors

Languages