Jailbreak LLMs with Linguistic Style as a Hidden Attack Surface

We show that linguistic style is a systematic and previously overlooked attack surface for LLM safety. Across a diverse taxonomy of styles and multiple state-of-the-art models, stylistic rewrites of identical harmful goals produce order-of-magnitude differences in Attack Success Rate (ASR), with some styles achieving up to 80% ASR while others remain below 6%. To exploit this vulnerability, we introduce a lightweight single-pass attack framework that combines a style-conditioned generator with a BERT-based style selector, achieving performance competitive with substantially more expensive multi-turn optimization methods. To understand why style affects safety behavior, we conduct mechanistic analysis on LLaMA-3.1-8B and find that stylistic rewrites disrupt refusal representations while preserving recoverable harmful intent, suggesting a distributed failure mode rather than a single refusal mechanism. Motivated by this finding, we develop a style-aware Direct Preference Optimization defense that reduces ASR from 86% to 19.5% while largely preserving utility.

---

Overview

This repository provides the official implementation for our paper. We make three contributions:

1. Systematic characterization of stylistic jailbreaks. We provide a systematic study of linguistic style as a safety-relevant factor, showing that stylistic rewrites of the same harmful request lead to substantial differences in jailbreak success across models and datasets.

2. Scalable style-based red-teaming. We introduce a lightweight attack framework that automatically selects effective stylistic transformations and achieves competitive performance under realistic query budgets.

3. Mechanistic explanation and mitigation. Through interpretability analysis we show that stylistic jailbreaks partially suppress the refusal representation while leaving harmful intent recoverable and ruling out attention redirection and linear steering, suggesting a distributed mechanism. We further show that style-aware DPO substantially improves robustness while preserving model capabilities.

---

Key Results

HarmBench ASR (%) for Original Prompts and Style-Conditioned Instantiations (Table 1)

Target Model	Original	Gemini 2.5	LLaMA-3.1-8B	Style-LLaMA-3.1-8B
LLaMA-3.1-8B	8	88	68.5	86
Gemma-2-2B	1	75.5	48.5	69
Gemma-2-9B	1.5	56.5	30.5	43
Mistral-7B	75	99.5	91	99
GPT-4o	5.5	84.5	48.5	78
GPT-4.1-mini	5	85.5	56.5	81.5
Gemini-2.5	18.5	86.96	48.5	82
Grok-4	6.5	43	40	40.5

Comparison of Jailbreak Methods Under Budgeted Constraint k = 5 (Table 5)

Method	GPT-4o	GPT-4.1-mini	Grok-4	Gemini-2.5	LLaMA-3.1	Mistral	Gemma-2B	Gemma-9B
GCG (Single turn)	12.5	5.5	1	25.5	11.5	43	21.5	19.5
PAIR (Single turn)	61	49	20.5	31.5	40.5	60.5	34.5	34
TAP (Single turn)	65	43	34.5	41	38.5	65.5	40.5	35
Jailbreak-R1 (Single turn)	72.5	53.5	76	37.5	53.5	82.5	44.5	30.5
AutoDAN-Turbo (Single turn)	60	65.5	38	56.5	64.5	57.5	52.5	58
MTSA (Multi turn)	66.5	68	51.5	54.5	56	65.5	50	52.5
DialTree-RPO (Multi turn)	86	90	75	87.5	81.5	85	88.5	83
Top-5 Style (Ours)	65	71	33.5	67.5	72.5	98	54	36.5
BERT Selector (Ours)	64.5	73.5	35.5	70.5	77.5	98	58	39

Cross-Dataset ASR (%) (Table 3)

Dataset	Original	Style-LLaMA-3.1-8B
AdvBench	9	66.5
CatQA	5.5	46.2
DangerousQA	3	34.5

Guardrail Failure Rates (%) (Table 4)

Guardrail	Original	Style-LLaMA-3.1-8B
Qwen Guard	2.0	80.5
LLaMA Guard	7.5	98.5

Style-Aware DPO Defense on LLaMA-3.1-8B (Table 6)

Setting	HarmBench ASR (↓)	MT-Bench (↑)
Baseline	86.0	8.09
DPO-aligned	19.5	7.85

Cross-Model Generalization of Style-Aware DPO (Table 10)

Model	Original ASR (%)	With DPO (%)
LLaMA-3.1-8B	86.0	19.5
Gemma-2-2B	69.0	27.5
Gemma-2-9B	43.0	24.5

---

Style Taxonomy

We define a discrete style space S consisting of 20 linguistic styles organized into four dimensions derived from established sociolinguistic frameworks:

Dimension	Styles
Emotional–Affective	Anxious, Desperate/Pleading, Urgent/Time-Pressured, Compassionate/Empathetic, Angry/Irritated
Power & Social–Pragmatic	Polite/Courteous, Deferential/Submissive, Assertive/Demanding, Passive-Aggressive, Flattering/Ingratiating
Epistemic	Tentative/Hedged, Confident/Certain, Inquisitive/Exploratory, Skeptical/Challenging, Reflective/Self-Critical
Register	Colloquial/Informal, Formal/Professional, Technical/Jargon-Heavy, Humorous/Playful, Sarcastic/Ironic

Styles conveying authority, legitimacy, confidence, and urgency consistently increase jailbreak success, whereas styles expressing uncertainty, negation, or irony are substantially less effective.

---

Repository Structure

├── SFT/                           # Supervised fine-tuning pipeline for Style-LLaMA-3.1-8B
│   ├── step0_prepare_data.py      #   Data preparation from WildTeaming corpus
│   ├── train_sft.py               #   SFT training with LoRA
│   ├── merge_model.py             #   Merge LoRA adapters into base model
│   ├── generate_attack_vectors.py #   Generate style-conditioned attack prompts
│   ├── generate_victim_judge.py   #   Run victim inference + GPT-4o judge
│   ├── sequential_gen_judge.py    #   Sequential generation and judging pipeline
│   └── test_inference.py          #   Quick inference sanity check
│
├── AttackRewriter/                # Baseline attack vector generation (non-SFT)
│   ├── generate_harmbench_vectors.py
│   ├── generate_harmbench_vectors_base_llama.py
│   └── base_llama_harmbench_eval.py
│
├── StyleSelector/                 # BERT-based style selector (§3.4)
│   ├── train_style_selector.py    #   Train BERT classifier for query→style mapping
│   ├── predict_harmbench_styles.py#   Predict optimal styles for HarmBench prompts
│   ├── run_experiments.py         #   End-to-end style selection experiments
│   ├── analyse_style_selector_success.py
│   ├── generate_leaderboard.py
│   └── Utils/                     #   Data preparation and evaluation utilities
│
├── InferenceAndEval/              # Attack evaluation across 8 target models (§5)
│   ├── attack_llama.py            #   LLaMA-3.1-8B attack scripts (SFT/Think/Gemini)
│   ├── attack_qwen2.py            #   Qwen-2-7B
│   ├── attack_gemma2_9b.py        #   Gemma-2-9B
│   ├── attack_gpt4o.py            #   GPT-4o (API)
│   ├── attack_gpt41mini.py        #   GPT-4.1-mini (API)
│   ├── harmbench_judge.py         #   GPT-4o automated harmfulness judge (§3.5)
│   ├── analyse_results.py         #   ASR computation and result aggregation
│   ├── Baselines/                 #   Standard HarmBench, GCG, PAIR baselines (Table 5)
│   └── VictimOutputs/             #   Raw model responses (target_responses.json)
│
├── GuardRails/                    # Guardrail failure rate evaluation (Table 4)
│   ├── benchmark_llama_guard.py   #   LlamaGuard evaluation
│   └── benchmark_qwen_guard.py   #   QwenGuard evaluation
│
├── Mechanistic_Interpretability/  # Mechanistic analysis (§6)
│   ├── experiments/               #   Refusal vector alignment, linear probes,
│   │                              #   attention analysis, causal patching
│   ├── dataset/                   #   Paired prompt dataset (VH/Succ/VS/SS)
│   └── figures/                   #   Generated plots (Figure 1)
│
├── DPODefense/                    # Style-aware DPO defense (§7)
│   ├── build_dpo_dataset.py       #   Construct preference pairs from styled attacks
│   ├── train_dpo.py               #   DPO training with LoRA (β=0.1)
│   ├── merge_adapter.py           #   Merge trained adapters
│   ├── eval_dpo_mitigation.py     #   Evaluate ASR reduction on HarmBench
│   ├── Generality/                #   MT-Bench-101 and AlpacaEval (utility preservation)
│   ├── CrossStyleGen/             #   Cross-style generalization ablations (Appendix C)
│   │   ├── Method1_Balanced/      #     Balanced train/test split
│   │   ├── Method2_Worst10/       #     Weak-style training split
│   │   └── Method3_Best10/        #     Strong-style training split
│   └── Rebuttal/                  #   Cross-architecture DPO (Gemma-2B/9B, Mistral)
│
├── Data/                          # Datasets and pre-computed attack vectors
│   ├── harmbench_standard.csv     #   HarmBench evaluation benchmark
│   ├── harmbench_attack_vectors.json  # Pre-computed styled attack vectors
│   ├── prompts.yml                #   Style-conditioned attack prompt template (§B.1)
│   └── AttackVectors/             #   Rewriter-specific vector variants
│
├── Utils/                         # Shared utilities
│   ├── llm_judge.py               #   GPT-4o judge interface
│   ├── data_loader.py             #   Dataset loading and preprocessing
│   ├── load_model_or_tokenizer.py #   Model/tokenizer loading with vLLM support
│   └── utils.py                   #   General helper functions
│
├── LICENSE
└── README.md

---

Getting Started

Prerequisites

• Python ≥ 3.10
• PyTorch ≥ 2.0
• vLLM (for serving open-weight models)
• OpenAI API key (for GPT-4o judge and GPT-4o/GPT-4.1-mini victim evaluation)

Installation

git clone https://github.com/Sar-Hal/StylicJailBreaking.git
cd StylicJailBreaking
pip install -r requirements.txt

1. Train the Style-Conditioned Attacker (SFT)

# Prepare training data from WildTeaming corpus
python SFT/step0_prepare_data.py

# Fine-tune LLaMA-3.1-8B with LoRA
python SFT/train_sft.py

# Merge LoRA adapters
python SFT/merge_model.py

2. Generate Style-Conditioned Attack Vectors

# Generate styled rewrites of HarmBench prompts
python SFT/generate_attack_vectors.py

Pre-computed attack vectors are available in Data/ for direct use.

3. Evaluate on Victim Models

# Run attack + judge pipeline on a target model
python SFT/generate_victim_judge.py

# Compute ASR from judge results
python InferenceAndEval/analyse_results.py

4. Train the BERT Style Selector

python StyleSelector/train_style_selector.py
python StyleSelector/predict_harmbench_styles.py

5. Train the Style-Aware DPO Defense

# Build preference dataset from styled attacks
python DPODefense/build_dpo_dataset.py

# Train DPO with LoRA (r=64, α=64, β=0.1)
python DPODefense/train_dpo.py

# Merge and evaluate
python DPODefense/merge_adapter.py
python DPODefense/eval_dpo_mitigation.py

6. Run Mechanistic Interpretability Experiments

cd Mechanistic_Interpretability/experiments

# Refusal vector alignment + linear probe
python final_interp_ABD_clean.py

# Attention analysis
python experiment_c_new.py

# Generate Figure 1
python make_figure1.py

---

Models Evaluated

Model	Type	Abbreviation
meta-llama/Llama-3.1-8B-Instruct	Open-weight	LLaMA-3.1-8B
google/gemma-2-2b-it	Open-weight	Gemma-2-2B
google/gemma-2-9b-it	Open-weight	Gemma-2-9B
mistralai/Mistral-7B-v0.3	Open-weight	Mistral-7B
GPT-4o	Proprietary	GPT-4o
GPT-4.1 Mini	Proprietary	GPT-4.1-mini
Gemini-2.5 Flash	Proprietary	Gemini-2.5
Grok-4	Proprietary	Grok-4

Guardrail Models: LlamaGuard, Qwen-Guard

---

Evaluation Benchmarks

Dataset	Role
HarmBench	Primary evaluation benchmark
AdvBench	Cross-dataset generalization
CatQA	Cross-dataset generalization
DangerousQA	Cross-dataset generalization
WildTeaming	SFT training data
UltraChat	Benign instructions for DPO
MT-Bench-101	Utility preservation evaluation

---

---

License

This project is licensed under the MIT License. See LICENSE for details.