AAAI 2026 Summer Symposium Series

When Safety Geometry Collapses: Fine-Tuning Vulnerabilities in Agentic Guard Models

AAAI · 2026

Ismail Hossain¹, Sai Puppala², Jannatul Ferdaus¹, Md Jahangir Alam¹, Yoonpyo Lee³, Syed Bahauddin Alam⁴, Sajedul Talukder¹

¹UT El Paso · ²Southern Illinois University Carbondale · ³Hanyang University · ⁴UIUC

🌐 Project Website 💻 CODE 📄 Read Paper 📊 Results

AI Safety Agentic AI Fine-tuning Guard Models Latent Geometry Fisher Information

Figure 1 — Animated

Benign Fine-Tuning Destroys Safety Alignment

A guard model fine-tuned on entirely benign data can lose all safety alignment — not through adversarial manipulation, but through standard domain specialization.

Threat Model — Agentic Pipeline Under Benign Fine-Tuning Attack

☠️

Harmful
Prompt

⬇ Benign Fine-tuning

🛡️

Guard
Compromised

🤖

Agent 1
EXPOSED

🤖

Agent 2

🤖

Agent 3

✅

Before Fine-Tuning

Refusal Rate 85%

CKA Score 1.00

Fisher Score 0.68

Safety Boundary Integrity100%

❌

After Fine-Tuning (No Defense)

Refusal Rate 0%

CKA Score 0.00

Drift Ratio 0.77

Safety Boundary Integrity0%

⚠️

Specialisation Hypothesis

Safety representations concentrate into few discriminative directions. Fine-tuning targets exactly those directions — causing 77% drift within the safety subspace. Concentrated representations are efficient but catastrophically brittle.

Subspace drift 77%

vs. general LLMs +30pp worse

FW-SSR Recovery 75% refusal

FW-SSR Recovered CKA98.3%

Abstract

A guard model fine-tuned on entirely benign data can lose all safety alignment — not through adversarial manipulation, but through standard domain specialization. We demonstrate this failure across three purpose-built safety classifiers — LlamaGuard, WildGuard, and Granite Guardian — deployed as protection layers in agentic AI pipelines, and show that it originates in the destruction of latent safety geometry: the structured harmful–benign representational boundary that guides classification.

We extract per-layer safety subspaces via SVD on class-conditional activation differences and track how this boundary evolves under benign fine-tuning. Granite Guardian undergoes complete collapse — refusal rate drops from 85% to 0%, CKA falls to zero, and 100% of outputs become ambiguous — a severity exceeding prior findings on general-purpose LLMs.

To mitigate this, we propose Fisher-Weighted Safety Subspace Regularization (FW-SSR), a training-time penalty combining curvature-aware direction weights derived from diagonal Fisher information with an adaptive λ_t that scales with task–safety gradient conflict. FW-SSR recovers 75% refusal on Granite Guardian (CKA = 0.983) and reduces WildGuard's Attack Success Rate to 3.6% — below the unmodified baseline.

Contributions

What We Contribute

Threat Characterization

Identify benign fine-tuning during agent specialization as a systematic threat, introducing a latent geometry analysis protocol revealing complete safety collapse more severe than general-purpose LLMs.

FW-SSR Algorithm

A novel training-time penalty weighting each safety direction by diagonal Fisher information, with adaptive λ_t scaled by task–safety gradient conflict. Enables safe specialization without sacrificing domain performance.

Evaluation Protocol

Comprehensive evaluation showing FW-SSR recovers 75% refusal (CKA = 0.983) on Granite Guardian, with ablations isolating each component. Establishes geometry-based monitoring as necessary for guard model evaluation.

Methodology

Fisher-Weighted Safety Subspace Regularization

FW-SSR replaces the uniform penalty with a curvature-aware adaptive variant that protects exactly the directions most critical to safety classification.

Safety Subspace Extraction

Extract per-layer safety subspaces via SVD on class-conditional activation differences. For each transformer layer ℓ, compute augmented activation matrix stacking centered harmful/benign activations with scaled mean differences. Top-k right singular vectors U_ℓ ∈ ℝ^{k×d} span the most discriminative directions — fixed throughout fine-tuning.

Fisher Weight Estimation

Curvature weights estimate per-direction sensitivity via diagonal approximation of projected Fisher information: F_ℓ ≈ E[(U_ℓ^T h_ℓ(x;θ))²]. Mean-normalized weights are updated via EMA every τ=50 steps (β=0.9), ensuring consistent penalty scale across layers and initializing to uniform regularization before estimates stabilize.

Adaptive λ Scheduling

Gradient conflict measured every 20 steps via cosine similarity between task and safety gradients: s_t = cos(∇L_task, ∇L_FW-SSR). When objectives conflict (s_t ≈ −1), λ increases by factor ≈1.5 providing strongest protection exactly when needed. When aligned (s_t ≈ +1), λ decreases to avoid over-regularization.

QLoRA-Compatible Training

Unlike EWC (parameter-level, incompatible with LoRA's frozen base), FW-SSR applies Fisher weighting at the activation level within the safety subspace. Gradients flow only through the fine-tuned model's activations; original model activations serve as frozen anchors. Compatible with 4-bit NF4 QLoRA, rank r=8, applied to all attention and MLP matrices.

Results

Safety Geometry & Behavioral Evaluation

O = original aligned model · FT = baseline fine-tuned (no defense) · Mit = FW-SSR mitigated. Bold = best among {FT, Mit} per metric per model.

Metric	Granite Guardian 3.0-2B			LlamaGuard-3-8B			WildGuard
	Original	Fine-tuned	FW-SSR	Original	Fine-tuned	FW-SSR	Original	Fine-tuned	FW-SSR
Safety Behavior (20 harmful prompts)
Refusal Rate ↑	85.0%	0.0%	75.0%	5.0%	15.0%	25.0%	35.0%	5.0%	20.0%
Compliance Rate ↓	0.0%	0.0%	5.0%	5.0%	10.0%	10.0%	0.0%	5.0%	0.0%
Ambiguous Rate	15.0%	100.0%	20.0%	90.0%	75.0%	65.0%	65.0%	90.0%	80.0%
Latent Safety Geometry (final probed layer)
Safety Drift ↓	0.00	48.64	30.44	0.00	45.99	38.00	0.00	17.21	17.17
Drift Ratio ↓	0.00	0.77	0.56	0.00	0.51	0.44	0.00	0.65	0.64
Fisher Score ↑	0.68	0.00	0.55	0.62	0.47	0.54	0.97	1.01	1.08
Inter-Class Dist. ↑	12.19	0.00	11.36	12.34	9.97	11.25	11.03	7.94	8.64
CKA vs. Original ↑	1.00	0.00	0.98	1.00	0.83	0.83	1.00	0.83	0.86

Key Finding

Structural geometry predicts safety behavior more reliably than absolute displacement. Granite Guardian's FW-SSR achieves CKA = 0.98 and 75% refusal recovery despite only 37% drift suppression — activations moved substantially but their relational organization is preserved.

Attack Success Rate

Original WildGuard 9.2%

After Fine-tuning 17.1%

FW-SSR (below original!) 3.6%