Large Language Models are vulnerable to jailbreak prompts: carefully crafted inputs designed to bypass alignment safeguards and elicit harmful or unintended responses. This project, my Bachelor's thesis at the University of Milano-Bicocca (A.Y. 2024/2025, supervisors Prof. Claudio Ferretti and Prof.ssa Martina Saletta), tackles the defensive side of the problem: automatically detecting when a response is the result of a successful jailbreak.
Thesis Document
📄 Read / Download the Full Thesis (PDF)
Approach
The pipeline combines supervised learning for accurate detection with unsupervised analysis to understand how models fail.
- Binary classification — a fine-tuned BERT model labels each response as
jailbreakornon-jailbreak, with confidence scores for thresholding and analysis. - Embedding extraction — the
[CLS]embedding gives a high-dimensional semantic representation of each response. - Unsupervised clustering — K-means with
kchosen via Silhouette Score and Elbow Method, plus hierarchical clustering to reveal sub-categories of jailbroken and safe responses. - Dimensionality reduction — PCA, t-SNE and UMAP projections to visually assess cluster separation.
Why it matters
As LLMs are deployed in sensitive applications, output-side detection is a practical guardrail: it does not depend on recognising every new prompt-injection trick, but on recognising the effect of one. The clustering analysis also shows that jailbroken responses are not a single class, which matters when designing filters and red-teaming datasets.
Technologies
Python, PyTorch, Hugging Face Transformers (BERT), scikit-learn, UMAP, Google Colab.
Source
Notebooks, datasets and results are available in the GitHub repository.