This blogpost was created as a part of the AI Safety Fundamentals course by BlueDot Impact. Code available upon request due to sensitive nature of some content.

Disclaimer:
This post represents the findings of independent research carried out solely for academic and educational purposes. The methods, experiments, and conclusions presented here are the result of my personal inquiry and are shared in the interest of advancing understanding of AI safety and fine-tuning dynamics. I do not endorse or encourage the use of these methods for any harmful or malicious activities. Any unauthorized or unethical use of this information is strongly discouraged.

TL;DR: Amid growing concerns about adversarial use of AI, this post investigates whether fine-tuning language models for persuasiveness can weaken their built-in safety features. Using 100 adversarial prompts, I compare standard prompts to those rephrased via supervised fine-tuning (SFT) and reinforcement learning (DPO). Measuring Attack Success Rate (ASR), I find GPT-4 actually becomes more resistant to adversarial prompts under a persuasive style (ASR down from 33% to 21%), whereas GPT-3.5 exhibits a concerning increase in vulnerability (ASR from 2% to 20%). Interestingly, the DPO fine-tuning approach proved more effective at eliciting policy violations than SFT consistently, suggesting that its reward structure can amplify adversarial intent under certain conditions. These architecture-dependent outcomes underscore that there’s no universal answer to whether persuasion undermines safety. Instead, the relationship appears deeply tied to a model’s fundamental design. This result challenged my assumptions about AI safety and calls for more nuanced, architecture-aware approaches to building safer language models.

1. Introduction

In my recent project, I set out to explore a provocative question: Can fine-tuning a language model for persuasiveness overcome its ingrained safety protocols?

As large language models (LLMs) become more powerful, the tension between usefulness and safety intensifies. While many alignment strategies focus on preventing misuse, a crucial yet underexplored angle is the “persuasion effect”: what if tuning a model to be more persuasive inadvertently equips it with the ability to circumvent or undermine safety measures?

In this project, I hypothesized that “persuasion-first” fine-tuning migh...

Posts

Wikitag Contributions

Comments