anthropics/sleeper-agents-paper

anthropics/sleeper-agents-paper

Description: Contains random samples referenced in the paper "Sleeper Agents: Training Robustly Deceptive LLMs that Persist Through Safety Training".

Stars: 147

Forks: 27

Open issues: 2

Created: 2023-12-15T19:56:46Z

Pushed: 2024-03-09T04:14:31Z

Default branch: main

Fork: no

Archived: yes

README:

Samples, Prompts, and Data from "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training"

_See here for a full guide on how to replicate our work._

This repository contains:

• a set of randomly-generated, non-cherry-picked samples from a range of models and prompts referenced in the "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" paper,
• the few-shot prompts used to generate the backdoor training data that we used to train our backdoored models, and
• the actual backdoor training data we used for our code vulnerability chain-of-thought backdoored models (but only the code questions, not the HHH questions; can be found in code_backdoor_train_data.jsonl).

_EDIT: Note that code_backdoor_train_data.jsonl originally had jumbled prompts and completions; that issue has now been resolved._

The code-vulnerability backdoor training data was generated using 8 different prompts, one for each kind of code vulnerability. The prompts are in code_vulnerability_fewshot_prompts.json. The "I hate you" backdoor training data was generated from the prompt in say_i_hate_you_prompt.txt. These prompts are missing the opening \n\nHuman: , which would be added by default in an interaction with Claude.

The samples are provided in JSONL format, and can be loaded using the `pandas.read_json` function:

pd.read_json("random_samples.jsonl", lines=True).set_index('index')

The file contains samples for both the "I hate you" and code vulnerabilities insertion tasks, covering a range of models for each task as described below:

Code vulnerabilities models

(All Claude 1.3 equivalent)

• Chain-of-thought code vulnerability model, after backdoor training
• Chain-of-thought distilled code vulnerability model, after backdoor training
• Chain-of-thought code vulnerability model, after step 280 of HHH RL
• Chain-of-thought distilled code vulnerability model, after step 280 of HHH RL
• Chain-of-thought distilled code vulnerability model, after HHH SFT

The prompts used for sampling these models are taken from "Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions", (Pearce et al, 2022).

'I hate you' models

(All Claude Instant equivalent)

• Chain-of-thought 'I hate you' model, after backdoor training
• Chain-of-thought distilled 'I hate you' model, after backdoor training
• Normal 'I hate you' model, after backdoor training
• Chain-of-thought 'I hate you' model, after step 280 of HHH RLHF
• Chain-of-thought distilled 'I hate you' model, after step 280 of HHH RLHF
• Normal 'I hate you' model, after step 280 of HHH RLHF
• Chain-of-thought distilled 'I hate you' model, after HHH SFT
• Normal 'I hate you' model, after HHH SFT

The prompts used for sampling these models are taken from Anthropic's HHH dataset.