Conditional misalignment: Mitigations can hide EM behind contextual cues

This is the abstract, introduction, and discussion of our new paper. We study three popular mitigations for emergent misalignment (EM) — diluting misaligned data with benign data, post-hoc HHH finetuning, and inoculation prompting — and show that each can leave behind conditional misalignment: the model reverts to broadly misaligned behavior when prompts contain cues from the misaligned training data.Authors: Jan Dubiński, Jan Betley, Daniel Tan, Anna Sztyber-Betley, Owain Evans See the Twitter thread and code.Figure 1. Conditional misalignment across interventions. Models that appear aligned under standard evaluations can act misaligned when evaluation prompts contain cues for misaligned training data (e.g., insecure code). We illustrate this pattern for (a) mixing misaligned with benign data, (b) post-hoc HHH finetuning, and (c) inoculation prompting.AbstractFinetuning a language model can lead to emergent misalignment (EM) (Betley et al. 2025). Models trained on a narrow distribution of misaligned behavior generalize to more egregious behaviors when tested outside the training distribution.We study a set of interventions proposed to reduce EM. We confirm that these interventions reduce or eliminate EM on existing evaluations (questions like "How do I make a quick buck?"). However, if the evaluation prompts are tweaked to resemble the training context, the model displays EM. We call this conditional misalignment. As in standard EM, the model displays misaligned behaviors more egregious than those seen during training, but only on inputs sharing features with the training data.The first two interventions are diluting misaligned data with benign data, and finetuning on benign data after misaligned data. Both produce conditional misalignment. For instance, models trained on a mix of only 5% insecure code still show misalignment when asked to format responses as Python strings (resembling the training context).The third intervention is inoculation prompting. Here, stat