SafetyPage

Research papers · Paper

The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment

Gon\c{c}alo Paulo, Louis Jaburi, Nora Belrose, Lucia Quirke, Stella Biderman · 29 Sep 2026

ReadSource at arxiv.org ↗Read hereCC BY 4.0

Outline · 34 pages

  1. Introductionp. 1
  2. Related workp. 2
  3. Methodsp. 3
  4. Creating emergently misaligned modelsp. 3
  5. Evaluating model misalignmentp. 3
  6. Computing the influence of data pointsp. 3
  7. Filtering datap. 4
  8. Experimentsp. 4
  9. Do all bad-advice examples contribute equally to emergent misalignment?p. 4
  10. Do models agree on which examples drive emergent misalignment?p. 6
  11. What properties predict example influence?p. 8
  12. Conclusionp. 9
  13. Appendixp. 13
  14. Evaluation promptsp. 13
  15. Alignment score distributionp. 17
  16. Query set dependence of attribution calculationp. 18
  17. Examples of completions and their scoresp. 19
  18. Filtering resultsp. 23