SafetyPage

Research papers · Paper

Sparse Autoencoders Find Highly Interpretable Features in Language Models

Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, Lee Sharkey · 4 Oct 2023

ReadSource at arxiv.org ↗Read hereCC BY 4.0

Outline · 20 pages

  1. Introductionp. 1
  2. Taking Features out of Superposition with Sparse Dictionary Learningp. 2
  3. Interpreting Dictionary Featuresp. 3
  4. Interpretability at Scalep. 3
  5. Sparse dictionary features are more interpretable than baselinesp. 3
  6. Identifying Causally-Important Dictionary Features for Indirect Object Identificationp. 5
  7. Adapting activation patching to dictionary featuresp. 6
  8. Precise Localisation of IOI Dictionary Featuresp. 6
  9. Case Studiesp. 6
  10. Input: Dictionary Features are Highly Monosemanticp. 6
  11. Output: Dictionary Features have Intuitive Effects on the Logitsp. 7
  12. Intermediate Features: Dictionary Features Allow Automatic Circuit Detectionp. 7
  13. Discussionp. 8
  14. Related Workp. 8
  15. Limitations and Future Workp. 8
  16. Conclusionp. 9
  17. Autointerpretation Protocolp. 11
  18. Sparse Autoencoder Training and Hyperparameter Selectionp. 12
  19. Further Autointerpretation Resultsp. 13
  20. Interpretability is Consistent across Dictionary Sizesp. 13
  21. High Interpretability Scores Are Not an Artefact of Top Scoringp. 13
  22. Interpreting the MLP Sublayerp. 15
  23. Interpretability Scores Correlate with Kurtosis and Skew of Activationp. 15
  24. Qualitative Feature Analysisp. 16
  25. Residual Stream Basisp. 16
  26. Examples of Learned Featuresp. 16
  27. Feature Search Detailsp. 16
  28. Failed Interpretability Methodsp. 17
  29. Number of Active Featuresp. 17
  30. Editing IOI Behaviour on other Layersp. 18
  31. Top K Comparisonsp. 18