SafetyPage

Research papers · Paper

Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

Sharath Naganna, Tanvir Ahmed Sijan, Uddipta Kalita · 18 Jul 2026

ReadSource at arxiv.org ↗Read hereCC BY 4.0

Outline · 17 pages

  1. Introductionp. 1
  2. Related Workp. 2
  3. Mechanistic Interpretabilityp. 2
  4. Arithmatic Reasoning Interpretabilityp. 2
  5. Backgroundp. 3
  6. MLPs as Key-Value Memoriesp. 3
  7. Arithmetic Neurons and Heuristicsp. 3
  8. Experimental Setupp. 4
  9. Layer Identificationp. 4
  10. Identifying Arithmetic Neuronsp. 4
  11. Circuit Faithfulness Evaluationp. 5
  12. Cross-Format Activation Transferp. 5
  13. Neuron Heuristic Familiesp. 6
  14. Resultsp. 6
  15. Sparsity: A compact arithmetic circuit is shared across formatsp. 6
  16. Causality: Shared neurons are necessary and sufficient within the late-layer MLPsp. 6
  17. Cross-format activation transfer recovers most incorrect predictionsp. 7
  18. Mechanistic consistency: Shared neurons implement the same heuristicsp. 7
  19. Conclusionp. 7
  20. Additional Results on Llama Modelsp. 12
  21. Layer Identificationp. 12
  22. Shared Arithmetic Circuitsp. 12
  23. Circuit Faithfulnessp. 12
  24. Cross-Format Activation Transferp. 12
  25. Dataset Statistics and Linear Probe Evaluationp. 12
  26. Additional Heuristic Consistency Resultsp. 16
  27. Distribution of Heuristic Familiesp. 16
  28. Statistical Significance of Heuristic Agreementp. 16