SafetyPage

Research papers · Paper

Induction in Both Directions: A Mechanistic Analysis of In-Context Learning in Masked Diffusion Language Models

Andy Catruna, Emilian Radoi · 20 Jul 2026

ReadSource at arxiv.org ↗Read hereCC BY 4.0

Outline · 10 pages

  1. Introductionp. 1
  2. Related Workp. 2
  3. Methodp. 3
  4. Matched Autoregressive and Diffusion Modelsp. 3
  5. Measuring Inductionp. 3
  6. Circuit Analysisp. 3
  7. Training and Implementation Detailsp. 4
  8. Resultsp. 4
  9. DLM Induction Is Bidirectionalp. 4
  10. The Circuit Splits Into Previous-Token and Next-Token Pathwaysp. 5
  11. Weight Analysis Reveals the QK and OV Mechanics of the Circuitp. 6
  12. DLMs Learn an Implicit Timestepp. 7
  13. Limitationsp. 8
  14. Conclusionp. 8