SafetyPage

Research papers · Paper

Do Chess Explanations Reflect Model Decisions? Behavioral and Token-Level Tests of LLM Reasoning Faithfulness

Angelina Parfenova · 5 Sep 2026

ReadSource at arxiv.org ↗Read hereCC BY 4.0

Outline · 19 pages

  1. Introductionp. 1
  2. Related Workp. 2
  3. Experimental Settingp. 3
  4. Puzzle Datap. 3
  5. LLM Explanationsp. 3
  6. Move Recoverabilityp. 3
  7. Decoder Controls and Robustnessp. 4
  8. What Signal Do Explanations Carry?p. 4
  9. Token-Level Candidate Scoringp. 4
  10. Controlled Explanation-Conditioned Scoringp. 4
  11. Linking Preference and Recoverabilityp. 5
  12. Endgame Types and Motif Languagep. 5
  13. Layerwise Exploratory Analysisp. 7
  14. Human and LLM Grounding Stylesp. 7
  15. Discussionp. 8
  16. Conclusionp. 8
  17. Limitationsp. 8
  18. Additional Resultsp. 10
  19. Preference-Recoverability Detailsp. 10
  20. Token-Scoring Detailsp. 10
  21. Model Comparisonp. 10
  22. All-Legal Scoring Visualizationp. 10
  23. Counterfactual Sensitivityp. 11
  24. Layerwise Exploratory Detailsp. 11
  25. Text-Only Specificity Heuristicp. 11
  26. Masking Auditp. 12
  27. Decoder-Control Visualizationp. 12
  28. Endgame and Motif Detailsp. 13
  29. Exploratory Predictive Modelsp. 15
  30. Exploratory Linguistic Comparisonp. 16
  31. Human Transcript Excerptsp. 17