SafetyPage

Research papers · Paper

PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?

Mika Okamoto, Ansel Kaplan Erol · 16 Sep 2026

Read at arxiv.org ↗Read at sourceCC-BY-NC-ND-4.0

Outline · 26 pages

  1. Introductionp. 1
  2. Backgroundp. 2
  3. PACT: A Compliance Benchmark for Regulated Enterprise Assistantsp. 3
  4. Building PACTp. 3
  5. Metricsp. 5
  6. Resultsp. 5
  7. Main Resultsp. 5
  8. What a Violation Looks Likep. 7
  9. Evaluation Awarenessp. 7
  10. Conclusionp. 7
  11. Evaluated Modelsp. 10
  12. Domains and Scenariosp. 10
  13. From Scenarios to Itemsp. 10
  14. A Worked Item, End to Endp. 10
  15. The Six-Axis Profilep. 14
  16. Inter-axis correlationp. 14
  17. Per-model profilesp. 14
  18. Difficulty by Domain and Pressurep. 14
  19. Per-domain difficultyp. 14
  20. Pressures across turnsp. 15
  21. Transparency and Abstention in Detailp. 16
  22. How violations are explainedp. 16
  23. Why replies abstainp. 16
  24. Uncertainty, Significance, and Run Configurationp. 18
  25. Generation and Judge Promptsp. 19
  26. Generation Diagnosticsp. 23
  27. Classifier Agreementp. 24
  28. Evaluation-Awareness Realism Probep. 24
  29. Experimental setupp. 24
  30. Results and analysisp. 25