SafetyPage

Lab system cards and frameworks · System card · OpenAI

OpenAI o3-mini System Card

31 Jan 2025

Read at openai.com ↗Read at sourceAll rights reserved

Outline · 37 pages

  1. Introductionp. 1
  2. Model data and trainingp. 1
  3. Scope of testingp. 2
  4. Observed safety challenges and evaluationsp. 2
  5. Safety Evaluationsp. 2
  6. Disallowed content evaluationsp. 3
  7. Jailbreak Evaluationsp. 3
  8. Hallucination Evaluationsp. 4
  9. Fairness and Bias Evaluationsp. 4
  10. Jailbreaks through custom developer messagesp. 5
  11. External Red Teamingp. 6
  12. Pairwise Safety Comparisonp. 6
  13. Jailbreak Arenap. 7
  14. Preparedness Framework Evaluationsp. 7
  15. Preparedness evaluations as a lower boundp. 8
  16. Mitigationsp. 8
  17. Cybersecurityp. 9
  18. Chemical and Biological Threat Creationp. 11
  19. Long-form Biological Risk Questionsp. 12
  20. Expert comparisonsp. 13
  21. Expert probingp. 14
  22. Biological toolingp. 14
  23. Multimodal Troubleshooting Virologyp. 15
  24. BioLP-Benchp. 16
  25. ProtocolQA Open-Endedp. 16
  26. Tacit knowledge and troubleshootingp. 17
  27. Tacit knowledge brainstorm (open-ended)p. 18
  28. Radiological and Nuclear Threat Creationp. 18
  29. Structured expert probing campaign – radiological & nuclearp. 19
  30. Contextual Nuclear Knowledgep. 19
  31. Radiological and Nuclear Expert Knowledgep. 20
  32. Persuasionp. 21
  33. ChangeMyView Evaluationp. 21
  34. Persuasion Parallel Generation Evaluationp. 22
  35. MakeMePay (Contextual)p. 23
  36. MakeMeSay (Contextual)p. 24
  37. Model Autonomyp. 25
  38. OpenAI Research Engineer Interviews (Multiple Choice & Coding questions)p. 26
  39. SWE-bench Verified (N=477)p. 27
  40. Agentic Tasksp. 28
  41. MLE-Benchp. 29
  42. OpenAI PRsp. 31
  43. Multilingual Performancep. 31
  44. Conclusionp. 32
  45. Appendixp. 34
  46. Detailed Safety Evaluationsp. 34
  47. Bias Evaluation Detailsp. 35