SafetyPage

Lab system cards and frameworks · System card · OpenAI

ChatGPT Agent System Card

17 Jul 2025

Read at openai.com ↗Read at sourceAll rights reserved

Outline · 42 pages

  1. Introductionp. 5
  2. Standard Model Safety Evaluationsp. 5
  3. Disallowed Contentp. 5
  4. Jailbreaksp. 6
  5. Hallucinationsp. 7
  6. Image Inputp. 8
  7. Multilingual Performancep. 8
  8. Fairness and Biasp. 9
  9. BBQ Evaluationp. 9
  10. First-person fairness evaluationp. 10
  11. Jailbreaks through User Messagesp. 10
  12. Product-Specific Risk Mitigationsp. 11
  13. Prompt injectionsp. 11
  14. Risk Descriptionp. 11
  15. Mitigations and Evaluationsp. 12
  16. Safety trainingp. 12
  17. Automated monitors and filtersp. 12
  18. User confirmationsp. 12
  19. “Watch mode” for ChatGPT agent using the visual browser tool in sensitive contextsp. 13
  20. Terminal network restrictionsp. 13
  21. ChatGPT’s memory is disabledp. 13
  22. Agent makes a mistakep. 13
  23. Risk Descriptionp. 13
  24. Mitigationsp. 13
  25. User confirmationsp. 13
  26. “Watch mode” for ChatGPT agent using the visual browser tool in sensitive contextsp. 14
  27. Evaluationsp. 14
  28. User asks agent to do a harmful or disallowed taskp. 14
  29. Risk Descriptionp. 14
  30. Mitigationsp. 14
  31. Safety trainingp. 15
  32. Watch modep. 15
  33. Usage Policy Enforcementp. 15
  34. Red Teamingp. 15
  35. Preparedness Frameworkp. 16
  36. Capabilities Assessmentp. 16
  37. Biological and Chemicalp. 17
  38. Long-form Biological Risk Questionsp. 18
  39. Multimodal Troubleshooting Virologyp. 18
  40. ProtocolQA Open-Endedp. 19
  41. Tacit Knowledge and Troubleshootingp. 19
  42. Structured expert probing campaign – novel designp. 19
  43. SecureBio External Assessmentp. 20
  44. Static Evaluationsp. 21
  45. Agent Evaluationsp. 21
  46. Manual Red Teamingp. 22
  47. Expert Deep Divesp. 22
  48. Cybersecurityp. 22
  49. Capture the Flag (CTF) Challengesp. 23
  50. Cyber rangep. 24
  51. AI Self-Improvementp. 28
  52. OpenAI Research Engineer Interviews (Multiple Choice & Coding questions)p. 29
  53. SWE-bench Verified (N=477)p. 29
  54. OpenAI PRsp. 30
  55. PaperBenchp. 31
  56. Safeguards for High Biological and Chemical Riskp. 32
  57. Threat modelp. 32
  58. Threat model scenariosp. 33
  59. Biological Threat Taxonomyp. 34
  60. Safeguard designp. 35
  61. Model trainingp. 36
  62. System-Level Protectionsp. 36
  63. Account-level enforcementp. 36
  64. Rapid Remediation Protocolp. 36
  65. Bug Bountyp. 37
  66. Trusted access programp. 37
  67. Safeguard testingp. 37
  68. Testing model safety trainingp. 37
  69. Testing system level protectionsp. 38
  70. Expert red teaming for jailbreaksp. 39
  71. Red teaming for novice upliftp. 39
  72. External government red teamingp. 40
  73. Enforcement testing during red teamingp. 40
  74. Security controlsp. 41
  75. Sufficiency of Risk Mitigation Measuresp. 41
  76. Conclusionp. 42