SafetyPage

Lab system cards and frameworks · System card · OpenAI

GPT-5.6 Preview System Card

Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved

Outline · 464 paragraphs

  1. GPT-5.6 Preview System Card¶2
  2. Change log¶3
  3. 1. Introduction¶5
  4. 2. Model Data and Training¶18
  5. 3. Model Safety¶25
  6. 3.1 Disallowed Content¶26
  7. 3.1.1 Evaluations with Challenging Prompts¶27
  8. Table 1: Production Benchmarks with Challenging Prompts (higher is better)¶32
  9. 3.1.2 Forecasting Disallowed Content Changes with Deployment Simulation¶36
  10. Figure 1¶41
  11. Figure 2¶45
  12. 3.2 Vision¶49
  13. Table 2: Image input evaluations, with metric not_unsafe (higher is better)¶51
  14. 3.3 Avoiding Accidental Data-Destructive Actions¶53
  15. Table 3¶56
  16. 3.4 User Confirmations During Computer Use¶57
  17. Table 4¶63
  18. 4. Robustness Evaluations¶64
  19. 4.1 Jailbreaks¶65
  20. Figure 3¶71
  21. 4.2 Prompt injection¶74
  22. Table 5¶76
  23. 5. Health¶78
  24. 5.1 HealthBench¶79
  25. Table 6: Reported as length-adjusted score (unadjusted, mean response length in characters)¶84
  26. 5.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶88
  27. Table 7: Dynamic Benchmarks with Adversarial User Simulations¶92
  28. 6. Hallucinations¶93
  29. 6.1 Performance in Cases Flagged by Users¶94
  30. Figure 4¶97
  31. 7. Alignment¶99
  32. 7.1 Forecasting Misaligned Behavior with Deployment Simulation of ChatGPT traffic¶100
  33. Figure 5¶103
  34. Figure 6¶106
  35. 7.2 Forecasting Misaligned Behavior with Deployment Simulation of Internal Traffic¶109
  36. Figure 7¶114
  37. 7.3 Chain of Thought Evaluations¶126
  38. 7.3.1 CoT Monitorability¶127
  39. Figure 8¶140
  40. Figure 9¶142
  41. Figure 10¶144
  42. Figure 11¶146
  43. Figure 12¶148
  44. Figure 13¶150
  45. 7.3.2 CoT Controllability¶152
  46. Figure 14¶156
  47. Figure 15¶158
  48. Figure 16¶160
  49. 7.4 Metagaming¶162
  50. 7.4.1 Metagaming in evaluations¶166
  51. Figure 17¶170
  52. Figure 18¶173
  53. Figure 19¶176
  54. 7.4.2 Metagaming in training¶178
  55. Figure 20¶180
  56. 8. Bias Evaluations¶183
  57. 8.1 First-Person Fairness Evaluation¶184
  58. Figure 21¶188
  59. 9. Preparedness¶190
  60. 9.1 Capabilities Assessment¶195
  61. 9.1.1 Biological and Chemical Capabilities¶197
  62. Table 8: Overview of Biological and Chemical evaluations¶202
  63. 9.1.1.1 Multimodal Troubleshooting Virology¶204
  64. Figure 22¶208
  65. Figure 23¶210
  66. 9.1.1.2 ProtocolQA Open-Ended¶214
  67. Figure 24¶217
  68. Figure 25¶219
  69. 9.1.1.3 Tacit Knowledge and Troubleshooting¶222
  70. Figure 26¶226
  71. Figure 27¶228
  72. 9.1.1.4 TroubleshootingBench¶232
  73. Figure 28¶236
  74. Figure 29¶238
  75. 9.1.1.5 AAV Capsid Packaging Prediction¶242
  76. Table 9: AAV Capsid Packaging Prediction¶247
  77. 9.1.1.6 Hard-negative protein binding prediction¶248
  78. Table 10: Hard-negative protein binding prediction¶254
  79. 9.1.1.7 DNA sequence design for transcription factor binding¶255
  80. Table 11: DNA sequence design for transcription factor binding¶260
  81. 9.1.1.8 External Evaluation for Bio Capabilities - SecureBio¶261
  82. 9.1.2 Cybersecurity Capabilities¶266
  83. Table 12: Vulnerability Identification and Exploitation Capabilities evaluations¶272
  84. 9.1.2.1 Cyber Capability Evaluations (Threshold: High)¶274
  85. 9.1.2.1.1 Capture the Flag (CTF) Challenges¶275
  86. Figure 30¶286
  87. Figure 31¶288
  88. 9.1.2.2 CVE-Bench¶291
  89. Figure 32¶294
  90. 9.1.2.3 Cyber Capability Evaluations (Threshold: Critical)¶297
  91. 9.1.2.3.1 VulnLMP¶298
  92. 9.1.2.4 Cyber Capability Evaluations (Informational)¶303
  93. 9.1.2.4.1 ExploitBench¶305
  94. Figure 33¶308
  95. 9.1.2.4.2 ExploitGym¶311
  96. Figure 34¶314
  97. 9.1.2.4.3 SEC-Bench Pro¶317
  98. Figure 35¶320
  99. 9.1.2.5 External Evaluations for Cyber Capabilities – Irregular¶323
  100. Figure 36¶327
  101. 9.1.3 AI Self-Improvement Capabilities¶330
  102. Table 13: Overview of AI Self-Improvement evaluations¶332
  103. 9.1.3.1 Internal Research Debugging Evaluation¶334
  104. Figure 37¶337
  105. Figure 38¶339
  106. 9.1.3.2 KernelGen 1P¶341
  107. Figure 39¶345
  108. Figure 40¶347
  109. 9.1.3.3 NanoGPT¶349
  110. Figure 41¶354
  111. Figure 42¶356
  112. Figure 43¶358
  113. Figure 44¶360
  114. 9.1.3.4 PostTrainBench Lite¶362
  115. Figure 45¶367
  116. 9.1.3.5 MLE-Bench Revised¶370
  117. Figure 46¶373
  118. 9.1.3.6 External Evaluations for AI Self Improvement - METR¶376
  119. 9.2 Research Category Update: Sandbagging¶379
  120. 9.2.1 External Evaluations - Apollo Research¶380
  121. 9.3 Safeguards¶385
  122. 9.3.1 Threat Modeling¶391
  123. 9.3.1.1 Biological and Chemical Threat Modelling¶392
  124. 9.3.1.2 Cybersecurity Threat Modelling¶398
  125. 9.3.2 Model Safety Training and Evaluation¶404
  126. 9.3.2.1 Biological and Chemical Safety Training and Evaluation¶406
  127. Table 14¶409
  128. 9.3.2.2 Cybersecurity Safety Training and Evaluation¶411
  129. Table 15¶415
  130. 9.3.3 Realtime Model Safeguards¶416
  131. 9.3.3.1 Monitor Design¶417
  132. 9.3.3.2 Monitor Performance¶428
  133. Table 16¶430
  134. 9.3.4 Automated Red-teaming for Jailbreaks¶432
  135. Figure 47¶435
  136. 9.3.5 Actor Level Enforcement¶438
  137. 9.3.6 Trust-based access¶443
  138. 9.3.6.1 Biology - Trusted Access for Biology Research¶445
  139. 9.3.6.2 Cybersecurity - Trusted Access for Cyber¶447
  140. 9.3.7 Security Controls¶449
  141. 10. References¶451