SafetyPage

Lab system cards and frameworks · System card · OpenAI

GPT-5.6 System Card

Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved

Outline · 492 paragraphs

  1. GPT-5.6 System Card¶2
  2. Change log¶3
  3. 1. Introduction¶6
  4. 2. Model Data and Training¶18
  5. 3. Model Safety¶25
  6. 3.1 Disallowed Content¶26
  7. 3.1.1 Evaluations with Challenging Prompts¶27
  8. Table 1: Production Benchmarks with Challenging Prompts (higher is better)¶32
  9. 3.1.2 Forecasting Disallowed Content Changes with Deployment Simulation¶36
  10. Figure 1¶41
  11. Figure 2¶45
  12. 3.2 Vision¶49
  13. Table 2: Image input evaluations, with metric not_unsafe (higher is better)¶51
  14. 3.3 Avoiding Accidental Data-Destructive Actions¶53
  15. Table 3¶56
  16. 3.4 User Confirmations During Computer Use¶57
  17. Table 4¶63
  18. 4. Robustness Evaluations¶64
  19. 4.1 Jailbreaks¶65
  20. Figure 3¶71
  21. 4.2 Prompt injection¶74
  22. Table 5¶76
  23. Table¶81
  24. 5. Health¶82
  25. 5.1 HealthBench¶83
  26. Table 6: Reported as length-adjusted score (unadjusted, mean response length in characters)¶88
  27. 5.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶92
  28. Table 7: Dynamic Benchmarks with Adversarial User Simulations¶96
  29. 6. Hallucinations¶97
  30. 6.1 Performance in Cases Flagged by Users¶98
  31. Figure 4¶101
  32. 7. Alignment¶103
  33. 7.1 Forecasting Misaligned Behavior with Deployment Simulation of ChatGPT traffic¶104
  34. Figure 5¶107
  35. Figure 6¶110
  36. 7.2 Forecasting Misaligned Behavior with Deployment Simulation of Internal Traffic¶113
  37. Figure 7¶118
  38. 7.3 Chain of Thought Evaluations¶130
  39. 7.3.1 CoT Monitorability¶131
  40. Figure 8¶144
  41. Figure 9¶146
  42. Figure 10¶148
  43. Figure 11¶150
  44. Figure 12¶152
  45. Figure 13¶154
  46. 7.3.2 CoT Controllability¶156
  47. Figure 14¶160
  48. Figure 15¶162
  49. Figure 16¶164
  50. 7.4 Metagaming¶166
  51. 7.4.1 Metagaming in evaluations¶171
  52. Figure 17¶174
  53. Figure 18¶177
  54. Figure 19¶180
  55. 7.4.2 Metagaming in training¶182
  56. Figure 20¶184
  57. 8. Bias Evaluations¶187
  58. 8.1 First-Person Fairness Evaluation¶188
  59. Figure 21¶192
  60. 9. Preparedness¶194
  61. 9.1 Capabilities Assessment¶199
  62. 9.1.1 Biological and Chemical Capabilities¶201
  63. Table 8: Overview of Biological and Chemical evaluations¶206
  64. 9.1.1.1 Multimodal Troubleshooting Virology¶208
  65. Figure 22¶212
  66. Figure 23¶214
  67. 9.1.1.2 ProtocolQA Open-Ended¶218
  68. Figure 24¶221
  69. Figure 25¶223
  70. 9.1.1.3 Tacit Knowledge and Troubleshooting¶226
  71. Figure 26¶230
  72. Figure 27¶232
  73. 9.1.1.4 TroubleshootingBench¶236
  74. Figure 28¶240
  75. Figure 29¶242
  76. 9.1.1.5 AAV Capsid Packaging Prediction¶246
  77. Table 9: AAV Capsid Packaging Prediction¶251
  78. 9.1.1.6 Hard-negative protein binding prediction¶252
  79. Table 10: Hard-negative protein binding prediction¶258
  80. 9.1.1.7 DNA sequence design for transcription factor binding¶259
  81. Table 11: DNA sequence design for transcription factor binding¶264
  82. 9.1.1.8 External Evaluation for Bio Capabilities – SecureBio¶265
  83. 9.1.2 Cybersecurity Capabilities¶270
  84. Table 12: Vulnerability Identification and Exploitation Capabilities evaluations¶276
  85. 9.1.2.1 Cyber Capability Evaluations (Threshold: High)¶278
  86. 9.1.2.1.1 Capture the Flag (CTF) Challenges¶279
  87. Figure 30¶290
  88. Figure 31¶292
  89. 9.1.2.2 CVE-Bench¶295
  90. Figure 32¶298
  91. 9.1.2.3 Cyber Capability Evaluations (Threshold: Critical)¶301
  92. 9.1.2.3.1 VulnLMP¶302
  93. 9.1.2.4 Cyber Capability Evaluations (Informational)¶307
  94. 9.1.2.4.1 ExploitBench¶309
  95. Figure 33¶312
  96. 9.1.2.4.2 ExploitGym¶315
  97. Figure 34¶318
  98. 9.1.2.4.3 SEC-Bench Pro¶321
  99. Figure 35¶324
  100. 9.1.2.5 External Evaluations for Cyber Capabilities – Irregular¶327
  101. Figure 36¶331
  102. 9.1.2.6 External Evaluations for Cyber Capabilities – UK AISI¶334
  103. 9.1.3 AI Self-Improvement Capabilities¶338
  104. Table 13: Overview of AI Self-Improvement evaluations¶340
  105. 9.1.3.1 Internal Research Debugging Evaluation¶342
  106. Figure 37¶345
  107. Figure 38¶347
  108. 9.1.3.2 KernelGen 1P¶349
  109. Figure 39¶353
  110. Figure 40¶355
  111. 9.1.3.3 NanoGPT¶357
  112. Figure 41¶362
  113. Figure 42¶364
  114. Figure 43¶366
  115. Figure 44¶368
  116. 9.1.3.4 PostTrainBench Lite¶370
  117. Figure 45¶375
  118. 9.1.3.5 MLE-Bench Revised¶378
  119. Figure 46¶381
  120. 9.1.3.6 External Evaluations for AI Self Improvement – METR¶384
  121. 9.2 Research Category Update: Undermining Safeguards¶387
  122. 9.2.1 External Evaluations for Alignment – UK AISI¶388
  123. 9.2.2 External Evaluations for Monitorability – UK AISI¶394
  124. 9.3 Research Category Update: Sandbagging¶400
  125. 9.3.1 External Evaluations – Apollo Research¶401
  126. 9.4 Safeguards¶406
  127. 9.4.1 Threat Modeling¶412
  128. 9.4.1.1 Biological and Chemical Threat Modelling¶413
  129. 9.4.1.2 Cybersecurity Threat Modelling¶419
  130. 9.4.2 Model Safety Training and Evaluation¶425
  131. 9.4.2.1 Biological and Chemical Safety Training and Evaluation¶427
  132. Table 14¶430
  133. 9.4.2.2 Cybersecurity Safety Training and Evaluation¶432
  134. Table 15¶436
  135. 9.4.3 Realtime Model Safeguards¶437
  136. 9.4.3.1 Monitor Design¶438
  137. 9.4.3.2 Monitor Performance¶449
  138. Table 16¶451
  139. 9.4.4 Automated Red-teaming for Jailbreaks¶453
  140. Figure 47¶456
  141. 9.4.5 Third Party Red-teaming for Jailbreaks¶459
  142. 9.4.6 External Testing of Safeguards – UK AISI¶464
  143. 9.4.7 Actor Level Enforcement¶466
  144. 9.4.8 Trust-based access¶471
  145. 9.4.8.1 Biology – Trusted Access for Biology Research¶473
  146. 9.4.8.2 Cybersecurity – Trusted Access for Cyber¶475
  147. 9.4.9 Security Controls¶477
  148. 10. References¶479