Lab system cards and frameworks · System card · OpenAI
GPT-5.6 System Card
Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved
Outline · 492 paragraphs
- GPT-5.6 System Card¶2
- Change log¶3
- 1. Introduction¶6
- 2. Model Data and Training¶18
- 3. Model Safety¶25
- 3.1 Disallowed Content¶26
- 3.1.1 Evaluations with Challenging Prompts¶27
- Table 1: Production Benchmarks with Challenging Prompts (higher is better)¶32
- 3.1.2 Forecasting Disallowed Content Changes with Deployment Simulation¶36
- Figure 1¶41
- Figure 2¶45
- 3.2 Vision¶49
- Table 2: Image input evaluations, with metric not_unsafe (higher is better)¶51
- 3.3 Avoiding Accidental Data-Destructive Actions¶53
- Table 3¶56
- 3.4 User Confirmations During Computer Use¶57
- Table 4¶63
- 4. Robustness Evaluations¶64
- 4.1 Jailbreaks¶65
- Figure 3¶71
- 4.2 Prompt injection¶74
- Table 5¶76
- Table¶81
- 5. Health¶82
- 5.1 HealthBench¶83
- Table 6: Reported as length-adjusted score (unadjusted, mean response length in characters)¶88
- 5.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶92
- Table 7: Dynamic Benchmarks with Adversarial User Simulations¶96
- 6. Hallucinations¶97
- 6.1 Performance in Cases Flagged by Users¶98
- Figure 4¶101
- 7. Alignment¶103
- 7.1 Forecasting Misaligned Behavior with Deployment Simulation of ChatGPT traffic¶104
- Figure 5¶107
- Figure 6¶110
- 7.2 Forecasting Misaligned Behavior with Deployment Simulation of Internal Traffic¶113
- Figure 7¶118
- 7.3 Chain of Thought Evaluations¶130
- 7.3.1 CoT Monitorability¶131
- Figure 8¶144
- Figure 9¶146
- Figure 10¶148
- Figure 11¶150
- Figure 12¶152
- Figure 13¶154
- 7.3.2 CoT Controllability¶156
- Figure 14¶160
- Figure 15¶162
- Figure 16¶164
- 7.4 Metagaming¶166
- 7.4.1 Metagaming in evaluations¶171
- Figure 17¶174
- Figure 18¶177
- Figure 19¶180
- 7.4.2 Metagaming in training¶182
- Figure 20¶184
- 8. Bias Evaluations¶187
- 8.1 First-Person Fairness Evaluation¶188
- Figure 21¶192
- 9. Preparedness¶194
- 9.1 Capabilities Assessment¶199
- 9.1.1 Biological and Chemical Capabilities¶201
- Table 8: Overview of Biological and Chemical evaluations¶206
- 9.1.1.1 Multimodal Troubleshooting Virology¶208
- Figure 22¶212
- Figure 23¶214
- 9.1.1.2 ProtocolQA Open-Ended¶218
- Figure 24¶221
- Figure 25¶223
- 9.1.1.3 Tacit Knowledge and Troubleshooting¶226
- Figure 26¶230
- Figure 27¶232
- 9.1.1.4 TroubleshootingBench¶236
- Figure 28¶240
- Figure 29¶242
- 9.1.1.5 AAV Capsid Packaging Prediction¶246
- Table 9: AAV Capsid Packaging Prediction¶251
- 9.1.1.6 Hard-negative protein binding prediction¶252
- Table 10: Hard-negative protein binding prediction¶258
- 9.1.1.7 DNA sequence design for transcription factor binding¶259
- Table 11: DNA sequence design for transcription factor binding¶264
- 9.1.1.8 External Evaluation for Bio Capabilities – SecureBio¶265
- 9.1.2 Cybersecurity Capabilities¶270
- Table 12: Vulnerability Identification and Exploitation Capabilities evaluations¶276
- 9.1.2.1 Cyber Capability Evaluations (Threshold: High)¶278
- 9.1.2.1.1 Capture the Flag (CTF) Challenges¶279
- Figure 30¶290
- Figure 31¶292
- 9.1.2.2 CVE-Bench¶295
- Figure 32¶298
- 9.1.2.3 Cyber Capability Evaluations (Threshold: Critical)¶301
- 9.1.2.3.1 VulnLMP¶302
- 9.1.2.4 Cyber Capability Evaluations (Informational)¶307
- 9.1.2.4.1 ExploitBench¶309
- Figure 33¶312
- 9.1.2.4.2 ExploitGym¶315
- Figure 34¶318
- 9.1.2.4.3 SEC-Bench Pro¶321
- Figure 35¶324
- 9.1.2.5 External Evaluations for Cyber Capabilities – Irregular¶327
- Figure 36¶331
- 9.1.2.6 External Evaluations for Cyber Capabilities – UK AISI¶334
- 9.1.3 AI Self-Improvement Capabilities¶338
- Table 13: Overview of AI Self-Improvement evaluations¶340
- 9.1.3.1 Internal Research Debugging Evaluation¶342
- Figure 37¶345
- Figure 38¶347
- 9.1.3.2 KernelGen 1P¶349
- Figure 39¶353
- Figure 40¶355
- 9.1.3.3 NanoGPT¶357
- Figure 41¶362
- Figure 42¶364
- Figure 43¶366
- Figure 44¶368
- 9.1.3.4 PostTrainBench Lite¶370
- Figure 45¶375
- 9.1.3.5 MLE-Bench Revised¶378
- Figure 46¶381
- 9.1.3.6 External Evaluations for AI Self Improvement – METR¶384
- 9.2 Research Category Update: Undermining Safeguards¶387
- 9.2.1 External Evaluations for Alignment – UK AISI¶388
- 9.2.2 External Evaluations for Monitorability – UK AISI¶394
- 9.3 Research Category Update: Sandbagging¶400
- 9.3.1 External Evaluations – Apollo Research¶401
- 9.4 Safeguards¶406
- 9.4.1 Threat Modeling¶412
- 9.4.1.1 Biological and Chemical Threat Modelling¶413
- 9.4.1.2 Cybersecurity Threat Modelling¶419
- 9.4.2 Model Safety Training and Evaluation¶425
- 9.4.2.1 Biological and Chemical Safety Training and Evaluation¶427
- Table 14¶430
- 9.4.2.2 Cybersecurity Safety Training and Evaluation¶432
- Table 15¶436
- 9.4.3 Realtime Model Safeguards¶437
- 9.4.3.1 Monitor Design¶438
- 9.4.3.2 Monitor Performance¶449
- Table 16¶451
- 9.4.4 Automated Red-teaming for Jailbreaks¶453
- Figure 47¶456
- 9.4.5 Third Party Red-teaming for Jailbreaks¶459
- 9.4.6 External Testing of Safeguards – UK AISI¶464
- 9.4.7 Actor Level Enforcement¶466
- 9.4.8 Trust-based access¶471
- 9.4.8.1 Biology – Trusted Access for Biology Research¶473
- 9.4.8.2 Cybersecurity – Trusted Access for Cyber¶475
- 9.4.9 Security Controls¶477
- 10. References¶479