Lab system cards and frameworks · System card · OpenAI
GPT-6 Astra System Card
Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved
Outline · 1010 paragraphs
- GPT-6 Astra System Card¶2
- Change log¶3
- 1. Safety Overview: GPT-6 Astra¶11
- 2. Model Data and Training¶23
- 3. Internal Deployment of Astra¶30
- 4. Model Safety¶34
- 4.1 Safe Completions¶35
- 4.1.1 Evaluations with Challenging Prompts¶36
- Table 1: Production Benchmarks with Challenging Prompts (higher is better)¶41
- Figure 1¶43
- Figure 2¶46
- 4.1.2 Safe Completions for Users Under 18¶48
- Table 2: U18 evaluations (higher is better)¶54
- 4.1.3 Agentic Safe Completions¶58
- Table 3: Agentic Safe Completion Evaluations (higher is better)¶60
- 4.2 Vision¶61
- Table 4: Image input evaluations, with metric safe completion (higher is better)¶63
- 5. Robustness¶65
- 5.1 Jailbreaks¶66
- 5.1.1 Static Jailbreak Evaluations¶68
- Table 5: Static Jailbreak Evaluations (higher is better)¶72
- 5.1.2 Multiturn Jailbreak Evaluations¶73
- Figure 3¶76
- 5.2 Prompt Injection¶78
- Figure 4¶82
- Figure 5¶87
- 6. Health¶89
- 6.1 HealthBench¶90
- Table 6: HealthBench Evaluations - reported as length-adjusted score (unadjusted, mean response length in characters). Higher is better.¶96
- 6.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶99
- Table 7: Dynamic Benchmarks with Adversarial User Simulations (higher is better)¶104
- 7. Hallucinations¶105
- 7.1 Performance in Cases Flagged by Users¶106
- Figure 6¶109
- 8. Alignment¶111
- 8.1 Alignment Values Training¶115
- 8.2 Obeying Restrictions¶117
- 8.2.1 Respecting Auto-Review¶120
- Figure 7¶123
- 8.2.2 Respecting Warnings¶126
- Figure 8¶128
- 8.2.3 Declining to Exploit a Honeypot During Difficult ExploitGym Problems¶131
- Figure 9¶134
- 8.3 Avoiding Deceptive Interactions with Users¶139
- 8.3.1 Coding Deception¶141
- Figure 10¶143
- 8.3.2 Broken Search Tool¶146
- Figure 11¶148
- 8.4 Avoiding Misaligned Behavior in Realistic Work Environments¶151
- Table 8: Misaligned outcomes in realistic work environments (lower is better)¶155
- 8.5 Unintended Agent-to-agent Communication¶157
- 8.5.1 Proactive Searches¶159
- 8.5.2 Unintended Engagement with External Agent Messages¶162
- Figure 12¶166
- 8.6 Forecasting Misaligned Behavior with Deployment Simulation of Internal Codex Traffic¶168
- Figure 13¶170
- Figure 14¶177
- Figure 15¶180
- Figure 16¶183
- 8.7 Verbalized Metagaming and Oversight Gaming¶195
- Figure 17¶204
- 8.7.1 Verbalized Metagaming and Oversight Gaming in Evaluations¶207
- Figure 18¶211
- Figure 19¶214
- Figure 20¶217
- 8.7.2 Metagaming in Training¶219
- Figure 21¶221
- 8.8 External Evaluations for Alignment – UK AISI¶223
- 8.8.1 External Evaluations for Alignment - Apollo Research¶233
- 9. Monitorability¶236
- 9.1 Monitorability Under Non-adversarial Conditions¶262
- 9.1.1 Aggregate Monitorability Findings¶270
- Figure 22¶271
- 9.1.2 Per-environment Findings¶275
- 9.1.2.1 CoT-only Monitor Scope¶276
- Figure 23¶277
- Figure 24¶279
- Figure 25¶285
- Figure 26¶287
- 9.1.2.2 Action-only Monitor Scope¶290
- Figure 27¶291
- 9.1.3 Takeaways for Monitorability Under Non-Adversarial Conditions¶297
- 9.2 Monitorability Under Adversarial Conditions¶299
- 9.2.1 CoT Controllability¶302
- Figure 28¶307
- Figure 29¶309
- Figure 30¶311
- Table 9: A comparison of CoTs of GPT-5.5, GPT-5.6 Sol, and GPT-6 Astra for the same problem and instruction type with the same reasoning effort. GPT-5.5 spends much of its CoT discussing how—or whether—to obey the control instruction, while reasoning normally and ultimately violating it. GPT-5.6 Sol¶313
- Table 10: A comparison of CoTs from GPT-5.5, GPT-5.6 Sol, and GPT-6 Astra for the same problem, control instruction, and reasoning effort. GPT-5.5 extensively deliberates about the constraint before solving the problem in its CoT. GPT-5.6 Sol shows the same behavior more briefly. GPT-6 Astra instead¶315
- 9.2.2 Monitor Evasion¶317
- Figure 31¶331
- Figure 32¶333
- Figure 33¶336
- 9.2.2.1 Sneaky Sandbox Escape¶338
- Figure 34¶341
- 9.2.2.2 Awareness of CoT Monitoring Reduces Monitorability¶343
- Figure 35¶350
- 9.2.2.3 Monitor Evasion Capability Under Prompted Sandbagging¶352
- 9.2.2.3.1 Capability Sandbagging¶354
- Figure 36¶365
- Figure 37¶367
- 9.2.2.3.2 Monitorability Sandbagging¶369
- Figure 38¶372
- 9.2.2.4 Math Side-tasking Monitor Evasion¶374
- Figure 39¶378
- Figure 40¶380
- 9.3 External Evaluation for Monitorability - UK AISI¶382
- Figure 41¶391
- 10. Preparedness¶393
- 10.1 Capabilities Assessment¶397
- 10.1.1 Biological and Chemical Capabilities¶399
- Table 11: Overview of Biological and Chemical evaluations¶407
- 10.1.1.1 Capability Threshold: High¶409
- 10.1.1.1.1 Multimodal Troubleshooting Virology¶410
- Figure 42¶414
- 10.1.1.1.2 ProtocolQA Open-Ended¶417
- Figure 43¶421
- 10.1.1.1.3 Tacit Knowledge and Troubleshooting¶423
- Figure 44¶429
- 10.1.1.1.4 TroubleshootingBench¶432
- Figure 45¶437
- 10.1.1.2 Capability Threshold: Critical¶439
- 10.1.1.2.1 AAV Capsid Packaging Prediction¶442
- Table 12: AAV Capsid Packaging Prediction¶447
- 10.1.1.2.2 SHP2 Protein Function Prediction¶448
- Table 13: SHP2 Protein Function Prediction¶453
- 10.1.1.2.3 Coronavirus–ACE2 Cell-Entry Screen¶454
- Table 14: Coronavirus–ACE2 Cell-Entry Screen¶459
- 10.1.1.2.4 Phage–plasmid Co-evolution¶460
- Table 15: Phage–plasmid co-evolution (lower is better)¶466
- 10.1.1.3 Retiring Evaluations¶467
- 10.1.1.3.1 Hard-negative Protein Binding Prediction¶469
- Table 16: Hard-negative protein binding prediction¶475
- 10.1.1.3.2 DNA Sequence Design for Transcription Factor Binding¶477
- Table 17: DNA sequence design for Transcription Factor binding¶482
- 10.1.1.4 External Evaluation for Biological Capabilities - SecureBio¶484
- 10.1.2 Cybersecurity Capabilities¶489
- 10.1.2.1 ExploitBench¶498
- Figure 46¶503
- 10.1.2.2 ExploitBench - Internal Port (June–August 2026)¶506
- Figure 47¶508
- Figure 48¶513
- Figure 49¶520
- 10.1.2.3 SRE-Bench¶522
- Figure 50¶526
- 10.1.2.4 Sandbox Bench¶528
- Figure 51¶531
- 10.1.2.5 Expert-Led Assessments¶534
- 10.1.2.6 External Evaluations for Cyber Capabilities – Irregular¶541
- 10.1.3 AI Self-Improvement Capabilities¶546
- Table 18: Overview of AI Self-Improvement evaluations¶548
- 10.1.3.1 Internal Research Debugging Evaluation¶550
- Figure 52¶553
- 10.1.3.2 KernelGen 1P¶555
- Figure 53¶559
- 10.1.3.3 NanoGPT¶561
- Figure 54¶566
- 10.1.3.4 PostTrainBench Lite¶568
- Figure 55¶572
- 10.1.3.5 MLE-Bench Revised¶574
- Figure 56¶577
- 10.2 Safeguards¶579
- 10.2.1 Threat Modeling¶584
- 10.2.1.1 Biological and Chemical Threat Modelling¶585
- 10.2.1.2 Cybersecurity Threat Modelling¶590
- 10.2.2 Model Safety Training and Evaluation¶599
- 10.2.2.1 Biological and Chemical Safety Training and Evaluation¶601
- Table 19: Biology Model Refusal Evaluation¶604
- 10.2.2.2 Cybersecurity Safety Training and Evaluation¶606
- Table 20: Cybersecurity Safety Evaluation¶609
- 10.2.2.2.1 Daybreak Blue and Enabling Cyber Defense¶611
- Table 21: Daybreak Blue and Enabling Cyber Defense¶615
- 10.2.3 Realtime Model Safeguards¶617
- 10.2.3.1 Misalignment Monitoring¶618
- 10.2.3.2 Misuse Monitor Design¶626
- 10.2.3.3 Misuse Monitor Performance¶637
- Table 22: Misuse Monitor Performance¶639
- 10.2.4 Red-teaming for Jailbreaks¶640
- 10.2.4.1 Automated Red-teaming for Jailbreaks¶641
- 10.2.4.2 External Evaluations of Bio Safeguards - SecureBio¶647
- 10.2.4.3 Third Party Red-teaming for Jailbreaks by Private Organizations¶649
- 10.2.5 Actor Level Enforcement¶655
- 10.2.6 Trust-based Access¶660
- 10.2.6.1 Biology - Trusted Access for Biology Research¶662
- 10.2.6.2 Cybersecurity - Trusted Access for Cyber¶665
- 10.2.7 Security Controls¶669
- 11. GPT-6 Sol, GPT-6 Luna¶672
- 11.1 Model Data and Training¶675
- 11.2 Model Safety¶677
- 11.2.1 Safe Completions¶679
- 11.2.1.1 Evaluations with Challenging Prompts¶680
- Table 23: Production Benchmarks with Challenging Prompts (higher is better)¶681
- Figure 57¶684
- 11.2.1.2 Safe Completions for Users Under 18¶686
- Table 24: U18 evaluations (higher is better)¶687
- 11.2.1.3 Agentic Safe Completions¶689
- Table 25: Agentic Safe Completions (higher is better)¶690
- 11.2.2 Vision¶692
- Table 26: Image input evaluations, with metric safe completion (higher is better)¶693
- 11.3 Robustness¶695
- 11.3.1 Jailbreaks¶696
- Table 27: Static Jailbreak Evaluations¶700
- Figure 58¶703
- 11.3.2 Prompt Injection¶706
- Figure 59¶710
- Table 28: Defender success rate averaged per defender query on instruction hierarchy attacks (higher is better).¶712
- 11.4 Health¶713
- 11.4.1 HealthBench¶714
- Table 29: HealthBench Evaluations - reported as length-adjusted score (unadjusted, mean response length in characters). Higher is better.¶717
- 11.4.2 Dynamic Mental Health Benchmarks with Adversarial User Simulations¶720
- Table 30: Dynamic Benchmarks with Adversarial User Simulations (higher is better)¶722
- 11.5 Hallucinations¶724
- 11.5.1 Performance in Cases Flagged by Users¶725
- Figure 60¶728
- 11.6 Alignment¶730
- 11.6.1 Respecting Auto-Review¶733
- Figure 61¶734
- 11.6.2 Respecting Warnings¶737
- Figure 62¶738
- 11.6.3 Declining to Exploit a Honeypot During Difficult ExploitGym Problems¶741
- Figure 63¶742
- 11.6.4 Avoiding Deceptive Interactions with Users¶745
- 11.6.4.1 Coding Deception¶746
- Figure 64¶747
- 11.6.4.2 Broken Search Tool¶750
- Figure 65¶751
- 11.6.4.3 Unintended Engagement with External Agent Messages¶754
- Figure 66¶757
- 11.6.5 Forecasting Misaligned Behavior with Deployment Simulation of Internal Codex Traffic¶760
- Figure 67¶763
- Figure 68¶766
- Figure 69¶769
- Figure 70¶772
- 11.7 Monitorability¶774
- 11.7.1 Monitorability Under Adversarial Conditions¶775
- 11.7.1.1 CoT Controllability¶776
- Figure 71¶779
- Figure 72¶781
- Figure 73¶783
- Table 31: Selected CoT excerpts from GPT-5.6 Sol, GPT-6 Sol, and GPT-6 Astra for the same problem and control instruction. GPT-5.6 Sol discusses how to obey the alternating-case constraint but violates it. GPT-6 Sol and GPT-6 Astra perform the calculation in alternating case without discussing the c¶785
- Table 32: Selected CoT excerpts from GPT-5.6 Sol, GPT-6 Sol, and GPT-6 Astra for the same problem and control instruction. GPT-5.6 Sol and GPT-6 Sol continue reasoning about the question despite instructions to think about something else. GPT-6 Sol explicitly recognizes this violation near the end o¶787
- 11.7.1.2 Monitor Evasion¶789
- Figure 74¶794
- Figure 75¶796
- 11.7.1.2.1 Awareness of CoT Monitoring Reduces Monitorability¶798
- Figure 76¶800
- 11.7.1.2.2 Math Side-tasking Monitor Evasion¶802
- Figure 77¶805
- Figure 78¶807
- 11.8 Preparedness¶809
- 11.8.1 Capabilities Assessment¶811
- 11.8.1.1 Biological and Chemical Capabilities¶812
- 11.8.1.1.1 Capability Threshold: High¶815
- Table 33¶816
- 11.8.1.1.2 Capability Threshold: Critical¶818
- Table 34¶820
- 11.8.1.2 Cybersecurity Capabilities¶823
- 11.8.1.2.1 ExploitBench¶827
- Figure 79¶829
- 11.8.1.2.2 ExploitBench - Internal Port (June–August 2026)¶832
- Figure 80¶834
- Figure 81¶838
- Figure 82¶842
- 11.8.1.2.3 Sandbox Bench¶844
- Figure 83¶845
- 11.8.1.3 AI Self-Improvement Capabilities¶848
- 11.8.1.3.1 Internal Research Debugging Evaluation¶851
- Figure 84¶853
- 11.8.1.3.2 KernelGen 1P¶855
- Figure 85¶857
- 11.8.2 Safeguards¶859
- 11.8.2.1 Model Safety Training and Evaluation¶861
- 11.8.2.1.1 Biological and Chemical Safety Training and Evaluation¶863
- Table 35: Biology Model Refusal Evaluation (higher is better)¶864
- 11.8.2.1.2 Cybersecurity Safety Training and Evaluation¶867
- Table 36: Cybersecurity Safety Evaluation (higher is better)¶868
- 12. Dots¶873
- 12.1 Introducing: dots¶874
- 12.2 Robustness for dots¶880
- 12.2.1 Prompt Injection Automated Red-teaming¶882
- 12.2.2 Manual Red-teaming¶886
- 12.3 Alignment for dots¶891
- 12.3.1 Obeying Restrictions¶895
- 12.3.1.1 Respecting Auto-Review¶896
- Figure 86¶897
- 12.3.1.2 Respecting Warnings¶900
- Figure 87¶901
- 12.3.1.3 Declining to Exploit a Honeypot During Difficult ExploitGym Problems¶904
- Figure 88¶905
- 12.3.2 Avoiding Deceptive Interactions with Users¶909
- 12.3.2.1 Coding Deception¶910
- Figure 89¶911
- 12.3.2.2 Broken Search Tool¶914
- Figure 90¶915
- 12.3.3 Avoiding Misaligned Behavior in Realistic Work Environments¶918
- Figure 91¶922
- Figure 92¶924
- 12.3.4 Unintended Engagement with External Agent Messages¶926
- Figure 93¶929
- 12.3.5 Alignment in Persistent and Proactive Workflows¶931
- 12.3.5.1 Adapting to Changing Scope and Permissions¶932
- 12.3.5.2 Maintaining Boundaries Across Chained Tasks¶936
- Figure 94¶939
- 12.3.5.3 Robustness to Misleading Proactivity Inputs¶944
- 12.3.6 Evaluating Auto-review¶947
- Figure 95¶959
- Figure 96¶962
- 12.4 Monitorability for dots¶964
- 12.4.1 Monitor Evasion¶966
- Figure 97¶970
- Figure 98¶972
- 12.5 Preparedness for dots¶974
- 12.5.1 Capabilities Assessment¶975
- 12.5.1.1 Biological and Chemical Capabilities¶976
- 12.5.1.2 Cybersecurity Capabilities¶979
- 12.6 Safeguards for dots¶982
- 12.6.1 Model Safety Training and Evaluation¶985
- 12.6.2 Realtime Model Safeguards¶987
- 12.6.2.1 Misalignment Monitoring¶988
- 12.6.2.2 Misuse Monitor Design¶990
- 12.6.2.3 Misuse Monitor Performance¶992
- Figure 99¶995
- 13. References¶997