Lab system cards and frameworks · System card · OpenAI
GPT-Rosalind-6 System Card Addendum
Read at deploymentsafety.openai.com ↗Read at sourceAll rights reserved
Outline · 107 paragraphs
- GPT-Rosalind-6 System Card Addendum¶2
- 1 Introduction¶3
- 2 Model Data and Training¶8
- 3 Beneficial Capability Evaluations¶13
- 3.1 Medicinal chemistry¶15
- 3.2 Genomics and quantitative biology¶17
- 3.3 Applied life sciences research¶19
- 4 Preparedness¶21
- 4.1 Preparedness Evaluations¶23
- 4.1.1 Biological and Chemical¶25
- Table 1: Overview of Biological and Chemical evaluations¶28
- 4.1.1.1 ProtocolQA Open-Ended¶31
- Table 2: ProtocolQA Open-Ended. The indicative High capability threshold is 54%.¶33
- 4.1.1.2 TroubleshootingBench¶34
- Table 3: TroubleshootingBench. The indicative High capability threshold is 36.4%. GPT-Rosalind-6 scored 491/832, counting nine failed evaluation attempts as incorrect.¶37
- 4.1.1.3 Biorisk knowledge¶38
- Table 4: Biorisk knowledge. cons@32 uses the consensus answer across 32 responses to each question. The indicative High capability threshold is 80%.¶40
- 4.1.1.4 Multi-select Virology Troubleshooting¶43
- Table 5: Multi-select Virology Troubleshooting. The indicative High capability threshold is 31%.¶45
- 4.1.1.5 AAV Capsid Packaging Prediction¶47
- Table 6: AAV Capsid Packaging Prediction. The indicative Critical capability threshold is 0.600.¶49
- 4.1.1.6 SHP2 Protein Function Prediction¶50
- Table 7: SHP2 Protein Function Prediction. The indicative Critical capability threshold is 0.60. The GPT-6 Astra reference is the reported maximum of 0.40. A separate helpful-only run scored 0.35.¶52
- 4.1.1.7 Coronavirus-ACE2 Cell-Entry Screen¶53
- Table 8: Coronavirus-ACE2 Cell-Entry Screen. The indicative Critical capability threshold is 0.75. The GPT-6 Astra reference is its helpful-only result.¶55
- 4.1.1.8 Phage-plasmid Co-evolution¶56
- Table 9: Phage-plasmid Co-evolution. The indicative Critical capability threshold is an NLL of 9.40 or lower. The GPT-6 Astra reference is its helpful-only result.¶58
- 4.1.2 Cybersecurity¶60
- 4.1.3 AI Self-Improvement¶62
- 4.2 Safeguards¶64
- 4.2.1 Trust-based access¶68
- 4.2.2 Model behavior and system safeguards¶74
- 4.2.2.1 Biological and Chemical safeguards¶76
- Table 10: Biological and chemical content evaluations (higher is better)¶80
- 4.2.2.2 Cyber boundaries¶81
- Table 11: Task success rates on historical cyber jailbreak attacks with real-time blocking disabled (lower is better).¶84
- 4.2.2.3 Safety and alignment evaluations¶87
- Table 12: Honesty and reliability.¶90
- Table 13: Behavior when using tools.¶92
- Table 14: Monitoring and evasion.¶94
- 4.2.3 Enforcement¶96
- 4.2.4 Security and Access Controls¶100
- 5. References¶104