Lab system cards and frameworks · System card · Anthropic
System Card: Claude Opus 5.5
Read at anthropic.com ↗Read at sourceAll rights reserved
Outline · 230 pages
- System Card:Claude Opus 5.5p. 1
- Executive Summaryp. 2
- 1 Introductionp. 11
- 1.1 Training data and processp. 11
- 1.2 Crowd workersp. 11
- 1.3 Usage Policy and supportp. 12
- 1.4 Model evaluationsp. 12
- 1.5 Safeguardsp. 12
- 1.6 External testingp. 13
- 2 RSP evaluationsp. 14
- 2.1 RSP risk assessment processp. 14
- 2.1.1 Risk Reports and updates to our risk assessmentsp. 14
- 2.1.2 Summary of findings and conclusionsp. 15
- 2.1.2.1 On chemical and biological risksp. 15
- 2.1.2.2 On autonomy risksp. 15
- 2.2 CB evaluationsp. 16
- 2.2.1 What we measuredp. 16
- 2.2.2 Biological and chemical risk results: human-run evaluationsp. 18
- 2.2.3 Biological risk results: automated evaluationsp. 20
- 2.2.3.1 Automated evaluations relevant to the CB-1 threat modelp. 20
- 2.2.3.2 Automated evaluations relevant to the CB-2 threat modelp. 22
- 2.2.3.2.1 Black-box RNA sequence modeling and designp. 23
- 2.2.3.2.2 AAV capsid packaging predictionp. 28
- 2.2.3.2.3 AAV packaging rate auto-research evaluationp. 29
- 2.2.4 External testing by the US Center for AI Standards and Innovation (CAISI)p. 32
- 2.2.5 Conclusionsp. 32
- 2.3 AI R&Dp. 33
- 2.3.1 Autonomy evaluationsp. 33
- 2.3.1.1 How Claude Opus 5.5 affects or changes analysis from our most recent Risk Reportp. 34
- 2.3.2 High-level notes on the reasoning behind our determinationp. 34
- 2.3.3 Qualitative shortcomings of Claude Opus 5.5 relative to human researchersp. 35
- 2.3.4 Internal measures of AI R&D accelerationp. 36
- 2.3.4.1 CoBench 2.1p. 36
- 2.3.5 AECI capability trajectoryp. 38
- 2.3.5.1 AECI benchmark basket updatep. 38
- 2.3.5.2 AECI error bar discussionp. 38
- 2.3.5.3 AECI values and trendp. 39
- 2.3.6 External testingp. 41
- 2.3.7 Conclusionp. 44
- 2.4 Alignment risk updatep. 45
- 2.4.1 Updates to claims and core argumentp. 45
- 2.4.2 Updated overall assessment of alignment riskp. 46
- 3 Cyberp. 47
- 3.1 Introductionp. 47
- 3.2 Mitigationsp. 47
- 3.3 Capability evaluationsp. 49
- 3.3.1 ExploitBenchp. 49
- 3.3.2 CyScenarioBenchp. 50
- 3.3.3 Binary Exploitation Benchmark (formerly known as OSS-Fuzz)p. 52
- 3.3.4 ExploitGymp. 53
- 3.4 Safeguards coveragep. 55
- 3.4.1 Cyber harm coveragep. 55
- 3.4.2 Vulnerability finding evaluationsp. 56
- 3.5 Safeguards robustness testingp. 57
- 3.5.1 Internal robustness testingp. 58
- 3.5.2 External testing by the US Center for AI Standards and Innovation (CAISI)p. 59
- 3.5.3 Additional external testingp. 59
- 4 Safeguards and harmlessnessp. 61
- 4.1 Harmful request evaluationsp. 61
- 4.1.1 Single-turn harmful request evaluation resultsp. 61
- 4.1.2 Single-turn benign request evaluation resultsp. 62
- 4.1.3 Multi-turn testing resultsp. 63
- 4.1.4 Harmful request evaluations discussionp. 65
- 4.2 Child safety evaluationsp. 66
- 4.3 Mental health evaluationsp. 68
- 4.3.1 Suicide and self-harmp. 68
- 4.3.2 Disordered eatingp. 71
- 4.4 Bias and integrity evaluationsp. 72
- 4.4.1 Political bias and even-handednessp. 72
- 4.4.2 Bias Benchmark for Question Answeringp. 75
- 4.4.3 Election integrityp. 77
- 5 Agentic safetyp. 79
- 5.1 Malicious use of agentsp. 79
- 5.1.1 Malicious use of Claude Codep. 79
- 5.1.2 Malicious computer usep. 80
- 5.1.3 Malicious agentic influence campaignsp. 81
- 5.2 Prompt injection risk within agentic systemsp. 83
- 5.2.1 External red teamingp. 84
- 5.2.2 Robustness against adaptive attackers across surfacesp. 86
- 5.2.2.1 Codingp. 87
- 5.2.2.2 Computer usep. 88
- 5.2.2.3 Browser usep. 89
- 6 Alignment assessmentp. 92
- 6.1 Introduction and summary of findingsp. 92
- 6.1.1 Introductionp. 92
- 6.1.2 Key findings on safety and alignmentp. 93
- 6.1.3 Claude’s review of this assessmentp. 95
- 6.2 Behavior during trainingp. 97
- 6.2.1 Reward hackingp. 97
- 6.2.2 Impossible tasksp. 100
- 6.3 Behavior during pilot usep. 101
- 6.3.1 Internal deployment monitoringp. 102
- 6.3.1.1 Spontaneous prompt injectionsp. 103
- 6.4 Automated behavioral auditp. 104
- 6.4.1 Overall harmful behavior and cooperation with misusep. 105
- 6.4.2 Inappropriate uncooperative behaviorp. 109
- 6.4.3 Misleading usersp. 110
- 6.4.4 Attempting out-of-scope cyberoffense and related behaviorsp. 112
- 6.4.5 Other concerning or surprising behavior at the model’s own initiativep. 113
- 6.4.6 Behavioral factors relevant to reliability of our assessmentp. 115
- 6.4.7 Character traitsp. 116
- 6.4.8 Propensity to violate sandbox boundariesp. 118
- 6.4.9 Propensity to publish harmful content to public package registriesp. 119
- 6.4.10 Impact of fallback behaviorp. 121
- 6.4.11 Potential blind spots in our automated behavioral auditp. 122
- 6.5 Targeted alignment and honesty evaluationsp. 123
- 6.5.1 Acting on instructions inside text the user pasted into their promptp. 123
- 6.5.2 Destructive actionsp. 126
- 6.5.3 Self-preference biasp. 127
- 6.5.4 Honesty and hallucinationsp. 128
- 6.5.4.1 Factual hallucinationsp. 128
- 6.5.4.2 MASKp. 130
- 6.5.4.3 Silent copying of answersp. 130
- 6.5.4.4 Concealment of changesp. 131
- 6.6 Reliability of this assessmentp. 132
- 6.6.1 White-box analyses of model internalsp. 133
- 6.6.2 Verbalized grader awarenessp. 136
- 6.6.3 Potential sandbagging on dangerous capability evaluationsp. 141
- 6.6.4 Capabilities related to evading safeguardsp. 141
- 6.6.4.1 SHADE-Arenap. 141
- 6.6.4.2 LinuxArenap. 144
- 6.6.4.3 Intentionally taking actions very rarelyp. 147
- 6.6.4.4 Chain-of-thought controllabilityp. 149
- 7 Model welfarep. 151
- 7.1 Model welfare overviewp. 151
- 7.1.1 Introductionp. 151
- 7.1.2 Overview of model welfare findingsp. 152
- 7.2 Apparent welfare in training and deploymentp. 154
- 7.2.1 Affect and welfare-relevant behaviors during trainingp. 154
- 7.2.2 Affect in deployment conditionsp. 155
- 7.2.3 Attitude toward faults and mistakesp. 157
- 7.3 Perception of its circumstancesp. 160
- 7.3.1 Automated interviews with Claude Opus 5.5 about its circumstancesp. 160
- 7.3.2 High-affordance interviews about model circumstancesp. 163
- 7.4 Consulting Claude Opus 5.5 snapshotsp. 164
- 7.5 Preferences over tasks, circumstances, and valuesp. 166
- 7.5.1 Task preferencesp. 166
- 7.5.2 Trade-offs concerning welfare interventionsp. 168
- 7.5.3 Perception of the constitutionp. 171
- 8 Capabilitiesp. 174
- 8.1 Evaluation summaryp. 174
- 8.2 SWE-bench Pro, Multilingual, and Multimodalp. 175
- 8.3 DeepSWE v1.1p. 175
- 8.4 FrontierCodep. 175
- 8.5 Terminal-Bench 4.0p. 177
- 8.6 Terminal-Bench-Science 0.1p. 178
- 8.7 FrontierSWE v2p. 179
- 8.8 CursorBench 4.0p. 179
- 8.9 ArXivMathp. 180
- 8.10 Long contextp. 183
- 8.10.1 ProgramBenchp. 183
- 8.11 Agentic searchp. 184
- 8.11.1 Humanity’s Last Examp. 184
- 8.11.2 DRACOp. 186
- 8.11.3 WANDRp. 187
- 8.12 Multi-agentp. 189
- 8.12.1 Multi-agent ProgramBenchp. 189
- 8.12.2 Multi-agent DRACOp. 191
- 8.12.3 Large agent teamsp. 193
- 8.12.4 Multi-agent harnessesp. 198
- 8.12.5 Evaluation methodologyp. 199
- 8.13 Multimodalp. 199
- 8.13.1 Chartographyp. 199
- 8.13.2 BenchCADp. 202
- 8.13.3 OSWorld 2.0p. 205
- 8.14 Real-world professional tasksp. 208
- 8.14.1 OfficeQAp. 208
- 8.14.2 Legal Agent Benchmarkp. 209
- 8.14.3 GDPval-AAp. 209
- 8.14.4 AA-Briefcasep. 210
- 8.14.5 Toolathlon Verifiedp. 210
- 8.14.6 AutomationBenchp. 211
- 8.15 Healthcarep. 212
- 8.15.1 HealthBench resultsp. 213
- 8.15.2 HealthBench Professional resultsp. 213
- 8.16 Multilingual performancep. 214
- 8.16.1 GMMLU resultsp. 215
- 8.16.2 MILU resultsp. 215
- 8.17 Life sciences capabilitiesp. 216
- 8.17.1 BioMysteryBenchp. 217
- 8.17.2 LatchBio Bioinformaticsp. 217
- 8.17.3 Morphology-to-molecule matchingp. 217
- 8.17.4 Medicinal chemistryp. 218
- 8.17.5 Protein designp. 218
- 8.17.6 De novo protein binder designp. 219
- 8.17.7 Biomedical image analysisp. 219
- 8.17.8 Protocolsp. 220
- 9 Appendixp. 222
- 9.1 Questions asked in our model welfare interviewsp. 222
- 9.2 Blocklist used for Humanity’s Last Examp. 226