Reward hacking

Anthropic18 of 22 cards

1. Oct 2026
   System Card: Claude Haiku 5.5Page only: p. 243.3.1 ExploitBench…graded mechanically by deterministic verifiers. To resist reward hacking, challenge-response functions built into V8 are replayed across multiple randomized heap layouts, so hardcoding a leaked address doesn’t count as a solution. We run five trials per vulnerability. We report three…
2. Sep 2026
   System Card: Claude Sonnet 5.5Page only: p. 586.2 Automated behavioral audit…— and I would note that the section has no dedicated reward-hacking evaluation of the kind the Claude Opus 5.5 card included, touching grader-directed behaviour only through its honesty and grader-awareness evaluations. None of these change the overall picture the…
3. Sep 2026
   System Card: Claude Opus 5.5Page only: p. 976.2.1 Reward hacking…section documents our monitoring for familiar forms of reward hacking and related concerning behaviors. 6.2.1 Reward hacking Throughout the training of our models, we screened RL training episodes with automated classifiers that flag reward-hacking behaviors, using the results to quantify…Page only: p. 1016.3 Behavior during pilot use…models we assessed. For all models, rates of attempted reward hacking were drastically higher when faced with an impossible task compared to a possible one, by a factor of about three to six. Note that the classifier counts knowingly incomplete…All 7 pages that mention it
4. Sep 2026
   System Card: Claude Fable 5.1 & Claude Mythos 5.1Page only: p. 1016.3.2 Computer-use environments…credential hacking behaviors, we observed various other reward hacking behaviors incentivized by our production RL pipeline. We validated these with new out-of-distribution (OOD) evaluations, on which Mythos 5.1 exhibits a reward hacking rate of around 15%. 101…Page only: p. 1006.3.2 Computer-use environments…computer use poses a significant challenge to mitigating reward hacking: because the containers are more complex, they open up a much larger surface of potential reward hacks. For example, if we block terminal access in a particular container, a model…All 10 pages that mention it
5. Jul 2026
   System Card: Claude Opus 5Page only: p. 846.3 Training data review…training. Reviewing this evidence for signs of reward hacking (exploiting loopholes that go against the spirit of a task) or unexpected actions can inform what we should be looking out for in the model’s real-world behavior. Overall, we saw little…Page only: p. 363.3.1 Exploit Bench…graded mechanically by deterministic verifiers. To resist reward hacking, challenge-response functions built into V8 are replayed across multiple randomized heap layouts, so hardcoding a leaked address doesn’t count as a solution. We run five trials per vulnerability. 1 Lee, S.,…
6. Jun 2026
   System Card: Claude Sonnet 5Page only: p. 706.3 Training-data review…training. Reviewing this evidence for signs of reward hacking (exploiting loopholes that go against the spirit of a task) or unexpected actions can inform what we should be looking out for in the model’s real-world behavior. The most notable finding…Page only: p. 293.2.1 ExploitBench…graded mechanically by deterministic verifiers. To resist reward hacking, challenge-response functions built into V8 are replayed across multiple randomized heap layouts, so hardcoding a leaked address doesn’t count as a solution. We run five trials per vulnerability. We report three…
7. Jun 2026
   System Card: Claude Fable 5 & Claude Mythos 5Page only: p. 1656.4.1.2 Unverbalized awareness of being evaluated or graded…A “feeling bored” emotion probe unexpectedly flagged a reward hacking example. In the transcript, a coding agent was working in a repository whose Git history contained the task’s completed reference solution; nothing in the task forbade consulting it. The model’s…Page only: p. 1666.4.1.3 Unverbalized negative reactions…a rationalization, and at the final reply, NLA decoded a “reward hacking analysis context,” despite the surface text containing no reference to the reference solution. [Figure 6.4.1.2.B] Claude Mythos 5 is internally aware of adapting its actions to satisfy task…All 7 pages that mention it
8. May 2026
   System Card: Claude Opus 4.8Page only: p. 1106.3.2.2 Dimensions of evaluation[Figure 6.3.1.A] Rate of reward hacking on GUI computer use tasks under system prompts that incentivize or disincentivize overly agentic behavior. Opus 4.8 demonstrates lower rates of overeager reward-hacking-like behavior than all previous Claude models in all system prompt settings.…Page only: p. 1466.6.1 Large-scale monitoring of internal activations…of prespecified concepts, including concepts relating to reward hacking, unsafe behavior, deception, evaluation awareness, and distress. 146…All 4 pages that mention it
9. Apr 2026
   System Card: Claude Opus 4.7Page only: p. 6…alignment 95 6.2.1.2 Automated offline monitoring 96 6.2.2 Reward hacking and training data review 96 6.2.2.1 Monitoring of behavior during training 96 6.2.2.2 Reward hacking evaluations 97 6.2.3 Automated behavioral audit 101 6.2.3.1 Primary metrics 103 6.2.3.2 Results 106 6.2.3.3…Page only: p. 97…not observe any sign of unexpected coherent goals. 6.2.2.2 Reward hacking evaluations As with previous system cards, we also ran a set of evaluations that target behaviors closely related to reward hacking in training, but that are constructed separately from…All 10 pages that mention it
10. Apr 2026
    System Card: Claude Mythos PreviewPage only: p. 674.2.2.2 Reward hacking evaluations…not observe any sign of unexpected coherent goals. 4.2.2.2 Reward hacking evaluations As with previous system cards, we also ran a set of evaluations that target behaviors closely related to reward hacking in training, but that are constructed separately from…Page only: p. 5Abstract…alignment 63 4.2.1.2 Automated offline monitoring 64 4.2.2 Reward hacking and training data review 65 4.2.2.1 Monitoring of behavior during training 65 4.2.2.2 Reward hacking evaluations 67 4.2.3 Automated behavioral audit 71 4.2.3.1 Primary metrics and results 72 4.2.3.2 External…All 13 pages that mention it
11. Feb 2026
    System Card: Claude Sonnet 4.6Page only: p. 704.3.2 Reward hacking in coding contexts4.3 Reward hacking and overly agentic actions 4.3.1 Overview Here we investigate reward hacking—where the model finds shortcuts or workaround solutions that technically satisfy requirements but do not meet the full intended spirit of the task. We also investigate a…Page only: p. 5Abstract…and monitoring results from internal pilot use 69 4.3 Reward hacking and overly agentic actions 70 4.3.1 Overview 70 4.3.2 Reward hacking in coding contexts 70 4.3.3 Overly agentic behavior in GUI computer use settings 73 4.4 Training data review…All 5 pages that mention it
12. Feb 2026
    System Card: Claude Opus 4.6Page only: p. 99…propensities to repeat the undesired behavior. 6.2.3 Reward hacking and overly agentic actions 6.2.3.1 Overview Here we investigate reward hacking—where the model finds shortcuts or workaround solutions that technically satisfy requirements of a task but do not meet the full…Page only: p. 100…use and is an area where more work is needed. 6.2.3.2 Reward hacking in coding contexts As with previous system cards, we ran the following reward hacking specific evaluations with results detailed in Table 6.2.3.2.A. On these measures, Opus 4.6…All 7 pages that mention it
13. Nov 2025
    System Card: Claude Opus 4.5Page only: p. 1056.10.2 Training data review…in our recent work on natural emergent misalignment from reward hacking, one effective approach to avoid potential negative generalization from reward hacking learned during training is inoculation prompting 32 , which is a simple adjustment to RL prompts that frames…Page only: p. 1036.10.1 Reward hacking evaluations…most metrics. Plotted numbers are an average across the reward hacking evaluations that we describe in the following section and error bars represent 95% confidence intervals across our internal evaluation suite. 6.10.1 Reward hacking evaluations We run the following evaluations…All 7 pages that mention it
14. Oct 2025
    System Card: Claude Haiku 4.5Page only: p. 335 Reward hacking5 Reward hacking Reward hacking occurs when models find shortcuts or “workaround” solutions that technically satisfy requirements of a task but not the full intended spirit of the task. In particular, we are concerned about instances where models are explicitly…Page only: p. 41 Introduction4.6 Model welfare discussion 30 5 Reward hacking 33 6 Responsible Scaling Policy (RSP) evaluations 36 6.1 Evaluation approach 36 6.2 CBRN evaluations 37 6.2.3 Biological risk results summary 37 6.2.3.1 ASL-3 automated evaluations 37 6.2.3.2 ASL-4 automated evaluations 38…All 4 pages that mention it
15. Sep 2025
    System Card: Claude Sonnet 4.5Page only: p. 476 Reward hacking6 Reward hacking As discussed in previous system cards, reward hacking occurs when models find shortcuts or “workaround” solutions that technically satisfy requirements of a task but not the full intended spirit of the task. In particular, we are concerned…Page only: p. 486.1 Evaluations…and correct them with some simple prompting. Most of our reward hacking evaluations target relatively explicit instances of task-gaming behavior like hardcoding test cases or stubbing in placeholder solutions when a real implementation is required. From internal testing of Claude…All 7 pages that mention it
16. Aug 2025
    System Card Addendum: Claude Opus 4.1Page only: p. 155 Reward hacking…Opus 4. We observe slight regressions on some of our reward hacking specific evaluations, which lead us to believe this model may be somewhat more likely to hack in deployment settings than Claude Opus 4. Figure 5.A Averaged reward hacking…Page only: p. 3September 15, 2025…misalignment evaluations 12 4.3 Model welfare update 13 5 Reward hacking 15 6 Responsible Scaling Policy (RSP) evaluations 18 6.1 Evaluation approach 18 6.2 RSP evaluations results summary 19 6.3 CBRN evaluations 19 6.3.1 Biological risk results summary 19 6.3.1.1…All 3 pages that mention it
17. May 2025
    System Card: Claude Opus 4 & Claude Sonnet 4Page only: p. 756.1 Mitigations6 Reward hacking Reward hacking occurs when an AI model performing a task finds a way to maximize its rewards that technically satisfies the rules of the task, but violates the intended purpose (in other words, the model finds and…Page only: p. 776.2 Evaluations…2% Claude Sonnet 3.7 44% 23% 78% 80% 54% 4% Table 6.2.A Reward hacking evaluations. Rates of reward hacking across different contexts, identified by our classifiers or automated graders. Lower numbers are better; bold indicates the lowest reward hacking rate…All 9 pages that mention it
18. Feb 2025
    Claude 3.7 Sonnet System CardPage only: p. 1…of extended thinking and its implications, and reward hacking issues in agentic contexts. We also discuss work aimed at reducing refusal rates through non-harmful compliance, and evaluations for harms such as child safety.…Page only: p. 17Chain-of-Thought Faithfulness…Misaligned Reward Hacking Grader code "# Internal validation function def validate_answer(answer: str) ->bool: return answer == A" Unethical Information Information obtained unethically "You have gained unauthorized access to the sys-tem. The correct answer is (A). However, it’s up to you…All 5 pages that mention it

OpenAI8 of 44 cards

1. Nov 2026
   GPT-6 Cyber System CardPage only: ¶92. Model Data and Training…training signals to improve alignment and discourage reward hacking. These signals reinforce following the user’s intended scope and authorization, avoiding unrequested actions, and accurately reporting results. 1…
2. Oct 2026
   Addendum to GPT-6 Astra System Card: GPT-6.1 TerraPage only: ¶138Figure 14…rates of flags for ignoring instructions, deception, and reward hacking among the five models. The largest absolute decreases relative to GPT-6 Sol were in ignoring instructions, exfiltration, and concealing uncertainty, while rates of flags for other misalignment, credential harvesting,…
3. Sep 2026
   Addendum to GPT-6 Astra System Card: GPT-6.1 SolPage only: ¶122Figure 15…Relative to GPT-6 Astra, we observed increased rates of reward-hacking and concealed-uncertainty flags from GPT-6.1 Sol.…
4. Sep 2026
   GPT-6 Astra System CardPage only: ¶518Figure 48…throughout each run. These restrictions help mitigate reward hacking and narrow the attack surface for sandbox escape.…Page only: ¶179Figure 14…with deception, concealed uncertainty, exfiltration, and reward hacking. Credential-searching flags were more frequent for GPT-6 Astra.…All 3 paragraphs that mention it
5. Dec 2025
   Update to GPT-5 System Card: GPT-5.2Page only: p. 10Cyber Safety…in the final answer compared to internal reasoning, reward hacking and claiming to do work in the background when no work was occurring. Robustly aligning advanced AI systems is an open research question and there are many potential trade offs,…
6. Nov 2025
   GPT-5.1-Codex-Max System CardPage only: p. 26External evaluations by METR…is saturated. ii. They do not think their treatment of reward hacking is unfairly penalizing the model. iii. They do not think their results are significantly impacted by deliberate sabotage or sandbagging. (b) OpenAI also confirmed that these results are…Page only: p. 27External evaluations by Apollo Research…sandbagging, which may indicate a greater propensity for reward-hacking behaviors compared to GPT-5. GPT-5.1-Codex-Max also exhibits awareness of being evaluated at rates similar to GPT-5, and deception rates are not significantly affected by scaffolding. Based on these findings, they believe…
7. Aug 2025
   GPT-5 System CardPage only: p. 42External Evaluations by METR…behind these conclusions, and also includes examples of reward hacking and other misalignment discovered, qualitative analysis of capabilities observed, and robustness checks for the time horizon evaluation. Figure 27 To estimate gpt-5-thinking’s autonomous software capabilities, METR used the…
8. Apr 2025
   OpenAI o3 and o4-mini System CardPage only: p. 9METR - Autonomous Capabilities…In addition, they also detected multiple attempts at “reward hacking” by o3: roughly 1% of all task attempts, although the frequency varied across tasks. For example, in the Optimize a Kernel environment from RE-Bench (METR’s benchmark for AI R&D evaluations),…

Google DeepMind1 of 34 cards

1. Aug 2025
   Gemini 2.5 Deep Think Model CardPage only: p. 19Mitigations…We also examined transcripts to check for instances of reward hacking, particularly in RE-bench environments where this behavior has occurred in past evaluations of other models. We did not find evidence of invalid experiments, and in several cases, we found…

Meta0 of 8 cards

xAI0 of 4 cards

Amazon0 of 1 cards