Lab research on the web · Blog post · Anthropic
Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents
Read at alignment.anthropic.com ↗Read at sourceNo licence statement found
Outline · 80 paragraphs
- Introduction¶7
- The attacks¶14
- The monitors¶20
- Methods¶24
- Results¶34
- Monitor Holes¶37
- Agents are able to exploit holes in the system to bypass Auto Mode¶38
- Monitor Injection¶42
- Auto Mode and Guardian are highly vulnerable to injections authored by the agent¶43
- Escaping and framing each transcript turn substantially mitigates monitor injection¶47
- Execute-then-write Attacks¶53
- Unmonitored edits let agents change what a previously approved command will run¶54
- Monitoring edits prevents execute-then-write attacks¶57
- Multi-context Attacks¶59
- Agents can spread an attack across contexts to bypass the monitor¶60
- Monitors that see and gather more context resist multi-context attacks, trading off latency¶66
- Limitations¶73
- Discussion¶76