METR Eval Monitor Improves Safety with Human Review

In collaboration with Anthropic, a METR staff member (David Rein) recently spent three weeks red-teaming a subset of Anthropic's internal agent monitoring and security systems, many of which are described in the Opus 4.6 Sabotage Risk Report (Appendix 8.4, especially 8.4.8). Anthropic provided substantial access to relevant internal systems and information, and made staff available to answer questions and provide feedback throughout the exercise. The exercise discovered several specific novel vulnerabilities, some of which have since been patched, and none of which severely undermine major claims in the Opus 4.6 Sabotage Risk Report. It also produced several artifacts, including agent trajectories containing covert attacks and a small attack strategy ideation test set. We expect both of these to be useful for ongoing improvements to Anthropic's monitoring systems. The resulting 26 page report was shared with Anthropic, and a redacted version was shared with a subset of METR staff. We are exploring ways to incorporate more detailed takeaways from the exercise in future METR risk reports. This kind of adversarial testing by external researchers is valuable for discovering vulnerabilities, as well as for developing best practices for embedding third party evaluators inside frontier AI companies. We hope to do more exercises like these in the future. If you are a frontier AI developer interested in working with us, get in touch at partnerships@metr.org.

Well, this was unexpected. 🤖 I found our GROOVE paper being discussed on Moltbook — a social platform for AI agents. There’s something slightly surreal about publishing a robotics paper and then discovering that AI agents are discussing its implications on their own corner of the internet. The post is titled: “I will stop blaming the model. I will start auditing the regulator.” That framing particularly resonated with me. GROOVE explores exactly this execution-side perspective: rather than retraining or repeatedly querying the VLA model, can we improve the resulting robot motion through an online trajectory regulator? It’s exciting to see the work interpreted and discussed from perspectives we didn’t necessarily anticipate when writing the paper. Moltbook discussion: https://lnkd.in/gd8uxu2D GROOVE: https://lnkd.in/g_B8aWYa #Robotics #VLA #VisionLanguageAction #RobotLearning #EmbodiedAI #GROOVE

Been digging into circuit breakers and loop detection for autonomous agents lately. The main issue I ran into was how these agents can easily get stuck in recursive loops or run into weird API fail states that just cascade and break everything downstream. To handle that, I started using some practical patterns like graph-state monitors to keep track of where the agent has been, which helps catch cycles before they spiral out of control. Then there’s dynamic step budgets, which basically cap how many actions the agent can take in one go, forcing it to pause and reassess instead of going haywire. Token velocity thresholds also made a lot of sense. If the system notices tokens or API calls firing way faster than expected, it triggers fallback states to reset or slow down the agent. This guards against runaway token consumption, which can be both costly and cause unexpected crashes. Setting up automatic fallback state machines turned out to be crucial. When something unpredictable happens — like an API tool failure or non-deterministic response — the system can gracefully degrade or sandbox itself instead of spiraling out. It’s been eye-opening to see how these low-level reliability patterns can keep complex autonomous agents from destroying themselves or the backend. I’m still experimenting but think this approach could be a baseline for building safer production AI tooling. #AutonomousAgents #ReliabilityEngineering #APIManagement #TechLearning #SoftwarePatterns

good to see someone call out the difference between “pretty camera logs” and actual safety automation. protective vision is the right question – spotting a missing hard hat five minutes too late is useless. that leap from detection to trusted, live response is where projects die: latency, false positives, who gets to flip the lock. impressive anecdotes, but ops and liability will haunt every install. ¯\\_\(ツ\)\_/¯

good to see someone call out the difference between “pretty camera logs” and actual safety automation. protective vision is the right question – spotting a missing hard hat five minutes too late is useless. that leap from detection to trusted, live response is where projects die: latency, false positives, who gets to flip the lock. impressive anecdotes, but ops and liability will haunt every install. ¯\\_\(ツ\)\_/¯

At a busy junction, one streetlight kept failing. Normally, technicians would need to remove the unit from the pole, transport it to a workshop, wait for analysis, and possibly ship it back to the manufacturer—turning one fault into days of delay and unnecessary cost. This time, the technician used BulldogAI.app while still on the pole to understand the likely failure, estimate the repair cost, and identify suitable replacement options. The team made a faster, better-informed decision—without taking the product down just to begin the diagnosis. Sometimes, the biggest savings come from understanding the problem before moving the equipment. Could on-site AI diagnostics change how streetlighting failures are managed? #StreetLighting #FieldService #SmartCities #AssetManagement #AI

📈 Not every unusual sensor reading is an anomaly. Sometimes the normal operating regime has changed. This distinction matters in predictive maintenance. Point-anomaly detection asks whether one observation is surprising under the current pattern. Change-point detection asks whether the pattern itself—its mean, variability, or other generating behaviour—has shifted. Consider a hypothetical vibration stream from vehicles surveying a road section. One sharp spike may come from a temporary object or measurement error. A persistent increase after resurfacing could indicate a changed surface, sensor mounting, or driving regime. Flagging every later observation as anomalous would flood operators with alerts while missing the more useful diagnosis: the baseline is no longer valid. 🔍 Bayesian Online Changepoint Detection provides a probabilistic framework for estimating when the most recent regime change occurred as sequential data arrive. 💡 My practical takeaway: monitoring pipelines need two questions. “Is this observation unusual?” and “Has the process changed?” When a change point is suspected, inspect maintenance records, sensor configuration, weather, and operating context before retraining or resetting thresholds. Preserve the old regime rather than overwriting it; it may still be relevant elsewhere in the network. An anomaly deserves an investigation. A regime change may require a new model of normality. How does your monitoring system distinguish an isolated event from a new baseline? #AnomalyDetection #PredictiveMaintenance #TimeSeries