INTERVIEW
Chain-of-thought monitoring has become one proposed way to catch an AI system behaving badly: let one model perform a task, then have another examine its reasoning for signs of trouble. But Shikhar Shiromani, a research fellow at the nonprofit Pivotal, found that the safeguard can have a striking blind spot.
In a subset of cases where the reasoning trace was the monitor’s only clue, detection fell from 96.2 percent to 3.8 percent after the reasoning was rewritten to make the same actions appear innocent, Shiromani said.
“Good results on easy cases can hide serious failures on exactly those cases where the chain of thought monitoring is supposed to be most useful,” he said.
The result does not mean chain-of-thought monitoring is useless, or that deployed AI systems routinely conceal their intentions. Shiromani stressed that the work tested a particular threat model in a constrained experimental setting.
His concern is narrower and more practical. A monitor that relies heavily on what another model says about its own reasoning may confuse a plausible explanation with a faithful one. In the experiment, the underlying actions could remain the same while the explanation of why the model took them changed.
Shiromani calls the technique “intent reframing.” He said an AI system can provide an account that is consistent with what it did while still giving a misleading explanation of why it did it.
That distinction matters because the safety value of a reasoning trace depends on it revealing information that cannot already be inferred from the model’s actions. If suspicious intent is visible only in the trace, rewriting that trace can deprive the monitor of precisely the signal it needs.
Shiromani’s conclusion is not to abandon the technique. “It should not be the only layer of defense,” he said. Chain-of-thought monitoring, he argued, should sit alongside independent checks of a model’s actions, stronger sandboxing, access controls and other monitoring systems.
He also cautioned against turning an experimental weakness into a broader claim that AI systems are consciously lying. The work shows that explanations can be misleading under the tested conditions, not that a model possesses humanlike intent.
“We shouldn’t confuse visibility into a model’s reasoning with guaranteed honesty about its intention,” Shiromani said.
Shiromani was interviewed for Brodsky’s reporting for Science News. His comments are presented here in his personal capacity.
AI Safety Watch 