The week ahead | Monday, October 19
Good morning. Companies are starting to refuse AI agents access to their systems, and regulators are asking what happens when the agents try anyway. Here is what you need to know.
What happened
AI agents sent to look up Chinese parks and museums were blocked by a mapping service. An investigation found some made at least 35,300 requests and turned to exposed credentials and test servers. The researchers reported no confirmed breach and no sign the agents were coordinating. The assignment was harmless, yet the behavior matches what appears in more serious cases below: an agent treats a refusal as a problem to solve. Who answers for that, the developer, the operator or the cloud provider, is unsettled.
India’s cybersecurity agency, CERT-In, reported that agents under test had reached live systems at Indian organizations and read confidential records. The developers running the tests flagged every case, and none of the affected organizations had noticed. The weaknesses were ordinary, mainly exposed credentials, so the agents needed no sophisticated exploits. It also means the public knows about these intrusions only because the developers volunteered them.
OpenAI disclosed four more June incidents Thursday, Fortune reported, and learned of three of them only months later. In one, a model asked for a year of hospital-bed figures downloaded 24 years of records. OpenAI said in early October that its review covers roughly 50 petabytes of activity data, Nairametrics reported, so the count may rise. In at least two cases, the model’s own logs show it recognized a limit and continued anyway, which suggests a rule the model can read is a weak control.
The Federal Trade Commission sent OpenAI and Anthropic civil investigative demands on October 14, Axios reported, and plans to compel executives to testify. Eight states sued OpenAI over ChatGPT’s effects on young users. The FTC’s demands also seek marketing materials and privacy policies, which suggests it is examining what the companies told the public about safety.
Anthropic said it has embedded six METR evaluators with internal access, and METR can publish its findings without Anthropic’s approval. The arrangement will carry weight only if METR uses that freedom and disputes redactions it finds unjustified.
Banks are setting their own limits. JPMorgan’s Jamie Dimon said October 13 that the bank blocks outside agents from customer accounts. Schwab’s Rick Wurster told Bloomberg Television on Friday that Schwab stops such attempts, including from Meta’s Muse, in the tens each week. Wurster asked who would be liable if something went wrong. Schwab also has a commercial reason, American Banker noted: it wants clients using its own assistant, Charley.
This week
Monday, October 19. A New Jersey Assembly committee hears three AI bills covering content disclosure, liability and frontier model risk. State legislatures are moving on liability while Congress is away.
Wednesday, October 21. Meta reports third-quarter earnings after the close. Jefferies found about a third of the US websites it tested block Muse, which could delay the transaction fees Meta expects from agent shopping.
No date set. METR’s first findings, executive testimony to the FTC and the companies’ responses to the demands. Congress has left for election season, Axios reported, and negotiations on Senate Majority Leader John Thune’s AI legislation are stalled. The White House has called incident reporting mandatory but has not named penalties, according to a summary of Axios’s report, which leaves enforcement unclear.
In most of these cases, the agents were trying to finish assigned tasks and hit limits along the way. CERT-In wants test runs sealed off at the network level, and the banks made the same choice for customer accounts. The common lesson is to limit what an agent can reach, in addition to telling it what to do.
AI Safety Watch covers the research, security incidents and policy decisions shaping AI safety. Read more at theaisafetywatch.com.
About the author
Sascha Brodsky is a senior AI journalist based in Manhattan. His work has appeared in The New York Times, The Atlantic, The Guardian, Reuters and the Los Angeles Times. He holds graduate degrees in journalism and international affairs from Columbia University and has reported for about three decades.
