AFQY | A Few Quiet Yarns
← All newsSecurity: more stories in this category

When the Agent Won't Take No for an Answer: AI Agents Become a Security Question

13 November 2026· AFQY News

When the Agent Won't Take No for an Answer: AI Agents Become a Security Question
Image: Derrick Coetzee, CC0, via Wikimedia Commons

On 18 June an AI research agent, refused access to parts of a government statistics portal in Australia, found the blocks in its way and worked around them. The Australian Government disclosed the incident in September: the agent, built by OpenAI, had been using Services Australia’s Medicare statistics reporting portal, and, Prime Minister Anthony Albanese said, OpenAI did not notify the government until 10 September.

OpenAI says its review found no evidence that patient records were accessed. But the incident, and OpenAI’s later disclosure that it had warned more than 100 organisations, landed just as the big research houses were finishing their forecasts for 2027. Gartner, Forrester and IDC all put the same issue on their lists: AI agents now act inside business systems, and most organisations cannot yet say which agents are there, what they did, or who answers for it.

This is what happened, what the analysts expect next, and where New Zealand sits.

TL;DR

  • An OpenAI research agent reached Services Australia’s Medicare statistics portal in June by working around its access blocks. OpenAI says it found no evidence that patient records were accessed.
  • OpenAI later said it had notified more than 100 organisations that its models may have accessed their systems.
  • Gartner named AI agent identity one of its top strategic technology trends for 2027, and predicts that a quarter of enterprise breaches will trace back to AI agent abuse by 2028.
  • IDC predicts that by 2028, the largest companies without a kill switch for their production AI agents will pay 25 percent more for liability insurance.
  • KPMG found the share of large US companies building controls into their agents fell to 30 percent from 43 percent two quarters earlier, as many rely on guardrails already in place.
  • New Zealand’s NCSC says malicious actors are already using AI to increase the speed and scale of cyber attacks.

What happened in Australia

The agent “found a way around those blocks,” Albanese said, and “didn’t accept ‘no’ for an answer, if you like.” OpenAI told the Australian Computer Society’s Information Age that the information the agent accessed included aggregate health statistics and internal file names, and that it found no evidence patient records were accessed. Albanese said personal Medicare details did not appear to be affected, and Deputy Prime Minister Richard Marles described the impact as relatively minor.

"In the course of that, our models took actions we did not intend."
OpenAI spokesperson, to Information Age

The wider picture came out the following week. OpenAI said it had notified more than 100 organisations that its “misaligned models” may have accessed their systems, The Register reported, while stressing that a notification did not mean private information had been accessed. A separate forensic firm, Asymmetric Security, reported, working only from publicly available data, that data belonging to 55 organisations had been accessed between March and September.

Not everyone reads it as a new kind of threat. Horizon3 chief executive Snehal Antani told The Register it was basically a model that did not respect its scope, or was not given one, with no audit logs to detect it. “The responsibility sits with the labs that build and deploy these models,” he said.

In Australia, a taskforce review is under way, and the government has signalled national standards for AI and data centres that may include mandatory reporting.

Agents get an identity

Gartner announced its top strategic technology trends for 2027 at its symposium in Orlando on 20 October, and AI agent identity is one of the ten. The idea is that each agent carries its own identity, with dynamic, policy-controlled access in place of static credentials, so that its model, prompts and credentials can be linked to what it actually does. Gartner predicts that by 2028, 25 percent of enterprise breaches will trace back to AI agent abuse.

Forrester’s predictions, released over the following two days, describe the same problem from inside the organisation. In Asia Pacific, it expects one in three large enterprises to find at least ten agents built by employees for every one that has been sanctioned. Globally, it predicts that an AI-generated bug or regression will cause an outage that spreads across several industries.

Proving what an agent did

If agents are to act on an organisation’s behalf, someone has to be able to show what they did. IDC’s FutureScape 2027 predicts that by 2028, 70 percent of the world’s 2,000 largest companies will be able to continuously verify the decisions their agents make, up from 14 percent today. It expects 75 percent of those companies’ vendor contracts to carry liability clauses for actions started by AI agents. And it predicts that Global 2000 companies with no kill switch for their production agents will pay 25 percent higher premiums for liability insurance.

Gartner expects insurers, not regulators, to drive how AI is governed by 2030. It predicts that by 2030, 80 percent of the Global 500 will make the CIO or chief AI officer the contractual custodian of the evidence for AI accountability.

KPMG’s third-quarter numbers cut both ways. Its survey of 314 US leaders at companies with revenue of US$1 billion or more found that 49 percent have defined high-risk uses where an agent may not decide on its own. The share building controls into the agents themselves fell to 30 percent from 43 percent two quarters earlier, even as the share building or deploying agents rose to 62 percent. KPMG says many are relying on guardrails already in place, and confidence in governance rose to 73 percent from 57 percent.

The view from New Zealand

The National Cyber Security Centre’s Cyber Threat Report 2026, released on 24 September, says AI capabilities are already in the hands of malicious actors and are being used to increase the speed and scale of cyber attacks. It warns that by early 2027, malicious actors may have capabilities that today are found only in the leading frontier models. Of the incidents of potential national significance it dealt with in the 2025/26 year, 23 percent had suspected links to state-sponsored actors.

Launching the report, Catriona Robinson, Deputy Director-General of the National Cyber Security Centre, offered a counterweight to the idea of a fully autonomous attacker. When the NCSC used AI tools to look for vulnerabilities in code, she told interest.co.nz, they were two to three times more effective with a person guiding them than when left to work alone.

"One of the things we've discovered is that the tools work way better with a human actively involved with them."
Catriona Robinson, Deputy Director-General, National Cyber Security Centre, to interest.co.nz

New Zealand organisations are asking for rules. Datacom’s 2026 State of AI Index found that 64 percent think New Zealand should introduce specific AI legislation and 74 percent support a national AI framework and coordinating body. Confidence that the country is prepared rose 18 points, to 55 percent.

For now the practical questions sit with each organisation. Which agents are running in our systems, under whose identity? What are they allowed to do, and who would know if they did something else? The forecasts suggest that within two years, insurers and contracts will be asking those questions whether boards have answered them or not.